Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming response bodies from an HttpClient in UWP in debug builds will hang on large messages. #108622

Closed
kinyoklion opened this issue Oct 7, 2024 · 15 comments
Labels
area-System.Net.Http question Answer questions and provide assistance, not an issue with source code or documentation.

Comments

@kinyoklion
Copy link

kinyoklion commented Oct 7, 2024

Description

When reading from a response body the stream will have available data, but the read will not complete. Instead the read completes when subsequent data is received. This only happens with a debug build in UWP. Any non-UWP dotnet platform works correct and a release UWP application also works correctly (seems to sometimes work in release).

This does not happen for short messages, at some size the behavior starts to exhibit itself.

Using Microsoft.NETCore.UniversalWindowsPlatform version 6.2.12
Targeting Windows 10, version 2004 (10.0; Build 19041)
Min Version Windows 10, version 1809 (10.0; Build 17763)

Reproduction Steps

First you need an SSE server this example server will work.

var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

Random random = new Random();

string RandomString(int length)
{
    const string chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
    return new string(Enumerable.Repeat(chars, length)
        .Select(s => s[random.Next(s.Length)]).ToArray());
}

app.MapGet("/", async (CancellationToken ct, HttpContext ctx) =>
{
    ctx.Response.Headers.Add("Content-Type", "text/event-stream");

    while (!ct.IsCancellationRequested)
    {
        await ctx.Response.WriteAsync($"event:put");
        await ctx.Response.WriteAsync($"\n");
        await ctx.Response.WriteAsync($"data: ");
        await ctx.Response.WriteAsync($"{{ \"value\":\"{RandomString(100000)}\"}}");
        await ctx.Response.WriteAsync($"\n\n");
        await ctx.Response.Body.FlushAsync();
        await Task.Delay(180000);
        await ctx.Response.WriteAsync($":");
        await ctx.Response.WriteAsync($"\n\n");
    }
});

app.Run();

When reading this response you would expect an event:put followed by data:{"value":"<random data>"}.

When using UWP you will indead not get the data field for 180 seconds.

Code to put in a UWP app.

var httpClient = new HttpClient();
var uri = new Uri("http://localhost:5199"); // Change port to that of the server.
var request = httpClient.GetStreamAsync(uri);

using (var reader = new StreamReader(request.Result))
{
    while (!reader.EndOfStream)
    {
        var read = reader.ReadLine();
        Debug.WriteLine(read);
    }
}

This may not be the correct way to do this, but the same behavior is exhibited with any of the async read methods or the synchronous read methods, aside from ReadByte.

Expected behavior

The second line of the sent message should be received shortly after being sent. To experience correct behavior you can reduce the size of the sent message from the example application. Using 1000 random characters, instead of 100000, will work correctly.

Actual behavior

The second line of the message is not received for 3 minutes.

Regression?

No response

Known Workarounds

Synchronously reading single bytes from the stream will receive every byte as it is available. This isn't very practical though.

Configuration

No response

Other information

No response

@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Oct 7, 2024
@ManickaP
Copy link
Member

ManickaP commented Oct 8, 2024

Can you do a packet capture for the problematic case?
I'm not an expert on UI applications, but if you're doing sync-over-async in a UI thread, that might lead to deadlocks.

@ManickaP ManickaP added question Answer questions and provide assistance, not an issue with source code or documentation. needs-author-action An issue or pull request that requires more info or actions from the author. labels Oct 8, 2024
@jkotas
Copy link
Member

jkotas commented Oct 8, 2024

For reference, the reported issue is in the UWP-specific implementation of HttpClient https://github.com/dotnet/corefx/tree/release/uwp6.2/src/System.Net.Http/src/uap . This implementation forwards to UWP-specific http stack that is part of Windows.

@kinyoklion You may want to try updating your app to use https://devblogs.microsoft.com/ifdef-windows/preview-uwp-support-for-dotnet-9-native-aot/

@kinyoklion
Copy link
Author

kinyoklion commented Oct 8, 2024

@ManickaP I did synchronous because it was simplest for an example, but all the async methods encounter the same problem.

The issue was originally encountered using one of our (LaunchDarkly) event source libraries : https://github.com/launchdarkly/dotnet-eventsource/blob/42cd339937a4affdbe94ab0d49694a86eb85c75f/src/LaunchDarkly.EventSource/EventSourceService.cs#L177

(I also tried a modified version of this that removed all the cancellation, timeouts, and other wrapping.)

Also:
https://github.com/launchdarkly/dotnet-eventsource/blob/42cd339937a4affdbe94ab0d49694a86eb85c75f/src/LaunchDarkly.EventSource/EventSourceService.cs#L155

I also made a thread that just ran the test code and it experiences the same issue.

I can see if I can figure out a windows packet capture.

@dotnet-policy-service dotnet-policy-service bot removed the needs-author-action An issue or pull request that requires more info or actions from the author. label Oct 8, 2024
@kinyoklion
Copy link
Author

@jkotas Thank you for the suggestion. I can see what happens when using the .Net 9 UWP AOT preview.

@kinyoklion
Copy link
Author

@ManickaP I've attached a capture.

reprocap.zip

I took a screenshot to add some context. Here we have the second part of the message has been received and it ends with a newline. We got the message with the event:put and then 2 packets containing the JSON payload.
image

Later we get our "heartbeat" and then the other content is read as well. If I were to run the same code in a dotnet console app, for instance, then I would get the the line read immediately when the selected packet was received.

@jkotas I tried to configure the .Net 9 preview, but encountered problems. I couldn't create an empty UWP app. It failed during project creation and created an empty solution. I was able to create an empty app with windows application packaging (I did have to edit some things to get it building.) Unfortunately when I ran the empty project it immediately exits with 3221226505. I tried on two different computers one with windows 10 and the other with windows 11. (With the latest preview visual studio, the vsix with project templates, a nightly .net 9 build, and the couple dll/targets files replaced with those from the article.)

@kinyoklion
Copy link
Author

As an additional data point it seems to work fine in a WinUI 3 application.

@jkotas
Copy link
Member

jkotas commented Oct 8, 2024

As an additional data point it seems to work fine in a WinUI 3 application.

WinUI 3 applications use the standard (non-UWP) networking stack, so it is expected that it works fine given that you have verified earlier that the standard networking stack does not have this issue.

@jkotas I tried to configure the .Net 9 preview, but encountered problems.

cc @Sergio0694

@Sergio0694
Copy link
Contributor

"It failed during project creation and created an empty solution."

That's expected, I mention it in the blog post. There's a bug in the wizard for UWP projects that I've fixed in VS 17.12 P3 (should be out shortly). For now, a workaround for that is to just create a new solution first (you can easily do so by creating any other project), and then once you're there, create a new UWP project. When you get the error dialog, just close it. The new project will still be created correctly, and it will show up in the solution. Note that there's a few caveats and preview tooling required for UWP on .NET 9 still, so if you'd like to try it out I strongly recommend following the instructions in the blog post carefully. It should be much easier in future updates but for now the tooling is still in early preview, so it's a bit rough around the edges 🙂

@kinyoklion
Copy link
Author

kinyoklion commented Oct 9, 2024

"It failed during project creation and created an empty solution."

That's expected, I mention it in the blog post. There's a bug in the wizard for UWP projects that I've fixed in VS 17.12 P3 (should be out shortly). For now, a workaround for that is to just create a new solution first (you can easily do so by creating any other project), and then once you're there, create a new UWP project. When you get the error dialog, just close it. The new project will still be created correctly, and it will show up in the solution. Note that there's a few caveats and preview tooling required for UWP on .NET 9 still, so if you'd like to try it out I strongly recommend following the instructions in the blog post carefully. It should be much easier in future updates but for now the tooling is still in early preview, so it's a bit rough around the edges 🙂
@Sergio0694

Thank you for pointing that out. I missed the tip box. I created a new one as prescribed here. I continue to encounter issues though. So if this is going to be available in a more refined way in the not-so distant future, then I will be waiting for that. I edited the targets file from the SDK to just remove the thing it was complaining about and it built and loaded. I can test if it works with my sample code tomorrow.

Before editing the targets file I was getting:
image

I did replace the targets and dll in the with the preview versions.

Thanks for the help.

@ManickaP
Copy link
Member

ManickaP commented Oct 9, 2024

Ok, so I did a bit of digging and the response stream gets converted into .NET stream here:
https://github.com/dotnet/corefx/blob/1814a950a4ed69da659d34ed5329e2eabd59b3cc/src/System.Net.Http/src/uap/System/Net/HttpHandlerToFilter.cs#L463-L473

And based on this docs https://learn.microsoft.com/en-us/windows/uwp/debug-test-perf/optimize-file-access#buffering-between-uwp-and-net-streams, the adapter in between employs its own buffering, which might explain the delay.

However, I don't personally think this is a wrong behavior per se. You're using streaming response content, the data might be split and written in different packets than your actual calls to WriteAsync. This also applies to the reading logic. And of course, there is buffering on different layers of the whole stack. If you've never received the data, that would concern me.

Either way, if you wish to still pursue reporting this, you'll have to contact the Windows team via official support channel. In this case, .NET is just a thin layer above UWP and we do not own that.

@kinyoklion
Copy link
Author

kinyoklion commented Oct 9, 2024

@ManickaP I think I didn't articulate correctly if this can be perceived as not a bug. The delay is 3 minutes because I sent more data after 3 minutes which seems to "wake" the stream up. If I don't send more data, then I never get that original data, or at least the delay would be far greater than 3 minutes.

This means that you cannot use the stream for implementations like SSE, which is where this bug was brought to my attention. We send a message with SSE, then we send heartbeats to keep the connection alive. Depending on the size of the payload you may not get the initial payload until the hearbeat.

If this was like 30ms delay that was intrinsic to the operation, or maybe even 500ms and combined the messages, that is different. This is that the stream has hung and never emits until it gets additional data.

I am in no way concerned with how the data is grouped. It could all be in one, it could split, that is all normal. The indefinite hang after what seems to be a message spanning packets, is not normal.

@kinyoklion
Copy link
Author

@jkotas It seems that things operate as expected when using preview UWP support for .NET 9. At least I have not gotten it to exhibit the behavior in a number of tries. (The existing one would sometimes intermittently succeed, but never consistently.)

@Sergio0694
Copy link
Contributor

Worth noting that UWP on .NET 9 just uses the normal BCL from .NET 9, it doesn't have anything UWP specific. If you wanted to confirm this is entirely just due to the underlying behavior of the WinRT web APIs, you might want to test this by using Windows.Web.Http.* directly. It's possible the issue doesn't actually have anything to do with UWP at all, but it's just a characteristic of those WinRT APIs.

@kinyoklion
Copy link
Author

Worth noting that UWP on .NET 9 just uses the normal BCL from .NET 9, it doesn't have anything UWP specific. If you wanted to confirm this is entirely just due to the underlying behavior of the WinRT web APIs, you might want to test this by using Windows.Web.Http.* directly. It's possible the issue doesn't actually have anything to do with UWP at all, but it's just a characteristic of those WinRT APIs.

Yeah, I was curious about this as well. Reading through the implementation code that was linked it doesn't appear there is anything beyond the interactions with the WinRT APIs.

This issue was brought to my attentions by a customer using a LaunchDarkly .Net SDK in UWP. I think that UWP would be the only platform that would be backing these APIs with WinRT implementations. So, I can instruct people not to use our SDK with UWP at the moment, or to configure it where it doesn't use streaming (we support polling as well and normal http responses with fixed sizes seem to work fine).

I may try with WinRT directly if I can find some bandwidth to do so.

@ManickaP
Copy link
Member

I'll close this issue for now. From what I can see here, this is not an issue with our networking stack. Feel free to pursue this with the Windows support.

@dotnet-policy-service dotnet-policy-service bot removed the untriaged New issue has not been triaged by the area owner label Oct 15, 2024
@github-actions github-actions bot locked and limited conversation to collaborators Nov 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Http question Answer questions and provide assistance, not an issue with source code or documentation.
Projects
None yet
Development

No branches or pull requests

4 participants