cannot hijack chunked or content length stream

[Alex-1357]

Hello, why this module https://github.com/dotnet/Docker.DotNet/blob/master/src/Docker.DotNet/Microsoft.Net.Http.Client/HttpConnectionResponseContent.cs said me - "cannot hijack chunked or content length stream".

This is my attempt to attach to container

Async Sub AttachToContainer(DockerHub As DockerClient, ContainerID As String, Cmd As IList(Of String), ExecCts As CancellationToken)
    Dim ExecTask = Await DockerHub.Exec.ExecCreateContainerAsync(ContainerID, New ContainerExecCreateParameters With {
                     .AttachStdin = False,
                     .AttachStderr = True,
                     .AttachStdout = True,
                     .Tty = False,
                     .Cmd = Cmd
                   },
                   ExecCts)
    Try
        Dim Stream = Await DockerHub.Exec.StartAndAttachContainerExecAsync(ExecTask.ID, False)
        Await Stream.CopyOutputToAsync(
                Nothing,
                Console.OpenStandardOutput(),
                Console.OpenStandardError(),
                CancellationToken.None)
    Catch ex As Exception
        Console.WriteLine(ex.Message)
    End Try
End Sub

I'm don't sure that this approach is right, I need to realize simple operation

# sudo docker exec -it 7dde487b4424 pwd

But receive this result

[wesley-pattison]

Hi, I'd like to also mention I am receiving this exception. My case is a bit different:

• My code is in net 6.0 C#
• I'm using Rancher Desktop with the dockerd (moby) option checked for Container Runtime, WSL Integration is also enabled.
• trying to leverage dotnet-testcontainers nuget package which underneath the hood, uses the Docker DotNet library.

I asked a colleague of mine who still had Docker Desktop installed, to try the example unit test they dotnet-testcontainers project supplies, and it works. Unfortunately using Docker Desktop for Windows is not an option due to licensing.

Some debugging information that may help:

This is the code in question that throws the exception.

        public WriteClosableStream HijackStream()
        {
            if (_responseStream != _connection.Transport)
            {
                throw new InvalidOperationException("cannot hijack chunked or content length stream");
            }

            return _connection.Transport;
        }

In my use case, _responseStream is marked as type Microsoft.Net.Http.Client.ChunkedReadStream, and _connection.Transport is marked as type Microsoft.Net.Http.Client.BufferedReadStream

[rossmasday]

I am also having the same issue using .net 6.0 C#

[joebone]

So I dug into this a little bit, and it seems the interaction with the Api when Rancher Desktop is being used always returns a ChunkedStream. If I drop the HTTP version from 1.1 to 1.0, it returns the BufferedReadStream, which seems to work fine...
I made the change in DockerClient.cs -> PrepareRequest()

//request.Version = new Version(1, 1);
request.Version = new Version(1, 0); // 1.0 doesnt allow chunking...

Are there any particular features needed from 1.1 or is this an ok workaround?

[joebone]

Hi, first time submitting a PR, no doubt I will need to revise it. In order to "solve" this issue without breaking everything else I have added the abiility to specify the Http Version in the Docker client configuration. If there are any issues or changes I need to make, please feel free to let me know - the PR is this one:
#566

[HofmeisterAn]

I've done a couple of tests and I guess the current implementation does not handle chunked streams (HTTP 1.1) very well. Looking into the Docker Engine API documentation, I assume chunked transfer encoding should be supported.

diff --git a/src/Docker.DotNet/DockerClient.cs b/src/Docker.DotNet/DockerClient.cs
index 30406dd..57fbcc1 100644
--- a/src/Docker.DotNet/DockerClient.cs
+++ b/src/Docker.DotNet/DockerClient.cs
@@ -333,6 +333,10 @@ namespace Docker.DotNet
                 throw new NotSupportedException("message handler does not support hijacked streams");
             }
 
+            var stream = await content.ReadAsStreamAsync()
+                .ConfigureAwait(false);
+            return (WriteClosableStream)stream;
+
             return content.HijackStream();
         }
 
diff --git a/src/Docker.DotNet/Microsoft.Net.Http.Client/ChunkedReadStream.cs b/src/Docker.DotNet/Microsoft.Net.Http.Client/ChunkedReadStream.cs
index 1076804..9dabcb2 100644
--- a/src/Docker.DotNet/Microsoft.Net.Http.Client/ChunkedReadStream.cs
+++ b/src/Docker.DotNet/Microsoft.Net.Http.Client/ChunkedReadStream.cs
@@ -6,7 +6,7 @@ using System.Threading.Tasks;
 
 namespace Microsoft.Net.Http.Client
 {
-    internal class ChunkedReadStream : Stream
+    internal class ChunkedReadStream : WriteClosableStream
     {
         private readonly BufferedReadStream _inner;
         private long _chunkBytesRemaining;
@@ -176,5 +176,13 @@ namespace Microsoft.Net.Http.Client
         {
             throw new NotSupportedException();
         }
+
+        public override bool CanCloseWrite 
+            => _inner.CanCloseWrite;
+
+        public override void CloseWrite()
+        {
+            _inner.CloseWrite();
+        }
     }
 }

With this patch, I was able to use the broken API calls like StartAndAttachContainerExecAsync incl. ReadOutputToEndAsync or AttachContainerAsync. This is probably not a proper fix yet. With some more information and guidance, I'm certain that I can create a pull request. The tests ran with Docker Desktop too.

[HofmeisterAn]

It's broken now for Docker Desktop for Windows (4.10.0) too. The patch above fixes the issue for 4.10.0 too.

[databrecht]

Fyi in case it helps, we have hit the same issue on a mac Monterey 12.4 environment. Docker was installed via docker desktop (version 4.10.0), we'll try to downgrade (edit: downgrading to 4.9.1 makes the error go away).

  System.AggregateException : One or more errors occurred. (cannot hijack chunked or content length stream) (cannot hijack chunked or content length stream)
---- System.InvalidOperationException : cannot hijack chunked or content length stream
---- System.InvalidOperationException : cannot hijack chunked or content length stream
 Stack Trace:
----- Inner Stack Trace #1 (System.InvalidOperationException) -----
  at [Microsoft.Net](http://microsoft.net/).Http.Client.HttpConnectionResponseContent.HijackStream()
  at Docker.DotNet.DockerClient.MakeRequestForHijackedStreamAsync(IEnumerable`1 errorHandlers, HttpMethod method, String path, IQueryString queryString, IRequestContent body, IDictionary`2 headers, TimeSpan timeout, CancellationToken cancellationToken)
  at Docker.DotNet.ContainerOperations.AttachContainerAsync(String id, Boolean tty, ContainerAttachParameters parameters, CancellationToken cancellationToken)
  at DotNet.Testcontainers.Clients.DockerContainerOperations.AttachAsync(String id, IOutputConsumer outputConsumer, CancellationToken ct)
  at DotNet.Testcontainers.Containers.Modules.TestcontainersContainer.Start(String id, CancellationToken ct)
  at DotNet.Testcontainers.Containers.Modules.TestcontainersContainer.StartAsync(CancellationToken ct)```

[HofmeisterAn]

@jstarks can you help here?

[Jejuni]

Getting this bug as well.

We're using a library that internally uses Docker.DotNet quite heavily for integration testing.
Luckily we didn't update docker on our Build Server, but none of our developers can run integration tests on their machines anymore.

Hoping for a quick fix.

[galvesribeiro]

Hey folks! The fix is in and released here: https://www.nuget.org/packages/Docker.DotNet/3.125.10

[JonasBenz]

@HofmeisterAn @galvesribeiro Thanks for the fix 👍

It also solved the issue we run into in our case.
Unfortunately we now have an other issue, when we try to write to the returned stream, then we get an System.NotSupportedException: 'Specified method is not supported'.

In our use-case we try to execute some bash commands and want to read the output of the commands as well.
This is how the implementation looks like, which works fine with Docker Desktop 4.9.1, but fails with version 4.10.1 and Docker.DotNet 3.125.10:

Like mentioned this works with Docker Desktop 4.9.1. A difference which I noticed is that the property CanWrite of the inner _stream is set to false with Docker Desktop 4.10.1, but it was set to true with Docker Desktop 4.9.1:

Should I create a new Issue for this case? Or do you think it is related?

[galvesribeiro]

Yeah please create a separated issue and share it there. I'm going to close this as the PR was merged and the original issue is fixed. Thanks!

[HofmeisterAn]

Should I create a new Issue for this case? Or do you think it is related?

I would prefer a new issue.

Did you check the _inner stream? ChunkedReadStream always returns false. Probably we just need to change the type and delegate CanWrite:

Docker.DotNet/src/Docker.DotNet/Microsoft.Net.Http.Client/ChunkedReadStream.cs

Lines 36 to 39 in f587486

	public override bool CanWrite
	{
	get { return false; }
	}