Internal Host DNS cannot be resolved (Windows Container 1803)

[n-junge]

Expected behavior

According to the Docs host.docker.internal resolves the host's ip.

Actual behavior

Ping request could not find host host.docker.internal
Network works fine. Pinging Host IP directly (ipconfig) works as expected.

Host: Windows 10 Enterprise
Container: microsoft/windowsservercore:latest

Neither stable (18.03) nor edge (18.04) works.

Probably important side note: DNS can be resolved in Linux Container (Ubuntu)

 PS C:\> ping host.docker.internal
 Ping request could not find host host.docker.internal. Please check the name and try again.
 PS C:\> ping gateway.docker.internal
 Ping request could not find host gateway.docker.internal. Please check the name and try again.
 PS C:\> ping google.com
 
 Pinging google.com [172.217.16.78] with 32 bytes of data:
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 Reply from 172.217.16.78: bytes=32 time=31ms TTL=46
 
 Ping statistics for 172.217.16.78:
     Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
 Approximate round trip times in milli-seconds:
     Minimum = 31ms, Maximum = 31ms, Average = 31ms

C:\WINDOWS\system32>docker version
Client:
 Version:       18.03.0-ce
 API version:   1.37
 Go version:    go1.9.4
 Git commit:    0520e24
 Built: Wed Mar 21 23:06:28 2018
 OS/Arch:       windows/amd64
 Experimental:  false
 Orchestrator:  swarm

Server:
 Engine:
  Version:      18.03.0-ce
  API version:  1.37 (minimum version 1.24)
  Go version:   go1.9.4
  Git commit:   0520e24
  Built:        Wed Mar 21 23:21:06 2018
  OS/Arch:      windows/amd64
  Experimental: false

[jasonbivins]

Hi @4xiz
Thanks for reporting - I can repro and have reported it as a bug

[ebriney]

It's a windows daemon limitation and needs to be fix there but we will add a workaround waiting MS to fix it upstream.

[n-junge]

Release 18.05.0-ce-win66 works. Thanks.

[atomaras]

Tested it on Windows 10, Docker 18.05.0-ce-win66 (17760) and it doesn't work

[n-junge]

@atomaras
Well the dns resolution seems to work for you too. So that's a different problem. I propose you open another issue.

[JoshCollinsMSFT]

I just tried this and it isn't working for me. Version info:

Windows
Microsoft Windows [Version 10.0.17134.112]

Docker

Client:
 Version:      18.05.0-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   f150324
 Built:        Wed May  9 22:12:05 2018
 OS/Arch:      windows/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.05.0-ce
  API version:  1.37 (minimum version 1.24)
  Go version:   go1.10.1
  Git commit:   f150324
  Built:        Wed May  9 22:29:00 2018
  OS/Arch:      windows/amd64
  Experimental: false

Ping output:

C:\>docker exec b61746027bc9 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

[cpumanaz]

@JoshCollinsMSFT I have the same windows build (17134.112) and docker (18.05.0-ce) and am also not able to resolve host.docker.internal. No joy with connection strings.

[prochnowc]

Still not working for me with 18.06.0-ce:

docker run --rm -i microsoft/nanoserver:1709 ping host.docker.internal
Ping request could not find host host.docker.internal. Please check the name and try again.

[Heurazio]

Same problem. docker.host.internal not working.

[prochnowc]

@n-junge @ebriney could you please re-open this issue?

[kierzo]

So it seems the way this was fixed is that it must write out to the local hosts file on the host machine.

I have a bit of a situation as Symantec Endpoint blocks write access to the hosts file :(.
Is there a better dns way of doing this?
Thanks

[ebriney]

They are 2 separate things.

First, for dockerd to get host.docker.internal resolution we must put it in windows hosts file...
So if you have endpoint protection or similar you must add an exception on com.docker.service executable.

Second, in the windows containers, we do a docker exec as soon as the container is started (using docker event) to patch the hosts file in the container (I don't know what happened if you are running in process isolation, because we don't support Windows server). So if you overwrite it at runtime you can lose those entries.

@Heurazio this is host.docker.internal, not docker.host.internal. you also have gateway.docker.internal but that is the same ip in windows containers.

@prochnowc can you try docker run microsoft/nanoserver powershell "Start-Sleep -s 2 ; if (! $(IPCONFIG /displayDNS | Select-String -Quiet -SimpleMatch host.docker.internal)) { exit 1 }"

[prochnowc]

@ebriney Interestingly ... if i wait 2 seconds after startup i can ping host.docker.internal. I will try this again within my traefik for windows container. traefik needs the gateway address.

[prochnowc]

Unfortunately it does not work.. traefik_1 | time="2018-08-25T01:30:05+02:00" level=error msg="Failed to retrieve information of the docker client and server host: error during connect: Get http://host.docker.internal:2375/v1.24/version: dial tcp: lookup host.docker.internal: getaddrinfow: This is usually a temporary error during hostname resolution and means that the local server did not receive a response from an authoritative server."

Does the host.docker.internal stuff require minimum nanoserver version?

[ebriney]

@prochnowc, we execute that script as soon as we receive the start event from any windows containers:

$hostsFile = "C:\Windows\System32\drivers\etc\hosts"
$src = [System.IO.File]::ReadAllLines($hostsFile)
$a = $src += ""
For ($i=0; $i -le $a.length; $i++) {
	if ($a[$i].Contains("host.docker.internal"))
	{
		$a[$i] = "%s host.docker.internal"
		$a[$i+1] = "%s gateway.docker.internal"
		[System.IO.File]::WriteAllLines($hostsFile, [string[]]$a)
		exit
	}
}
$a = $a += "# Added by Docker for Windows"
$a = $a += "%s host.docker.internal"
$a = $a += "%s gateway.docker.internal"
$a = $a += "# End of section"
[System.IO.File]::WriteAllLines($hostsFile, [string[]]$a)

Perhaps the hosts file doesn't exist and there is an exception reading it.

[prochnowc]

@ebriney Am I right when saying that this requires working PowerShell in the windows container? My traefik container does not have PowerShell.

[ebriney]

@prochnowc Arfff, that's why it didn't work. Do you have a suggestion on how to solve that?

[prochnowc]

@ebriney So actually this fix only works on windows containers which have PowerShell installed (which is not the case for windows nanoserver). I dont think the hosts file editing can be done with "standard" windows command line utils. My idea would be to implement this in the docker DNS.

[ebriney]

@prochnowc hummm, nanoserver includes powershell (see command line above).
And yes your image must include powershell for now.
Microsoft is managing all the network part on WCOW so except if they handle it upstream we cannot do anything (but to do that I think it must be generalized in moby/moby).
On Linux containers the vm traffic is going throught vpnkit so we can tweak dns requests ...

[prochnowc]

@ebriney Sorry, but nanoserver does not include PowerShell by default. See https://docs.microsoft.com/en-us/windows-server/get-started/nano-in-semi-annual-channel. Your command line didnt work with pure nanoserver image. I had to use mcr.microsoft.com/powershell:6.0.2-nanoserver.

[ebriney]

and so why docker run microsoft/nanoserver powershell "Start-Sleep -s 2 ; if (! $(IPCONFIG /displayDNS | Select-String -Quiet -SimpleMatch host.docker.internal)) { exit 1 }" is working?

[ebriney]

ok that's in 1803 image

[kierzo]

@ebriney , Would you be able to update your powershell script slightly to only write to the host file if needed?

I can see you do an if string contains "host.docker.internal" but then also add the comment strings to the file after the check by the looks of it.

Because of this second write to the file outside of this condition, this one is causing me a bit of pain because I've had my I.T update my hosts file manually to fix this, but because of this second write statement that happens unconditionally, when I click switch to Windows containers, there is a "can't write to hosts file error" still and crashes Docker.

Also in my case the wrong host ip was getting selected too as we have multiple network adaptors.

A suggestion if you could make if possible, would be to first check the DNS resolves first, if it doesn't then check if the hosts file has the additions it needs, and write to it if its not in there, also would move the second write in to the if statement.

Hope this makes sense.

I've coded and tested my suggestions above (W10) in the below, so feel free to use parts or all of it if needed:
Really appreciate any help as ping ponging with my IT dept here on this which is painful.

$hostsFile = "${env:windir}\system32\drivers\etc\hosts"
# Check DNS resolves before touching the hosts file.
if((Resolve-DnsName -Name host.docker.internal) -And (Resolve-DnsName -Name gateway.docker.internal)){}
else {
    # Check if in the hosts file and write it in if its not there...
    if((cat $hostsFile | Select-String -Pattern "host.docker.internal") -And (cat $hostsFile | Select-String -Pattern "gateway.docker.internal")){}
    else {
        $ip = (Resolve-DNSName (Hostname) | Select IPAddress | select IPAddress -expandproperty IPAddress | select -last 1)
        $addtohost = "`n#Added by Docker for Windows`n$ip    host.docker.internal`n$ip    gateway.docker.internal`n# End of section`n"
        echo $addtohost | Out-File -Append -filepath $hostsFile
    }
}

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[tihomir-kit]

/remove-lifecycle stale

[keystroke]

Still getting this issue, my containers (same images) randomly fail on startup. sometimes they start, sometimes they hang in "created" state and application log filled with repeated errors like below. I am running windows containers on windows 10 surface book laptop, calling docker run from PowerShell ISE elevated (running as admin) and still hit this issue at least 50% of the time, without changing anything in my images:

exec's CreateProcess() failed [module=libcontainerd namespace=moby container=a06ed0f5c4f8b94c5462d4d64c5b370e4db7cc95422dda325f1fc44abf186a6c exec=a6ccdf64ab7e3b48360dc82630cbb028263e1232ace044f1285d2b71e3a489da error=container a06ed0f5c4f8b94c5462d4d64c5b370e4db7cc95422dda325f1fc44abf186a6c encountered an error during hcsshim::System::CreateProcess: failure in a Windows system call: The user name or password is incorrect. (0x52e)
[Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
[Event Detail:  Provider: 00000000-0000-0000-0000-000000000000]
[Event Detail: onecore\vm\compute\management\orchestration\vmhostedcontainer\processmanagement.cpp(173)\vmcomputeagent.exe!00007FF7E2119D2B: (caller: 00007FF7E20CE13A) Exception(2) tid(4d4) 8007052E The user name or password is incorrect.
    CallContext:[\Bridge_ProcessMessage\VmHostedContainer_ExecuteProcess] 
 Provider: 00000000-0000-0000-0000-000000000000] extra info: {"CommandLine":"cmd.exe /C \"ECHO 192.168.0.117    host.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts \u0026 ECHO 192.168.0.117    gateway.docker.internal \u003e\u003e %systemroot%\\system32\\drivers\\etc\\hosts\"","User":"Administrator","WorkingDirectory":"/","CreateStdInPipe":true,"CreateStdOutPipe":true,"CreateStdErrPipe":true,"ConsoleSize":[0,0]}]

[brianharwell]

Is anyone looking into the root cause of this?

[BogusDude99]

/remove-lifecycle stale

[gabrielpagu]

Docker 3.3.3 Same issue

[DeoluA]

Updated to Docker Desktop 3.3.3 a few days ago, and this issue seems to have crop up again (hostname gateway.docker.internal was resolving fine before then). Perhaps something broke in this latest update?

[netpoetica]

I am on Docker 3.3.3 and I also am seeing this issue - anyone know a specific version of Docker where it actually did work?

[KyronSr]

I am on "Docker version 20.10.6, build 370c289" and have the same problem. I've used @brunnotelma 's workaround successfully, thanks for that!

[netpoetica]

If I use https://docs.docker.com/docker-for-windows/release-notes/#docker-desktop-community-2501 with docker.for.win.localhost or host.docker.internal I see the same issues - @brunnotelma's workaround did not fix this issue with Docker 3.x either. I am tracking this issue with reproducible images here: envoyproxy/envoy#16710 in case it is of interest to the Docker team

[pthivierge]

I found a workaround.
Had the same issue w mcr.microsoft.com/windows/servercore:ltsc2019 image and and docker engine 20.10.6.
host.docker.internal would not resolve from the container during build, tested with ping.
I could get a ping back after passing the docker build --network "Default Switch" .
Maybe this can help someone.

[warrenboucher]

I found a workaround.
Had the same issue w mcr.microsoft.com/windows/servercore:ltsc2019 image and and docker engine 20.10.6.
host.docker.internal would not resolve from the container during build, tested with ping.
I could get a ping back after passing the docker build --network "Default Switch" .
Maybe this can help someone.

Thank You!

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[tihomir-kit]

/lifecycle frozen

[tihomir-kit]

/remove-lifecycle stale

[supanadit]

/remove-lifecycle stale

[hasibr]

I have some Windows containers that were resolving host.docker.internal and gateway.docker.internal, along with some other Windows containers that weren't resolving those names, all in the same docker-compose stack.

After the last update, none of them are working anymore for some reason, so the workaround I did (based on some comments in this same issue) is this:

Dockerfile:

ENTRYPOINT ["powershell", "/docker-entrypoint.ps1;"]

Entrypoint script:

# DNS Entries
Write-Host("Configuring DNS entries...")
C:\fix-hosts.ps1

fix-hosts.ps1:

$hostsFile = "C:\Windows\System32\drivers\etc\hosts"

try {
    $DnsEntries = @("host.docker.internal", "gateway.docker.internal")
    # Tries resolving names for Docker
    foreach ($Entry in $DnsEntries) {
        # If any of the names are not resolved, throws an exception
        Resolve-DnsName -Name $Entry -ErrorAction Stop
    }

    # If it passes, means that DNS is already configured
    Write-Host("DNS settings are already configured.")
} catch {
    # Gets the gateway IP address, that is the Host's IP address in the Docker network
    $ip = (ipconfig | where-object {$_ -match "Default Gateway"} | foreach-object{$_.Split(":")[1]}).Trim()
    # Read the current content from Hosts file
    $src = [System.IO.File]::ReadAllLines($hostsFile)
    # Add the a new line after the content
    $lines = $src += ""
    
    # Check the hosts file and write it in if its not there...
    if((cat $hostsFile | Select-String -Pattern "host.docker.internal") -And (cat $hostsFile | Select-String -Pattern "gateway.docker.internal")) {
        For ($i=0; $i -le $lines.length; $i++) {
            if ($lines[$i].Contains("host.docker.internal"))
            {
                $lines[$i] = ("{0} host.docker.internal" -f $ip)
                $lines[$i+1] = ("{0} gateway.docker.internal" -f $ip)
                break
            }
        }
    } else {
        $lines = $lines += "# Added by Docker for Windows"
        $lines = $lines += ("{0} host.docker.internal" -f $ip)
        $lines = $lines += ("{0} gateway.docker.internal" -f $ip)
        $lines = $lines += "# End of section"
    }
    # Writes the new content to the Hosts file
    [System.IO.File]::WriteAllLines($hostsFile, [string[]]$lines)
    
    Write-Host("New DNS settings written successfully.")
}

When the container starts, before executing the main command it starts the fix-hosts.ps1 script.

What the script does is:

1. Checks if the names can be resolved.

   • If they can, assumes it's configured properly.

2. If not, looks up in the hosts file if there are entries there already.

   • If there is, replace them with the gateway IP address, which is the host's IP address.

3. If not, just adds the new entries for both gateway.docker.internal and host.docker.internal in the hosts file.

4. Write the new content to the hosts file.

5. The end.

That's it! It's the neatest approach I could get so far without having to hack it too much, and it works with pretty much any Windows image.

@brunnotelma apologies since this is an old comment, but would you be able to let me know which image you were able to get this script to work on?

I'm trying to run your startup script on a mcr.microsoft.com/powershell:lts-nanoserver-20h2 image. But the ContainerUser does not have permissions to write to the hosts file. I've tried this with a mcr.microsoft.com/windows/servercore:20H2 image as well.

Configuring DNS entries...
MethodInvocationException: C:\fix-hosts.ps1:51
Line |
  51 |      [System.IO.File]::WriteAllLines($hostsFile, [string[]]$lines)
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Exception calling "WriteAllLines" with "2" argument(s): "Access to the path 'C:\Windows\System32\drivers\etc\hosts' is denied."

This is the gist of my Dockerfile (some irrelevant details removed):

FROM mcr.microsoft.com/powershell:lts-nanoserver-20h2 AS base
ENV JAVA_HOME="$ProgramFiles\Java"
USER ContainerAdministrator
RUN setx /M PATH "%PATH%;%JAVA_HOME%\\bin"
USER ContainerUser
COPY ["./startup-script.ps1", "startup-script.ps1"]
COPY ["./fix-hosts.ps1", "fix-hosts.ps1"]
ENTRYPOINT ["pwsh", "startup-script.ps1"]

I've tried granting the ContainerUser full access to that directory, but I'm unable to as I get AccessDenied during this stage of the Docker build:

USER ContainerAdministrator
RUN icacls "C:\Windows\System32\drivers\etc" /grant "User Manager\ContainerUser":(OI)(CI)M /t
USER ContainerUser

My Docker engine is v20.10.14.

[brunnotelma]

Hi @hasibr, as far as I remember, the default user for Windows images is always "ContainerAdministrator".

I have many containers running this startup script off of these base images:

• mcr.microsoft.com/windows/servercore:ltsc2019
• mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-ltsc2019
• mcr.microsoft.com/dotnet/framework/aspnet:4.8-windowsservercore-ltsc2019

Just wondering: why do you need to change your user to ContainerUser?

[hasibr]

Hi @hasibr, as far as I remember, the default user for Windows images is always "ContainerAdministrator".

I have many containers running this startup script off of these base images:

• mcr.microsoft.com/windows/servercore:ltsc2019
• mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-ltsc2019
• mcr.microsoft.com/dotnet/framework/aspnet:4.8-windowsservercore-ltsc2019

Just wondering: why do you need to change your user to ContainerUser?

Hi @brunnotelma , thanks for the tip. I changed to the ContainerAdministrator user and I was able to run the script to configure the DNS entries on a mcr.microsoft.com/windows/servercore:20H2 image.

However, I'm still unable to ping host.docker.internal or google.ca, as it times out.

PS C:\> powershell .\startup-script.ps1                    
Configuring DNS entries...                                 
New DNS settings written successfully.                     
PS C:\> ping host.docker.internal                          
                                                           
Pinging host.docker.internal [172.30.144.1] with 32 bytes o
Request timed out.                                         
                                                           
Ping statistics for 172.30.144.1:                          
    Packets: Sent = 1, Received = 0, Lost = 1 (100% loss), 
Control-C                                                  

Also the reason I used ContainerUser was because I found this note on Microsoft's Windows Nanoserver image that mentions:
Comment link

Note that changing user on nanoserver is not recommended
See, https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/container-base-images#base-image-differences
But we are working around a bug introduced in the nanoserver image introduced in 1809

I suppose there is no issue using ContainerAdministrator on the servercore image though, so this should not be relevant here.