ACS Kubernetes failing to pull windows container image

[alexxie]

Is this a request for help?:

yes

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
Kubernetes

What happened:
Failing to pull windows container image.

• Kubernetes dashboard show the following errors
  -- Error syncing pod
  -- Failed to pull image "winpoc.azurecr.io/rolemanager:v1": rpc error: code = 2 desc = failed to register layer: re-exec error: exit status 1: output: time="2018-01-23T05:26:56Z" level=error msg="hcsshim::ImportLayer failed in Win32: The system cannot find the file specified. (0x2) layerId=\\?\C:\ProgramData\docker\windowsfilter\c62c8555ff2d4efca2a9ddbde457a17efc4951fd1bf1dbe4b46c88054e515566 flavour=1 folder=C:\Windows\TEMP\hcs339069859" hcsshim::ImportLayer failed in Win32: The system cannot find the file specified. (0x2) layerId=\?\C:\ProgramData\docker\windowsfilter\c62c8555ff2d4efca2a9ddbde457a17efc4951fd1bf1dbe4b46

What you expected to happen:
Pull windows container image and start the pod

How to reproduce it (as minimally and precisely as possible):

• The image is a ASP.NET (full) application built for Windows Container on Windows 10 laptop.
• The image is pushed to Azure Container Image registry. tag: winpoc.azurecr.io/app1:v1
• Deploy with KubeCtl create -f deployoment.yaml:

apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: asp-windows
spec:
replicas: 1
strategy:
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
template:
metadata:
labels:
app: app1
spec:
containers:
- name: app1
image: winpoc.azurecr.io/app1:v1

Anything else we need to know:

Also tried pulling the same image from Docker Hub, but getting timeout error.

[andyzhangx]

@PatrickLang

[darstahl]

Looks like it might be moby/moby#32838, for which a fix was submitted to the docker daemon in moby/moby#35924. The issue was the some weird unicode chars were left in the layer, which was causing Golang to fail to open the file. In the linked case, it was files left in the recycle bin.

[alexxie]

Thanks, please keep me informed of any progress.

[bublai]

@alexxie, I'm new on this development & landed with same error as mentioned by you.

• Installed Docker version 18.03.1-ce, build 9ee9f40.
  the exposed service shows up with External-IP, but could not browse API.
  any help will be appreciated.

[alexxie]

@bublai Our focus is now .net core and Linux container image which seems to be more stable then the Windows container image. I have not received further update on this issue and have'nt tested it further.