Trying to use buildx and .net core not working, .net core issue?

[RamblingGeekUK]

Is this a .net issue or a buildx issue ?

Building for AMD/64 works fine :

d:\Dev\dockertesting>docker buildx build --platform linux/amd64 -t ramblinggeekuk/dockertesting --push . [+] Building 211.7s (15/15) FINISHED => [internal] load build definition from Dockerfile 0.2s => => transferring dockerfile: 32B 0.0s => [internal] load .dockerignore 0.3s => => transferring context: 2B 0.0s => [internal] load metadata for mcr.microsoft.com/dotnet/core/aspnet:2.2 1.1s => [internal] load metadata for mcr.microsoft.com/dotnet/core/sdk:2.2 1.2s => [build-env 1/6] FROM mcr.microsoft.com/dotnet/core/sdk:2.2@sha256:b4c25c26dc73f498073fcdb4aefe1677 110.2s => => resolve mcr.microsoft.com/dotnet/core/sdk:2.2@sha256:b4c25c26dc73f498073fcdb4aefe167793eb3a8c79ef 0.0s => => sha256:b4c25c26dc73f498073fcdb4aefe167793eb3a8c79effa76df768006b5c345b8 2.19kB / 2.19kB 0.0s => => sha256:36add2b62b779b538e3062839e1978d78e12e26ec2214940e9043924a29890c0 1.80kB / 1.80kB 0.0s => => sha256:ade3808af4d74b6181b7412b73314b5806fa39d140def18c6ee1cdbcb3ed41b1 300.69MB / 300.69MB 40.3s => => sha256:52c7fe5918815504427b3168845267e876464f8b010ccc09d0f61eb67dd6a17e 4.41kB / 4.41kB 0.0s => => sha256:dbdc36973392a980d56b8fab63383ae44582f6502001d8bbdd543aa3bf1d746e 10.79MB / 10.79MB 9.3s => => sha256:aaef3e0262580b9032fc6741fb099c7313834c7cf332500901e87ceeb38ac153 50.07MB / 50.07MB 58.7s => => sha256:a4d8138d0f6b5a441aaa533faf5fe0c3996a6ca42643c46f4402c7e8bda53742 45.34MB / 45.34MB 53.0s => => sha256:f59d6d019dd5b8398eb8d794e3fafe31f9411cc99a71dabfa587bf732b4a7385 4.34MB / 4.34MB 62.4s => => sha256:f62345fbba0dbbb77ba8aca5b81a4f0d8ec16c5d540def66c0b8e8d6492fa444 13.25MB / 13.25MB 59.3s => => sha256:373065ab5fafec0e8bcfd74485dcd728f40b80800867c553e80c7cd92cd5d504 173.83MB / 173.83MB 79.8s => => unpacking mcr.microsoft.com/dotnet/core/sdk:2.2@sha256:b4c25c26dc73f498073fcdb4aefe167793eb3a8c7 29.0s => [stage-1 1/3] FROM mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:b18d512d00aff0937699014a9ba44234 54.5s => => resolve mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:b18d512d00aff0937699014a9ba44234692ce424c 0.0s => => sha256:b18d512d00aff0937699014a9ba44234692ce424c70248bedaa5a60972d77327 2.19kB / 2.19kB 0.0s => => sha256:8bd16a07ec8b72f4131a1747ff479048db65b48f54ae2ced1ffb1b42798c952e 1.16kB / 1.16kB 0.0s => => sha256:318149b63beb70e442e84e530f4472f9354e3906874c35be2ba5045b5f7a8c7a 4.06kB / 4.06kB 0.0s => => sha256:fc7181108d403205fda45b28dbddfa1cf07e772fa41244e44f53a341b8b1893d 22.49MB / 22.49MB 27.7s => => sha256:2c86df27317feb8a2806928aa12f27e6c580894e0cb844cb25aaed1420964e3d 17.69MB / 17.69MB 40.7s => => sha256:66dd687a6ad17486c0e3bc4e3c3690cefb7de9ad55f654e65cf657016ed4194c 2.98MB / 2.98MB 41.5s => => sha256:a7638d93f1fe40e3393bfb685305ce5022179c288f5b2a717978ccae465b4d7a 62.13MB / 62.13MB 48.1s => => unpacking mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:b18d512d00aff0937699014a9ba44234692ce42 5.2s => [internal] load build context 0.5s => => transferring context: 6.63kB 0.0s => [stage-1 2/3] WORKDIR /app 0.4s => [build-env 2/6] WORKDIR /app 0.2s => [build-env 3/6] COPY *.csproj ./ 0.7s => [build-env 4/6] RUN dotnet restore 10.7s => [build-env 5/6] COPY . ./ 0.4s => [build-env 6/6] RUN dotnet publish -c Release -o out 3.6s => [stage-1 3/3] COPY --from=build-env /app/out . 0.3s => exporting to image 83.5s => => exporting layers 1.0s => => exporting manifest sha256:6ac874b02ae2ce6c86c5d79290a04694778b2f86ff787285650c11dce4b2a37e 0.2s => => exporting config sha256:e9625bb3b3e783bcb6f6b7dd8b3ad4a1f090a1156be3bf237d5d4b7c8f97ebcc 0.2s => => pushing layers 81.3s => => pushing manifest for docker.io/ramblinggeekuk/dockertesting:latest 0.6s

Building for linux/arm/v7 - fails...

'd:\Dev\dockertesting>docker buildx build --platform linux/arm/v7 -t ramblinggeekuk/dockertesting --push .
[+] Building 112.6s (11/14)
=> [internal] load .dockerignore 0.2s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.3s
=> => transferring dockerfile: 443B 0.0s
=> [internal] load metadata for mcr.microsoft.com/dotnet/core/aspnet:2.2 1.5s
=> [internal] load metadata for mcr.microsoft.com/dotnet/core/sdk:2.2 1.6s
=> [build-env 1/6] FROM mcr.microsoft.com/dotnet/core/sdk:2.2@sha256:b4c25c26dc73f498073fcdb4aefe1677 102.6s
=> => resolve mcr.microsoft.com/dotnet/core/sdk:2.2@sha256:b4c25c26dc73f498073fcdb4aefe167793eb3a8c79ef 0.0s
=> => sha256:b4c25c26dc73f498073fcdb4aefe167793eb3a8c79effa76df768006b5c345b8 2.19kB / 2.19kB 0.0s
=> => sha256:27c2d2f8b92b964c1e3f4de6c8025b1f0362a1f3436118d77b3dbfa921cfd9c9 1.80kB / 1.80kB 0.0s
=> => sha256:493f51ba80c0d5fd46ea25516eb221089190b416d5a8cc2c898517dea68519a4 4.91kB / 4.91kB 0.0s
=> => sha256:e06d849c15a63e2cf30d5c5af0d9aa87b2f7c6cbfe0e8c3e351fa4c5d4666d11 300.71MB / 300.71MB 44.8s
=> => sha256:41835060b113803e2ca628a32805c2e1178fe441b81d3e77427749fec4de06e9 9.49MB / 9.49MB 45.9s
=> => sha256:da770cd5eae6caeefe9468e318964be31036c06e729c2d983756906ede859b17 46.39MB / 46.39MB 51.2s
=> => sha256:582caf5d2e7bf5e75a96afc2254a97f6e86ad72c8815429ada61280467cc6d6f 3.92MB / 3.92MB 45.0s
=> => sha256:dd04b2ffc5474ba8df46350a273baaf841243fda01cfe05d3e5429e4ecc9bb19 144.38MB / 144.38MB 73.9s
=> => sha256:fa48f739865746afb4020d2d370105be51d23dd6ad6faa8663e1365b607d46c2 13.04MB / 13.04MB 52.3s
=> => sha256:dcb61f1d45657be196f648f75a07805b856fb8f4aebb61138c03c12e2919ee9e 42.08MB / 42.08MB 57.5s
=> => unpacking mcr.microsoft.com/dotnet/core/sdk:2.2@sha256:b4c25c26dc73f498073fcdb4aefe167793eb3a8c7 27.0s
=> [stage-1 1/3] FROM mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:b18d512d00aff0937699014a9ba44234 18.5s
=> => resolve mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:b18d512d00aff0937699014a9ba44234692ce424c 0.0s
=> => sha256:b18d512d00aff0937699014a9ba44234692ce424c70248bedaa5a60972d77327 2.19kB / 2.19kB 0.0s
=> => sha256:9ad51bcfeeb6e58218f23fb1f4c5229b39008cc245c9df1fcf8c9330c18a2acb 1.16kB / 1.16kB 0.0s
=> => sha256:8b7eead4e00d6228dbbf945848d78b43580687575eb8cba1d7a2b11129186f77 4.07kB / 4.07kB 0.0s
=> => sha256:a51e654c7ec5bf1fd3f38645d4bc8aa40f86ca7803d70031a9828ae65e3b67ae 63.47MB / 63.47MB 8.9s
=> => sha256:2eead4197fac409644fd8aaf115559d6383b0d56f1ad04d7116aaabbcbea8bed 19.28MB / 19.28MB 10.3s
=> => sha256:9358a462710e1891aec7076e8674e6f522f08a9f3624dc1f55554c2fc7cb99ea 16.30MB / 16.30MB 12.0s
=> => sha256:14144450932b5358107e71ebcd25ec878cb799ccc75ec39386e374d0dad903b3 2.88MB / 2.88MB 12.2s
=> => unpacking mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:b18d512d00aff0937699014a9ba44234692ce42 4.5s
=> [internal] load build context 0.2s
=> => transferring context: 7.04kB 0.0s
=> [stage-1 2/3] WORKDIR /app 0.2s
=> [build-env 2/6] WORKDIR /app 0.3s
=> [build-env 3/6] COPY *.csproj ./ 0.4s
=> ERROR [build-env 4/6] RUN dotnet restore 7.0s'

[tonistiigi]

cc @tuonga

@RamblingGeekUK You should have more logs after the failure. You can try --progress=plain that creates logs more suitable for copying here.

[RamblingGeekUK]

Sorry it took me a little while to get back to you here you go :

`d:\Dev\dockertesting>docker buildx build --platform linux/amd64,linux/arm64 -t ramblinggeekuk/dockertesting --push . --progress=plain
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 32B done
#1 DONE 0.2s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.3s

#3 [linux/arm64 internal] load metadata for mcr.microsoft.com/dotnet/core/s...
#3 ERROR: no match for platform in manifest sha256:a50e175acd618c3e90bc91dceb5194e6c3764c5b4d179390cef874a887476ba9: not found

#4 [linux/arm64 internal] load metadata for mcr.microsoft.com/dotnet/core/a...
#4 ERROR: no match for platform in manifest sha256:c7bf4bc15468dd59af0a400e1f71d944d3ff8d91fa3c615a617d75b4df62e3e0: not found

#11 [internal] load build context
#11 DONE 0.0s

#9 [linux/arm64 build-env 1/6] FROM mcr.microsoft.com/dotnet/core/sdk:2.2
#9 resolve mcr.microsoft.com/dotnet/core/sdk:2.2 0.0s done
#9 ERROR: no match for platform in manifest sha256:a50e175acd618c3e90bc91dceb5194e6c3764c5b4d179390cef874a887476ba9: not found

#7 [linux/arm64 stage-1 1/3] FROM mcr.microsoft.com/dotnet/core/aspnet:2.2
#7 resolve mcr.microsoft.com/dotnet/core/aspnet:2.2 0.1s done
#7 ERROR: no match for platform in manifest sha256:c7bf4bc15468dd59af0a400e1f71d944d3ff8d91fa3c615a617d75b4df62e3e0: not found

#5 [linux/amd64 internal] load metadata for mcr.microsoft.com/dotnet/core/s...
#5 DONE 0.6s

#6 [linux/amd64 internal] load metadata for mcr.microsoft.com/dotnet/core/a...
#6 DONE 0.6s

[linux/arm64 internal] load metadata for mcr.microsoft.com/dotnet/core/sdk:2.2:

---

---

[linux/arm64 internal] load metadata for mcr.microsoft.com/dotnet/core/aspnet:2.2:

---

---

[linux/arm64 build-env 1/6] FROM mcr.microsoft.com/dotnet/core/sdk:2.2:

---

---

[linux/arm64 stage-1 1/3] FROM mcr.microsoft.com/dotnet/core/aspnet:2.2:

---

failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to build LLB: failed to load cache key: no match for platform in manifest sha256:a50e175acd618c3e90bc91dceb5194e6c3764c5b4d179390cef874a887476ba9: not found

d:\Dev\dockertesting>`

[tonistiigi]

mcr.microsoft.com/dotnet/core/aspnet:2.2 used by your build doesn't seem to exist for linux/arm64

> docker buildx imagetools inspect mcr.microsoft.com/dotnet/core/aspnet:2.2
Name:      mcr.microsoft.com/dotnet/core/aspnet:2.2
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:c7bf4bc15468dd59af0a400e1f71d944d3ff8d91fa3c615a617d75b4df62e3e0

Manifests:
  Name:      mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:50920223f8caa8a5f5cdc7a1593a3881cf17a9c350da929e6b76a45cd4db23ef
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64

  Name:      mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:e24c3009eea90ca3776029ecd0f209ead0dbd7e04920be992e064bea206f5611
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7

  Name:      mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:f087e80c9d3b22ea55f2581a6c7d54fcb39ff9e1e3a4bf213734fc8ccadbe02a
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  windows/amd64
  OSVersion: 10.0.17134.885

  Name:      mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:e58b6f86d0b14c577ceca09646f6d8f12fd54df34a5929d792b7c97db3c86f66
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  windows/amd64
  OSVersion: 10.0.17763.615

  Name:      mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:431afc87c6ead96769e6c6049d0aa6c3ae372ce37ac24b5253ce315e39d1015b
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  windows/arm
  OSVersion: 10.0.17763.615

  Name:      mcr.microsoft.com/dotnet/core/aspnet:2.2@sha256:dbf8f3512924198486a91ac67d30e362ff586c27c2c339c16da5569a0d0fbab9
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  windows/amd64
  OSVersion: 10.0.18362.239

[RamblingGeekUK]

yeah, that's what I'm confused by, if .net core runs on arm, how do I tell it to grab the correct version?

[tonistiigi]

If the base images are only for linux/amd64 and linux/arm/v7 you can only build for these platforms. I'm not sure about this image, regarding why there is no arm64 variant.

[scyto]

I am not sure what you are trying to run, but the current runtime has amd64 arm64 and armv7 (not no v6) for example

FROM mcr.microsoft.com/dotnet/core/runtime

I just used this and buildx to build a multiplatform release - sooo much easier thanks, this is awesome project

[franciscojsc]

Environment

Machine= Xubuntu 18.04, 12GB RAM

Installed qemu-user-static.

sudo apt-get install qemu-user-static

whereis qemu-user-static
qemu-user-static: /usr/share/man/man1/qemu-user-static.1.gz

Using the command docker buildx ls :

NAME/NODE    DRIVER/ENDPOINT             STATUS  PLATFORMS
frbuilder *  docker-container                    
  frbuilder0 unix:///var/run/docker.sock running linux/amd64, linux/arm64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
default      docker                              
  default    default                     running linux/amd64, linux/386

Command Used to generate the images

docker buildx build --platform linux/arm,linux/arm64,linux/amd64 -t franciscojsc/my-projeto . --push

Error

A fatal error occurred, the folder [/usr/share/dotnet/host/fxr] does not contain any version-numbered child folders

I used the Dockerfile:

FROM mcr.microsoft.com/dotnet/core/sdk:3.0 AS build
WORKDIR /app
COPY *.csproj ./
RUN dotnet restore
COPY . ./
RUN dotnet publish -c Release -o output

FROM mcr.microsoft.com/dotnet/core/aspnet:3.0
WORKDIR /app
COPY --from=build /app/output .
ENTRYPOINT ["dotnet", "Project.dll"]

Solution

Replacing sdk:3.0 with sdk:3.0-alpine and it worked.

FROM mcr.microsoft.com/dotnet/core/sdk:3.0-alpine AS build
WORKDIR /app
COPY *.csproj ./
RUN dotnet restore
COPY . ./
RUN dotnet publish -c Release -o output

FROM mcr.microsoft.com/dotnet/core/aspnet:3.0
WORKDIR /app
COPY --from=build /app/output .
ENTRYPOINT ["dotnet", "Project.dll"]

[syediddi]

@franciscojsc It worked but how did you figure out?

[zaneclaes]

Despite using -alpine for the SDK and linux/arm for buildx, I now get: failed to solve: rpc error: code = Unknown desc = failed to load cache key: no match for platform in manifest.

[tonistiigi]

which image? mcr.microsoft.com/dotnet/core/sdk:3.0-alpine is not multi-arch

» docker buildx imagetools inspect mcr.microsoft.com/dotnet/core/sdk:3.0-alpine
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 6434,
      "digest": "sha256:378d7a54a1928ec3017d2f4b81ffb8a621e2aa5aaa1145d273c3043c03d75af6"

[zaneclaes]

which image?

Tested sdk:3.0-alpine, but I'd like to build for 5.0.

mcr.microsoft.com/dotnet/core/sdk:3.0-alpine is not multi-arch

Yep, that's my point. Though, mcr.microsoft.com/dotnet/sdk:5.0 (which is my desired target) does list as multi-arch:

docker buildx imagetools inspect mcr.microsoft.com/dotnet/sdk:5.0                                                                                                                                                                                                                                                                  
Name:      mcr.microsoft.com/dotnet/sdk:5.0
MediaType: application/vnd.docker.distribution.manifest.list.v2+json
Digest:    sha256:ac49854ff6dcc1a2916ffc0981503f571698458187f925da0c2f2b6a0bec8dee
           
Manifests: 
  Name:      mcr.microsoft.com/dotnet/sdk:5.0@sha256:ba563c05a198fb112a24dfb0cb9bcc1bc07c6cdd2ed3f6ce0603f4424968973c
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64
             
  Name:      mcr.microsoft.com/dotnet/sdk:5.0@sha256:d71623eda2633f1a5d8de0a8b77f6564cff8d289d71da2543ebf4a1e0fa54bfe
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm/v7

... but when I try to build any of the multi-arch images with buildx v0.4.2 on linux/arm/v7, I get the same error pasted above by @franciscojsc :

A fatal error occurred, the folder [/usr/share/dotnet/host/fxr] does not contain any version-numbered child folders

[zaneclaes]

Related: dotnet/dotnet-docker#1537

My goal is to use Travis to build a multi-arch docker image that can target amd64, as well as Raspberry Pis (armv7, at least). I have this working in another repository that targets Node.js, but am struggling with dotnet.

Importantly, I can build using the mcr.microsoft.com/dotnet/sdk:5.0 base image to build on a Raspberry Pi itself. I suppose I could manually stitch the different images together after-the-fact. But this leaves me in a situation where I need a Raspberry Pi as part of my CI, which AFAICT is not something offered by Travis.

[devedse]

I've also ran into this issue. The only way I managed to fix it is by manually doing a build with a specific docker file. Basically following this pattern:
dotnet/dotnet-docker#1537 (comment)

Apparently the issue is that you can't build ARM32 images on an ARM64 host. I don't understand why or how we can fix this with buildx though.

[robtaylor]

It makes sense you can't build for an arch if the source image doesn't contain that arch. Could the error message be made more comprehensible though?

[robtaylor]

You also get this error if you mistype your arch...

[crazy-max]

You can test with QEMU from https://github.com/tonistiigi/binfmt that is made for running containers:

$ docker run -it --rm tonistiigi/binfmt:latest --version
binfmt/8703596 qemu/v6.0.0 go/1.16.5