Extracting config.json

[jeb2239]

/home/barriosj/starter_app/app/BUILD.bazel:30:1: //app:app_image: missing input file '@rhel7//image:config.json'

This error occurs with the following rules called in BUILD

go_binary(
    name = "main",
    importpath = "nova/apps/main",
    library = ":go_default_library",
    visibility = ["//visibility:public"],
)


container_image(
    name="app_image",
    base="@rhel7//image",
    files = [":main"],
    cmd = ["main"]
    )

There is no config.json in the tar but there is a manifest.json which points to a config json that is in the tar.

[mattmoor]

Can you include the docker_pull rule from your WORKSPACE and perhaps the full output you see? Sometimes Bazel failures cascade in strange ways, and emit multiple misleading error messages.

I bring it up because if that file's missing, something clearly went wrong earlier, and I'd like to diagnose. thanks for reporting!

[jeb2239]

container_pull(
  name = "rhel7",
  registry = "dev-docker:5000",
  repository = "nova/rhel7",
  # 'tag' is also supported, but digest is encouraged for reproducibility.
  tag = "latest",
)

[jeb2239]

Any ideas what could be wrong?

[mattmoor]

No, can you share the full output? I suspect the true error is buried :(

[schoren]

I have the same issue with this container_pull:

container_pull(
    name = "debian_stretch",
    registry = "index.docker.io",
    repository = "library/debian",
    digest = "sha256:0a5fcee6f52d5170f557ee2447d7a10a5bdcf715dd7f0250be0b678c556a501b",
)

However, if I use tag instead of digest, it works fine:

container_pull(
    name = "debian_stretch",
    registry = "index.docker.io",
    repository = "library/debian",
    tag = "latest",
)

Running bazel like this bazel run -s //my:target --verbose_failures does not show any additional output.

[mattmoor]

Ah, this is almost certainly manifest lists. @jonjohnsonjr can you TAL?

[jonjohnsonjr]

@schoren that digest points to a multi-arch image. We don't currently handle these in a very useful way, see #300.

The amd64/linux form of that image would be this:

container_pull(
    name = "debian_stretch",
    registry = "index.docker.io",
    repository = "library/debian",
    digest = "sha256:02741df16aee1b81c4aaff4c48d75cc2c308bade918b22679df570c170feef7c",
)

Pulling by tag works because dockerhub will default to returning the amd64/linux image in this scenario.

@jeb2239 you might be seeing the same thing if the registry you're pulling from doesn't implement this backwards compatibility.

@mattmoor I think we should probably be resolving these images based on platform, but I don't think the right thing to do is blindly assume amd64/linux.

[mattmoor]

That sounds like a noble strategy for once we can build anything other than amd64/linux images :)

[mattmoor]

@jonjohnsonjr here's where we currently hardcode this.

I specifically asked @dekkagaijin to make these configurable like so in some changes he made to the library a while back, so this should just be a matter of plumbing. (famous last words)

[schoren]

Thanks guys. Very useful information. How do you get the digest for a specific image arch?
What I did was to docker pull the image, and then copy/paste the digest displayed.

[jeb2239]

Is there anyway to fix this for my case?

[mattmoor]

@jeb2239 Can you please share the full error output from Bazel? Sorry for letting this get a bit hijacked.

[jeb2239]

/usr/bin/bazel --output_user_root=/tmp/bazel/$USER build //app:app_image.tar
...............
DEBUG: /tmp/bazel/barriosj/47c85c2dccdb7d0c22e3d08d57cb3594/external/io_bazel_rules_go/go/def.bzl:124:3: DEPRECATED: go_repositories has been deprecated. go_rules_dependencies installs dependencies the way nested workspaces should, and go_register_toolchains adds the toolchains
DEBUG: /tmp/bazel/barriosj/47c85c2dccdb7d0c22e3d08d57cb3594/external/io_bazel_rules_go/go/private/rules/wrappers.bzl:51:5: 
DEPRECATED: //app:main : the library attribute on go_binary is deprecated. Please migrate to embed.
INFO: Analysed target //app:app_image.tar (192 packages loaded).
INFO: Found 1 target...
ERROR: missing input file '@rhel7//image:config.json'
ERROR: /home/barriosj/starter_app/app/BUILD.bazel:30:1: //app:app_image: missing input file '@rhel7//image:config.json'
Target //app:app_image.tar failed to build
ERROR: /home/barriosj/starter_app/app/BUILD.bazel:30:1 1 input file(s) do not exist
INFO: Elapsed time: 8.819s, Critical Path: 0.02s
FAILED: Build did NOT complete successfully

[mattmoor]

Interesting, does this happen without --output_user_root? I don't see too many other variables that might be upsetting this and my initial hypothesis seems wrong.

[jonjohnsonjr]

@mattmoor thanks, I'll take a look tomorrow.

@schoren assuming you have jq installed, this should work:

curl \
 -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" \
 -H "Authorization: Bearer $(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/debian:pull" | jq -r .token)" \
 https://index.docker.io/v2/library/debian/manifests/latest

[jeb2239]

The tarball looks like

-rw-r--r-- 0/0            1875 2017-08-03 17:34 328edcd84f1bbf868bc88e4ae37afe421ef19be71890f59b4b2d8ba48414b84d.json
drwxr-xr-x 0/0               0 2017-08-03 17:34 bf024e9c220242c2fb4b600824a66fec472c0235dc66d61003dc6c162ac88b0d/
-rw-r--r-- 0/0               3 2017-08-03 17:34 bf024e9c220242c2fb4b600824a66fec472c0235dc66d61003dc6c162ac88b0d/VERSION
-rw-r--r-- 0/0            1370 2017-08-03 17:34 bf024e9c220242c2fb4b600824a66fec472c0235dc66d61003dc6c162ac88b0d/json
-rw-r--r-- 0/0       200138752 2017-08-03 17:34 bf024e9c220242c2fb4b600824a66fec472c0235dc66d61003dc6c162ac88b0d/layer.tar
-rw-r--r-- 0/0             197 1969-12-31 19:00 manifest.json
-rw-r--r-- 0/0              84 1969-12-31 19:00 repositories

[mattmoor]

@jeb2239 Was the JSON above answering Jon? Was it the DockerHub url, or did you adapt it for your registry?

Can you share the contents of manifest.json and 328ed...jsonfrom inside the tarball? The latter is what should becomeconfig.json`, but the former points it there.

[jeb2239]

This is manifest.json of the centos image

[
  {
    "Config": "328edcd84f1bbf868bc88e4ae37afe421ef19be71890f59b4b2d8ba48414b84d.json",
    "RepoTags": [
      "centos:7"
    ],
    "Layers": [
      "bf024e9c220242c2fb4b600824a66fec472c0235dc66d61003dc6c162ac88b0d/layer.tar"
    ]
  }
]

[jeb2239]

And the 328edcd84f1bbf868bc88e4ae37afe421ef19be71890f59b4b2d8ba48414b84d.json is

{
  "architecture": "amd64",
  "config": {
    "Hostname": "c7947be2eb9d",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/bash"
    ],
    "ArgsEscaped": true,
    "Image": "sha256:627164ca1d6982948167cf47e75abcfe951dc69dc9f283597e2d990d9cdf349f",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": {
      "build-date": "20170801",
      "license": "GPLv2",
      "name": "CentOS Base Image",
      "vendor": "CentOS"
    }
  },
  "container": "a77cb215266a7806d19c290368a8d058fbe1bda6a0bea05730d1630b7e9a9e1c",
  "container_config": {
    "Hostname": "c7947be2eb9d",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/sh",
      "-c",
      "#(nop) ",
      "CMD [\"/bin/bash\"]"
    ],
    "ArgsEscaped": true,
    "Image": "sha256:627164ca1d6982948167cf47e75abcfe951dc69dc9f283597e2d990d9cdf349f",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": {
      "build-date": "20170801",
      "license": "GPLv2",
      "name": "CentOS Base Image",
      "vendor": "CentOS"
    }
  },
  "created": "2017-08-03T21:34:56.600704666Z",
  "docker_version": "17.03.2-ce",
  "history": [
    {
      "created": "2017-08-03T21:34:51.580892197Z",
      "created_by": "/bin/sh -c #(nop) ADD file:63492ba809361c51e75605d70390b549ff1187076b6d00485a1a0bb175daa40e in / "
    },
    {
      "created": "2017-08-03T21:34:55.624825166Z",
      "created_by": "/bin/sh -c #(nop)  LABEL name=CentOS Base Image vendor=CentOS license=GPLv2 build-date=20170801",
      "empty_layer": true
    },
    {
      "created": "2017-08-03T21:34:56.600704666Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/bash\"]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:b362758f4793674edb79ec5c7192074b2eacf200c006e127069856484526ccf2"
    ]
  }
}

[jeb2239]

The output user root is necessary other wise the output would be on NFS.

[jeb2239]

From my docker registry

curl -H "Accept: application/vnd.docker.distribution.manifest.list.v2+json" http://server.place.com:2375

{"message":"page not found"}

[andreykulik]

Same happened:

1. built an image with docker
2. pushed to gcp registry
3. tried to build another image using bazel and gotthe following:
   ERROR: BUILD:70:1: //foobar/a:a_image: missing input file '@gsfuse//image:config.json'

[{"Config":"8052e565ea22da1c06c44f3177e9040eed7c2e55f0e88ed08ddbc5086fd63dea.json","RepoTags":null,"Layers":["b2039296b7a9d102dc9b1e5c87e0bcde21e4ce4d22bf486e975645334c81375f/layer.tar","7be7e0c8444bded7af57ff6341fa64ee959e49f5e229d9b21e29e649a47e8366/layer.tar","e9f3b5fdf6c177c82c2b36409f7702894030f9d86e54596501db02eff88b7210/layer.tar","3f62f94afe91db9fc1a5fbb8c1fdabb51026adfb1f1b30d1c926802e386aab3f/layer.tar","65272cc726485520b5480fa532809741a7181d32526aa86f492948d0de6024b9/layer.tar"]}]

[mtsgrd]

I am experiencing the same problem, and am confused about what change causes this.. All my dependencies are specified with commits/hashes, and even if I checkout a commit I know worked yesterday, I still get the same message.

Edit:
A little more info, this happens when I specify a base image for a container_rule. It looks like I don't actually use commits/hashes for this base image, but can confirm that neither of the following work, while the default base image succeeds just fine.

 container_pull(
   name = "java_openjdk_base",
   registry = "index.docker.io",
   repository = "library/openjdk",
   tag = "8u151-jdk-stretch",
 )

 container_pull(
   name = "java_base",
   registry = "gcr.io",
   repository = "distroless/java",
 )

Edit 2:
Non-issue, this was solved for me by running bazel clean --expunge. Not sure if --expunge was necessary.

[nlopezgi]

is this still an issue? please reopen if so.