pulling image via SHA results in problems

[thoraxe]

/kind bug

Description
When pulling an image with the SHA as opposed to a straight tag, the sha ends up in the image name locally.

Steps to reproduce the issue:

[root@ip-172-31-4-216 images]# podman pull centos/nginx-112-centos7@sha256:42330f7f29ba1ad67819f4ff3ae2472f62de13a827a74736a5098728462212e7                                                                        
Trying to pull centos/nginx-112-centos7@sha256:42330f7f29ba1ad67819f4ff3ae2472f62de13a827a74736a5098728462212e7...Getting image source signatures
Skipping fetch of repeat blob sha256:469cfcc7a4b3947a4fa549c68cf4f8570be53779725f0c19f3d33d1520b08db0
Copying blob sha256:e11c8e053fb5d1761f80fbfe7369765ecf2942fb2d975d3dd1d2792fcd8cabbd
 9.74 MB / 9.74 MB [========================================================] 1s
Copying blob sha256:3198ba1a52956d97ff53912cfbd20cc333fef02bb0bf0dabf32659af1ea6d272
 4.63 KB / 4.63 KB [========================================================] 0s
Copying blob sha256:f55f016bb97cae4fab16eefa5f65b85f3112ce9758614d3b308f4f55e280d9d3
 168.90 KB / 168.90 KB [====================================================] 0s
Copying blob sha256:855937a11ee9df162ccf9b54ce13d5c26f7b6b9e821dcdbb74b3fdae5b4fc082
 29.82 MB / 29.82 MB [======================================================] 1s
Copying blob sha256:b0ea0c9c4eceb02c2f45fe2a441aecc27018c388c377921b1459a141f0aa7f54
 1.03 KB / 1.03 KB [========================================================] 0s
Copying blob sha256:0b4a8f5b072bb9bfad67b40af94a501bf0e4c3f045eee7a7473b4329603aa99a
 2.83 KB / 2.83 KB [========================================================] 0s
Copying blob sha256:97f174de3aa40275d186db852e729600313c7d9bb532ee8a53a394c5ec22223d
 168.38 KB / 168.38 KB [====================================================] 0s
Copying config sha256:925db2cb64d436a3bfdae2117c42771952f946bb4dc7a80158b4bb64e24530ba
 16.14 KB / 16.14 KB [======================================================] 0s
Writing manifest to image destination
Storing signatures
925db2cb64d436a3bfdae2117c42771952f946bb4dc7a80158b4bb64e24530ba
[root@ip-172-31-4-216 images]# podman images
REPOSITORY                                  TAG                                                                IMAGE ID       CREATED        SIZE
docker.io/projectodd/action-nodejs-8        8ee5579                                                            13eded11691b   5 weeks ago    508MB
docker.io/projectodd/action-java-8          8ee5579                                                            f637e4a864e8   5 weeks ago    461MB
docker.io/projectodd/action-python-3        8ee5579                                                            78314693a181   5 weeks ago    447MB
docker.io/projectodd/action-python-2        8ee5579                                                            7084c6a33219   5 weeks ago    362MB
docker.io/projectodd/action-php-7           8ee5579                                                            e946d73f22b0   5 weeks ago    331MB
docker.io/projectodd/whisk_couchdb          8ee5579                                                            03ca7eeb109e   5 weeks ago    501MB
docker.io/projectodd/whisk_alarms           8ee5579                                                            367ed4c38240   5 weeks ago    356MB
docker.io/projectodd/whisk_catalog          8ee5579                                                            82723460cf6b   5 weeks ago    245MB
docker.io/centos/nginx-112-centos7@sha256   42330f7f29ba1ad67819f4ff3ae2472f62de13a827a74736a5098728462212e7   925db2cb64d4   35 hours ago   344MB
[root@ip-172-31-4-216 images]# podman inspect centos/nginx-112-centos7@sha256:42330f7f29ba1ad67819f4ff3ae2472f62de13a827a74736a5098728462212e7                                                                     
[
    {
        "Id": "925db2cb64d436a3bfdae2117c42771952f946bb4dc7a80158b4bb64e24530ba",
        "Digest": "sha256:b63dd9fcb9e59f1bc26c73491ea9894bbf8373e27862fad56d71aada7735046a",
        "RepoTags": [
            "docker.io/centos/nginx-112-centos7@sha256:42330f7f29ba1ad67819f4ff3ae2472f62de13a827a74736a5098728462212e7"
        ],
        "RepoDigests": [
            "docker.io/centos/nginx-112-centos7@sha256@sha256:b63dd9fcb9e59f1bc26c73491ea9894bbf8373e27862fad56d71aada7735046a"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2018-05-31T06:57:21.682696087Z",
        "ContainerConfig": {
            "User": "1001",
            "ExposedPorts": {
                "8080/tcp": {},
                "8443/tcp": {}
            },
...

Describe the results you expected:
I expected "sha" not to be in the image name and I expected the local digest to match what I pulled.

Output of podman version:

[root@ip-172-31-4-216 images]# podman version
Version:       0.4.1
Go Version:    go1.9.2
OS/Arch:       linux/amd64

Output of podman info:

[root@ip-172-31-4-216 images]# podman info
host:
  MemFree: 159133696
  MemTotal: 8200544256
  SwapFree: 0
  SwapTotal: 0
  arch: amd64
  cpus: 2
  hostname: ip-172-31-4-216.ec2.internal
  kernel: 3.10.0-862.el7.x86_64
  os: linux
  uptime: 5h 33m 4.77s (Approximately 0.21 days)
insecure registries:
  registries:
  - 172.30.0.0/16
registries:
  registries:
  - registry.access.redhat.com
store:
  ContainerStore:
    number: 0
  GraphDriverName: overlay
  GraphOptions:
  - overlay.override_kernel_check=true
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
  ImageStore:
    number: 9
  RunRoot: /var/run/containers/storage

Additional environment details (AWS, VirtualBox, physical, etc.):

Linux ip-172-31-4-216.ec2.internal 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux on AWS (RHEL 7.5 cloud access ami, not updated)

[rhatdan]

I am not sure if this is still a problem, but you are using an older version of podman. We are just about to ship podman 0.6.1 for RHEL systems.
Any chance you could attempt ot build from github? Or test on a different OS.

[thoraxe]

Can I just download the Koji stuff that lsm5 pointed me at the other day?

[rhatdan]

Sure.

[rhatdan]

@thoraxe Were you able to see if this was fixed?

[mheon]

Reproduced this on master, so it's definitely still an issue. I wonder if this is our parsing code, or c/image?

[rhatdan]

@mtrmac WDYT?

[mtrmac]

The docker.io/centos/nginx-112-centos7@sha256 423… output is a display matter: see how cmd/podman/images.go more or less always expects a tag to exist, and in particular libpod/image/ReposToMap.

The

        "RepoDigests": [
            "docker.io/centos/nginx-112-centos7@sha256@sha256:b63…"
        ],

output is also a display matter, libpod/image/Image.RepoDigests.

It’s not immediately clear to me why the digest value is sha256:b63…. Does that part reproduce on fresh versions?

[mtrmac]

display matter…

FWIW looking at c/image/storage, it seems that Image.Names should be parseable with c/image/docker/reference.ParseNormalizedNamed — OTOH those that fail this parsing are ignored in c/image/storage, so I don’t know whether there are any other acceptable values.

[mtrmac]

It’s not immediately clear to me why the digest value is sha256:b63…. Does that part reproduce on fresh versions?

It does - it is a digest of the :latest tag, which is the one actually pulled:

• imageParts can not represent digest references at all; so, decompose drops the digest on the floor, and applies a default tag latest; so, that’s what gets actually pulled (and the result is then tagged in containers/storage with the original digest reference).
• (Secondarily, decompose relies on c/image/docker/reference.Parse().Domain(), which parses centos/nginx-… as domain=centos, path=nginx…; tests indicate that it is an intended behavior of docker/distribution/reference for some reason. But this puts Image.createNamesToPull into the decomposedImage.hasRegistry path, which essentially applies the docker.io[/library] normalization — and that happens to work because docker.io is the first on the search path by default, at least on Fedora.)

[mheon]

@baude This seems like an issue with the image name parsing code, if we can't even recognize/handle digests