Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Badge Request: GitHub Container Registry #5594

Open
cicirello opened this issue Sep 23, 2020 · 28 comments
Open

Badge Request: GitHub Container Registry #5594

cicirello opened this issue Sep 23, 2020 · 28 comments
Labels
service-badge New or updated service badge

Comments

@cicirello
Copy link

📋 Description

GitHub recently introduced the GitHub Container Registry (see: https://github.blog/2020-09-01-introducing-github-container-registry/). It would be nice if there were badges available similar to those that shields has for Docker image version, image size, and pulls, but for the new GitHub Container Registry.

🔗 Data

I'm not sure if GitHub has an API for this yet.

🎤 Motivation

Motivation is the same as that of the equivalent badges that already exist for Docker Hub, but targeting the GitHub Container Registry instead. Providing a nice concise visual summary of some of the key statistics one might be interested in with a Docker image, but for images available in the GCR.

@cicirello cicirello added the service-badge New or updated service badge label Sep 23, 2020
@chris48s
Copy link
Member

Given this is essentially a feature related to the package registry, I suspect this one is probably going to end up blocked on #4169 (tldr: the github tokens we have don't have the necessary scope to read package metadata) once the API endpoints are available, but we'll see..

@adriens
Copy link

adriens commented Oct 20, 2021

🙏

@kidager
Copy link

kidager commented Feb 9, 2022

@eggplants
Copy link

To get the pull counter of the image published on ghcr, you can use PackageStatistics or PackageVersionStatistics in GraphQL API (needs auth).

To get information of tags and sizes of public images on ghcr.io, you can use https://ghcr.io/v2/ as Docker Registry HTTP API V2's endpoint like DockerHub.

Here is my example with curl:

# public image's {USER}/{IMAGE}
USER_IMAGE=eggplants/asciiquarium-docker

# get fake token ('{"token":"***"}' -> '***')
TOKEN="$(
  curl "https://ghcr.io/token?scope=repository:${USER_IMAGE}:pull" |
  awk -F'"' '$0=$4'
)"

_curl(){ curl -H "Authorization: Bearer ${TOKEN}" "$1" }

# get tags
_curl "https://ghcr.io/v2/${USER_IMAGE}/tags/list"
# >>> {"name":"eggplants/asciiquarium-docker","tags":["0.0.0","0.0","latest"]}

# get manifest of the latest image
_curl "https://ghcr.io/v2/${USER_IMAGE}/manifests/latest"
# >>>
# {
#    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
#    "schemaVersion": 2,
#    "config": {
#       "mediaType": "application/vnd.docker.container.image.v1+json",
#       "digest": "sha256:fb4552eacdc8e01082f87bae6a0fec6e1b61630f0a1e5a0e6a699d05ff0337fd",
#       "size": 5551
#    },
#    "layers": [
#       {
#          "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
#          "digest": "sha256:4be315f6562fccf08fd6c749557e31e45ab6d987370e20e2c4933ddb04ddd5ff",
#          "size": 27140664
#       }, ...

@calebcartwright
Copy link
Member

Thank you for sharing @eggplants though to be clear, this is not blocked on the basics of the api request/response, but due to the associated scopes required by github, which our tokens do not have.

This is blocked on that much bigger topic detailed in #4169

@eggplants
Copy link

eggplants commented May 10, 2022

The number of pulls seems difficult to fetch, but it would be relatively easy to get just the tags and their image sizes, which do not require authentication.

@calebcartwright
Copy link
Member

The number of pulls seems difficult to fetch, but it would be relatively easy to get just the tags and their image sizes, which do not require authentication.

Gotcha, thanks for the clarification! In that case think it would probably make sense at this point to extract a separate issue for version and size badges to discuss the viability of the proposed approach, and we narrow the scope of this blocked one to pulls

@eggplants
Copy link

eggplants commented May 12, 2022

I have implemented eggplants/ghcr-badge on a trial basis:
latest_tag
size
tags

@asssaf
Copy link

asssaf commented May 15, 2022

you can also hard code the token per image:

curl -H "Authorization: Bearer $(echo "v1:$USER_IMAGE:0" | base64)" "https://ghcr.io/v2/$USER_IMAGE/tags/list"

@eggplants
Copy link

@asssaf Thanks! eggplants/ghcr-badge@f5504c4

scottrigby added a commit to scottrigby/prometheus-community-helm-charts that referenced this issue Nov 3, 2022
…HTTP repos

Note: did not yet differentiate the downloads badge between HTTP and OCI. A
badge for GHCR package downloads is currently blocked. See:
badges/shields#5594

Signed-off-by: Scott Rigby <[email protected]>
@simonmicro
Copy link

With the coming shutdown of free organizations on Docker Hub, we also plan to migrate to ghcr.io - what is the state of this implementation right now? I really do not want to stress this 😁 ! Take your time ❤️‍🔥

@trajan0x

This comment was marked as off-topic.

@nvuillam
Copy link

nvuillam commented Oct 20, 2023

i'm trying with GraphQL api, and even with a Personal Access Token set in headers, I don't receive any package in the results
Would anyone have a tip ? :)

  • Request

Authorization: bearer XXXXXXXXX

query {
  organization(login: "oxsecurity") {
      name
      isVerified
    packages(first: 100) {
        nodes {
            name
            packageType
            statistics {
                downloadsTotalCount
            }
        }
    }
  }
}
  • Result
{
    "data": {
        "organization": {
            "name": "OX Security",
            "isVerified": true,
            "packages": {
                "nodes": []
            }
        }
    }
}

@eggplants
Copy link

@nvuillam It doesn't seem to work.
eggplants/ghcr-badge#72 (comment)

@nvuillam
Copy link

@nvuillam It doesn't seem to work. eggplants/ghcr-badge#72 (comment)

That' really sad :(

Thanks for the reply !

@cicirello
Copy link
Author

@nvuillam It doesn't seem to work. eggplants/ghcr-badge#72 (comment)

That' really sad :(

Thanks for the reply !

@nvuillam try a repository query (instead of an organization query) to get the packages for the megalinter repository. It looks like all of your public packages are associated with that repository, so that might work. I didn't try this so I don't know for sure if it will work, but I think it might.

@eggplants
Copy link

eggplants commented Dec 9, 2023

@cicirello You can try this query from https://docs.github.com/en/graphql/overview/explorer. And repository.packages is also empty.

query {
  repository(owner: "oxsecurity", name: "megalinter") {
    name    
    packages(first: 1) {
      nodes {
        name
        packageType
        statistics {
          downloadsTotalCount
        }
      }
    }
  }
}

image

Note that oxsecurity/megalinter has 165 Packages.

@nvuillam
Copy link

nvuillam commented Dec 9, 2023

I confirm, I still did not find any way to query stats for MegaLinter packages :/

@jovezhong
Copy link

It's sad that the GraphQL API from GitHub doesn't work.

I ended up with a tool to parse the HTML page and list only named versions

range 1 5 page
|| open https://github.com/timeplus-io/proton/pkgs/container/proton/versions?page=$page$ --cache false --html --hashtml|| filter "(nodeName=='LI') and attributes.class =='Box-row'" || sequence
|| html innerHTML || filter "(_html.nodeName == 'A')" ||rename _html.innerText version || startswith version '1.3.'  || html innerHTML || filter "(_html.nodeName == 'SPAN' and _html.attributes.style =='white-space:nowrap;')"||rename _html.innerText downloads || replace downloads 'Version downloads' ''||table version downloads

image

@nvuillam
Copy link

I thought to build something similar, like github-dependent-infos scraps to get the number and list of dependent repos...

But a working API would be the best, we're just looking for simple numbers :(

@ipitio
Copy link

ipitio commented Jun 10, 2024

I made a JSON endpoint for the downloads, size, and more (including for each version/tag!) that can be used to make a badge. Just star the repo to have your packages added!

backage

Example:

downloads

@TheCaptain989
Copy link

TheCaptain989 commented Jun 10, 2024

I made a JSON endpoint for the total downloads that can be used to make a badge. Submit an issue if you want me to also track counts for each version/tag.

ghcr-pulls

This is fantastic!

I would love support for tags, though. My main use case is here: https://github.com/linuxserver/docker-mods/pkgs/container/mods/
To pick one tag: https://github.com/linuxserver/docker-mods/pkgs/container/mods/227277266?tag=universal-docker

Trying to mess with it now...

@ipitio
Copy link

ipitio commented Jun 16, 2024

@TheCaptain989 I've got version support mostly implemented!

I'm also expanding support to all package types on GitHub Packages and want to add sizes for each version to the json, but I haven't looked into whether it's possible to get the sizes via rest api/scraping/graphql.

Does anyone know how/if I could get the sizes? ghcr-badge and, at the moment, my repo cheat for containers by using their docker manifests. Cheating on the other ones would also be fine!

@eggplants
Copy link

@ipitio I read your current approach and you are only scraping the page source of GitHub Package directly without using docker manifests. Is it used in commits that have not yet been pushed in version support? Anyway, this may be broken in the future due to modification of the page structure.

The version and number of downloads can be obtained from the HTML source, but the size must be referenced in the docker manifest.


Based on my research for now:

GHCR Docker Registry HTTP API GitHub REST API GitHub GraphQL API Scraping Package Page
Versions o o x o
Image size o x x x
Downloads x x x o

@ipitio
Copy link

ipitio commented Jun 17, 2024

I've got size working using the manifest on the dev branch, where I've implemented version support too. Will merge the changes to master soon!

https://github.com/ipitio/ghcr-pulls/blob/dev/update.sh

@TheCaptain989
Copy link

@TheCaptain989 I've got version support mostly implemented!

Awesome.
I got tags/versions working for my purposes: https://github.dev/TheCaptain989/ghcr-pulls/

@douglasparker
Copy link

I guess I’m a little confused why I am getting emails about another project within this issue. Wouldn’t it be better that you open a discussion on your own repo?

@ipitio
Copy link

ipitio commented Jun 17, 2024

Sorry! I've made a discussion here: ipitio/backage#9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
service-badge New or updated service badge
Projects
None yet
Development

No branches or pull requests