Build both agent and inbound-agent container images in this repository

[lemeurherve]

What

I propose to build both images in this repository.

Why

• Faster agents release:
  • Both images build in the same job
    • in parallel in case of Linux images
    • sequentially in case of Windows images, with inbound-agent benefiting from the agent container image build cache
  • No need to wait merging docker-agent image, wait for the tag then the publication on Docker Hub, then updatecli PR on docker-inbound agent, merge, tag and publication again.
• Less work and maintenance
  • One common Dockerfile per OS flavor (Alpine, Debian, Nanoserver, Windows Server Core)
    • There is no Archlinux inbound-agent image, a new one could easily be added but Decide the future of the Archlinux image: either add "Archlinux JDK17" or remove "Archlinux JDK11" #422
  • One common set of tests helpers
  • One common images and tags definition
    • docker bake file for Linux images
    • docker compose file for Windows images
• No change whatsoever for final users
  • Same Docker Hub repositories, images, tags and content
• No practical change for contributors
  • Better possibilities for custom local builds and tests

How

Code wise

I'm using multistages Dockerfiles, docker bake's matrix for Linux images (allowing parallel builds), and a loop over "agent" and "inbound-agent" with docker compose and --target for Windows images.

I've splitted my pull requests commits:

• Renamed agent files
• Imported files from inbound-agent, with their related git history reapplied as patch
• One atomic commit with all code changes

New file

• README.md

Modified

• .gitignore
• alpine/Dockerfile
• build.ps1
• build.sh
• debian/Dockerfile
• debian/preview/Dockerfile
• docker-bake.hcl
• Jenkinsfile
• Makefile
• tests/agent.Tests.ps1
• tests/test_helpers.bash
• tests/test_helpers.psm1
• windows/nanoserver/Dockerfile
• windows/windowsservercore/Dockerfile

Renamed from agent

• README_agent.md

Renamed and modified from agent

• tests/tests_agent.bats

Imported from inbound-agent

• jenkins-agent
• jenkins-agent.ps1
• tests/netcat-helper/Dockerfile
• tests/netcat-helper/Dockerfile-windows

Imported and renamed from inbound-agent

• README_inbound-agent.md

Imported and modified from inbound-agent

• tests/inbound-agent.Tests.ps1

Imported, renamed and modified from inbound-agent

• .github/workflows/sync-dockerhub-readme.yml
• tests/tests_inbound-agent.bats

PR preview: lemeurherve/docker-agent@master...feat-build-both-agent-and-inbound-agent
Main commit: lemeurherve@8f27a4e

Repo wise

At first I was about to ask about which repo use as final repo between docker-agent and docker-inbound-agent, but after more reflexion (and all the work on git history), I think this repo makes more sense as the only benefit of inbound-agent repo would be some more GitHub stars and watchers.

If going that way, merging the PR will allow testing a real publication of agent and inbound-agent for this repo to validate the process.

Then, we could:

• Add a "inbound-label" to every inbound-agent repo issues
• Transfer these issues here
• (Optional) Create a last 999.999.999 empty inbound-agent release with a warning/note in the release note mentioning the move to the docker-agent repo, so users will get a notification
• Add the same kind of mention in inbound-agent README
• Archive inbound-agent repo
• (Optional) Rename this repo to reflect it contains both agent. Suggestions: "docker-agents", "docker-agent-inbound"
• Adapt jobs on ci.jenkins.io and trusted.ci.jenkins.io
  • Rename the "agent" ones
  • Remove the "inbound-agent" ones

Alternative:
Create a new repo like "docker-agent-inbound" for example, then label and transfer issues from the existing repos, then archive then. (+ optional "warning" releases).
This option is fine to me too as I would simply have to change the git reference in my local repo config to push to it.

@jenkinsci/team-docker-packaging WDYT?

[timja]

sounds good to me

[lemeurherve]

Opened #570

[lemeurherve]

@jenkinsci/team-docker-packaging now that #570 has been merged and released, is there any objection or suggestion about the following next steps plan?

• Update https://github.com/jenkinsci/docker-agent/
  • Add an "agent" label to every issue
  • Update repo description
  • [-] Rename this repo to reflect it contains both agent. Suggestions: "docker-agents", "docker-agent-inbound" See Rename this repository to reflect that it concerns both agent and inbound-agent? #749
  • Update READMEs
  • Update/fix Docker labels
  • Add a note on https://github.com/jenkinsci/docker-agent/releases/tag/3206.vb_15dcf73f6a_9-2 explaining that previous inbound-agent release notes are archived in https://github.com/jenkinsci/docker-inbound-agent/
• Adapt jobs on ci.jenkins.io and trusted.ci.jenkins.io
  • [-] Rename the "agent" ones See Rename this repository to reflect that it concerns both agent and inbound-agent? #749
    • on ci.jenkins.io
    • on trusted.ci.jenkins.io
  • Disable the "inbound-agent" ones
    • on ci.jenkins.io, by changing the Packaging folder config filter
      • from ^docker$|^docker-.*-agents?$|^docker-agent$|^winsw-maven-packaging$|^packaging$
      • to ^docker$|^docker-agent$|^docker-ssh-agent$|^docker-inbound-agents$|^winsw-maven-packaging$|^packaging$
    • on trusted.ci.jenkins.io
  • Remove the "inbound-agent" ones
    • on ci.jenkins.io
    • on trusted.ci.jenkins.io
      • Add a note in the job description pointing to here and the TTL date
      • Add a calendar event for Jenkins infra team with the TTL and the link here
      • On the TTL date remove the job
• Deprecate https://github.com/jenkinsci/docker-inbound-agent/
  • Add an "inbound-agent" label to every issue except Build both agent and inbound-agent container images in the same repository docker-inbound-agent#460 (pinned)
  • Transfer these issues here with gh issue list -s all -L 500 --json number | jq -r '.[] | .number' | grep -v 460 | xargs -I% gh issue transfer % jenkinsci/docker-agent executed from the docker-inbound-agent repo
  • Disable GitHub Actions
  • Create a last 3206.vb_15dcf73f6a_9-2 empty inbound-agent release pointing to https://github.com/jenkinsci/docker-agent/releases/tag/3206.vb_15dcf73f6a_9-2 and this repo, with a warning/note in the release note mentioning the move to the docker-agent repo, so users/bots get a notification
  • Add the same kind of mention in https://github.com/jenkinsci/docker-inbound-agent/blob/master/README.md + archiving mention with a link to this issue
  • Add a note in the repository description
  • Archive inbound-agent repo

[timja]

Makes sense

[lemeurherve]

Update:

• Issues from both repos tagged with "agent" or "inbound-agent" labels
• Docker-inbound-agent issues transferred here (sorry for the noise I didn't realize a notification would be sent for each of them)
• Docker-inbound-agent jobs disabled on ci.jenkins.io and trusted.jenkins.io (they will be removed at the end)
• Docker-inbound-agent GitHub Actions disabled
• Description of this repo updated

Next steps are blocked by this repo renaming.

As the SSH agent image hasn't the same lifecycle, no other repo is expected to be merged here.
My favorite suggestion as new name is docker-agent-inbound, I've asked for feedback about it here and in the docker and infra matrix channels but no response yet.

If there is no objection in the next day(s), I'll proceed to this renaming so pull requests like #583 can be avoided and this issue continued.

[lemeurherve]

As there are some discussions about the renaming, I'll keep the name docker-agent for now to proceed to the next steps, then I'll open a new issue dedicated to the choice of the new repo name, with the current suggestions.

[lemeurherve]

Final GitHub inbound agent release published: https://github.com/jenkinsci/docker-inbound-agent/releases/tag/3206.vb_15dcf73f6a_9-2

Deprecation notice before archival: jenkinsci/docker-inbound-agent#469

[lemeurherve]

I haven't found out how to remove https://ci.jenkins.io/job/Packaging/job/docker-inbound-agent/ from ci.jenkins.io, it's not in the parent Packaging folder filter anymore but even after a scan there is no option to delete it.
Any idea?

[lemeurherve]

Should we keep the inbound-agent job on trusted.ci.jenkins.io for history purpose? (Tag builds)

[timja]

I normally remove the jobs on disk for this sort of problem. In theory it’ll go away eventually but it’s a bit of a short coming of the organisation folder setup

[dduportal]

I normally remove the jobs on disk for this sort of problem. In theory it’ll go away eventually but it’s a bit of a short coming of the organisation folder setup

In this case, as it is trusted.ci, I recommend to disable the job to keep it, and add a TTL of 6 month;

• Add a note in the job description pointing to here and the TTL date
• Add a calendar event for Jenkins infra team with the TTL and the link here

[timja]

The question was in reference to ci.jenkins.io not trusted.ci.jenkins.io fwiw.

[lemeurherve]

• Add a note in the job description pointing to here and the TTL date
• Add a calendar event for Jenkins infra team with the TTL and the link here

Thanks, added these steps for trusted.ci.jenkins.io to the plan.