Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for building Agent containers to the CI #3109

Merged

Conversation

webbnh
Copy link
Member

@webbnh webbnh commented Dec 9, 2022

This PR adds support to the CI for building a single set of Agent containers suitable for running functional tests.

The CI pipeline builds a full set of Agent containers (base, tools, tool-meister, tool-data-sink, workloads, and all) for a single distro (currently Fedora-36) and pushes them to the container registry. For PR builds, it pushes them to the internal registry; for branch builds (like main), it pushes them to the external registry.

Details:

  • Modify the Agent containers Makefile:
    • allow the image repository and the image tag to be specified by command line arguments
    • add a special target for use by the CI for pushing images to the repository: this target pushes the target distribution's container-set to the specified repository using the specified tag, without adding further complexity to the existing push script (i.e., without adding another entry in the TAG_TYPES list)
    • explicitly declare as "phony" all the phony targets
  • Modify the Jenkins Pipeline:
    • add support for referencing both internal and external container image registries
    • make use of per-stage environment directives where appropriate instead of defining variables for the entire pipeline
      rename the local references of PB_SERVER_IMAGE_TAG to PB_IMAGE_TAG so it can be used for both the Server and Agent container images; define PB_SERVER_IMAGE_TAG to the value of PB_IMAGE_TAG in the stages which need it; use a better idiom to generate its value
    • add a stage which builds the Agent container images and pushes them to the registry

PBENCH-332

@webbnh webbnh added Agent DO NOT MERGE!!! Code Infrastructure Containerization Of and relating to the process of setting up and maintaining container images Functional Testing labels Dec 9, 2022
@webbnh webbnh added this to the v0.72 milestone Dec 9, 2022
@webbnh webbnh self-assigned this Dec 9, 2022
Copy link
Member

@dbutenhof dbutenhof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You asked for review, but it's Draft: so a few comments but no cookie.

jenkins/Pipeline.gy Outdated Show resolved Hide resolved
agent/containers/images/Makefile Show resolved Hide resolved
@webbnh webbnh force-pushed the ci-build-agent-containers branch 8 times, most recently from dec5686 to 259a8e2 Compare December 10, 2022 00:44
@webbnh webbnh force-pushed the ci-build-agent-containers branch 7 times, most recently from bfe23fe to ccbde59 Compare January 13, 2023 20:34
@webbnh webbnh force-pushed the ci-build-agent-containers branch from ccbde59 to f6b2c23 Compare January 13, 2023 21:20
@webbnh webbnh marked this pull request as ready for review January 13, 2023 21:31
@webbnh webbnh requested a review from dbutenhof January 14, 2023 03:16
@webbnh
Copy link
Member Author

webbnh commented Jan 14, 2023

Ready for review, at last.

Copy link
Member

@portante portante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to know why we have to use the internal registry for container images build for PRs when there is not publishing restriction.

jenkins/Pipeline.gy Outdated Show resolved Hide resolved
dbutenhof
dbutenhof previously approved these changes Jan 16, 2023
@webbnh webbnh force-pushed the ci-build-agent-containers branch from fc0cf76 to ba4dc1e Compare January 17, 2023 19:04
@webbnh webbnh requested review from portante and dbutenhof January 17, 2023 19:37
@webbnh webbnh merged commit 0e92342 into distributed-system-analysis:main Jan 17, 2023
@webbnh webbnh deleted the ci-build-agent-containers branch January 17, 2023 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Agent Code Infrastructure Containerization Of and relating to the process of setting up and maintaining container images Functional Testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants