generated from berkeley-dsep-infra/hub-user-image-template
-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
6000a56
commit efa83a7
Showing
2 changed files
with
18 additions
and
98 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,82 +1,26 @@ | ||
| # hub-user-image-template :paperclip: | ||
| # dev-primary-image | ||
|
|
||
| This is a template repository for creating dedicated user images for UC Berkeley hubs. | ||
| This is the repository for the dev datahub and local user images. | ||
|
|
||
| ## Overall workflow :gear: | ||
| See this repository's [CONTRIBUTING.md](https://github.com/berkeley-dsep-infra/dev-primary-image/blob/main/CONTRIBUTING.md) for instructions. That information will eventually be migrated to docs.datahub.berkeley.edu. | ||
|
|
||
| The overall workflow is to: | ||
| # building the image locally | ||
|
|
||
| 1. Create a new repository using this one as a template. Be sure to set the owner as `berkeley-dsep-infra`. | ||
| You should use [repo2-docker](https://repo2docker.readthedocs.io/en/latest/) to build and use/test the image on your own device before you push and create a PR. It's better (and typically faster) to do this first before using CI/CD. There's no need to waste Github Action minutes to test build images when you can do this on your own device! | ||
|
|
||
| 2. Fork that repository to create your image repository (optional, but recommended). | ||
| Run `repo2docker` from inside the cloned image repo. To run on a linux/WSL2 linux shell: | ||
| ``` | ||
| repo2docker . # <--- the path to the repo | ||
| ``` | ||
|
|
||
| 3. Set the appropriate values in the Actions environment variables for `HUB` and `IMAGE`. | ||
| If you are using an ARM CPU (Apple M* silicon), you will need to run `jupyter-repo2docker` with the following arguments: | ||
|
|
||
| 4. Customize the image by editing repo2docker files in your image repository. | ||
| ``` | ||
| jupyter-repo2docker --user-id=1000 --user-name=jovyan \ | ||
| --Repo2Docker.platform=linux/amd64 \ | ||
| --target-repo-dir=/home/jovyan/.cache \ | ||
| -e PLAYWRIGHT_BROWSERS_PATH=/srv/conda \ | ||
| . # <--- the path to the repo | ||
| ``` | ||
|
|
||
| Changes can either be done by direct commits to main on your image repository, or through a pull request from a fork of your image repository. Direct commits will build the image and push it to Google Artifact Registry (GAR) on merge. PRs will also build the image and offer a link to test it using Binder (currently disabled). Merging the PR will also create and push a commit to the [datahub repo](https://github.com/berkeley-dsep-infra/datahub/), which requires a human to open a PR to merge said commit and deploy that image to the proper hub(s). | ||
|
|
||
| 5. Configure your Hub to use this new image | ||
|
|
||
| More detailed instructions are [located below](https://github.com/berkeley-dsep-infra/hub-user-image-template/#in-depth-guide). | ||
|
|
||
| ### Modifying the new image | ||
|
|
||
| Detailed instructions showing the workflow to modify an image and push it | ||
| the CI/CD workflow are located in the [contribution guide](CONTRIBUTING.md) | ||
|
|
||
| ### In-depth guide | ||
|
|
||
| Check out the 2i2c docs for an in-depth guide on how to use this template repository to create a custom user image and use it for your hub :arrow_right: https://infrastructure.2i2c.org/howto/update-env/#split-up-an-image-for-use-with-the-repo2docker-action. | ||
|
|
||
| Here's a rough guide on how to create your own fresh user image :arrow_right: https://docs.datahub.berkeley.edu/en/latest/admins/howto/new-image.html. | ||
|
|
||
| After creating a new image repo from here as a template, and bringing in the commit history (if any) of the image, you will need to set two [Github Actions Repository Variables](https://docs.github.com/en/actions/learn-github-actions/variables) for the image: `HUB` and `IMAGE`. | ||
|
|
||
| `HUB` is the short name of the hub (eg: `data100`, `datahub`, etc). | ||
| `IMAGE` is the path to the image in the Artifact Registry (eg: `ucb-datahub-2018/user-images/<hubname>-user-image`) | ||
|
|
||
| Next, you will need to give the newly created repo access to two organizational-level [secrets in the berkeley-dsep-infra repo](https://github.com/organizations/berkeley-dsep-infra/settings/secrets/actions): `GAR_SECRET_KEY` (to allow pushes to the Artifact Registry) and `DATAHUB_USER_IMAGE_BRANCH_PUSH` (to allow commits to be pushed to the [datahub](https://github.com/berkeley-dsep-infra/datahub) repo). | ||
|
|
||
| ## About this template repository :information_source: | ||
|
|
||
| This template repository enables [jupyterhub/repo2docker-action](https://github.com/jupyterhub/repo2docker-action). | ||
| This GitHub action builds a Docker image using the contents of this repo and pushes it to the [Google Artifact Registry](https://cloud.google.com/artifact-registry) registry. | ||
|
|
||
| ### The environment | ||
|
|
||
| It provides an example of a `environment.yml` conda configuration file for repo2docker to use. | ||
| This file can be used to list all the conda packages that need to be installed by `repo2docker` in your environment. | ||
| The `repo2docker-action` will update the [base repo2docker](https://github.com/jupyterhub/repo2docker/blob/HEAD/repo2docker/buildpacks/conda/environment.yml) conda environment with the packages listed in this `environment.yml` file. | ||
|
|
||
| **Note:** | ||
| A complete list of possible configuration files that can be added to the repository and be used by repo2docker to build the Docker image, can be found in the [repo2docker docs](https://repo2docker.readthedocs.io/en/latest/config_files.html#configuration-files). | ||
|
|
||
| ### Making changes to a single user server image | ||
|
|
||
| Once you've created the new image repo from this template, please refer to [the contribution instructions](CONTRIBUTING.md) located in the repo for detailed instructions. | ||
|
|
||
| ### The GitHub Action workflows | ||
|
|
||
| This template repository provides some GitHub Action workflows that can build and push the image to Google Artifact Repository when configured, and test the image on Binder. | ||
|
|
||
|  | ||
|
|
||
| #### 1. Build and test container image :arrow_right: [test.yaml](https://github.com/berkeley-dsep-infra/hub-user-image-template/blob/main/.github/workflows/test.yaml) | ||
|
|
||
| This workflow is triggered when a Pull Request is opened against the default branch (`main`).. | ||
| During PR builds, the image is **only** built and **not** pushed, unless explicitly configured to do so. | ||
|
|
||
| #### 2. Test this PR on Binder Badge :arrow_right: [binder.yaml](https://github.com/berkeley-dsep-infra/hub-user-image-template/blob/main/.github/workflows/binder.yaml.disable) | ||
|
|
||
| *Temporarily disabled* | ||
|
|
||
| Since our images are typically large and take > 10m to build, this means that Binderhub builds will currently time out. | ||
|
|
||
| This workflow posts a comment inside a pull request, every time a pull request gets opened. The comment contains a "Test this PR on Binder" badge, which can be used to access the image defined by the PR in [mybinder.org](https://mybinder.org/). | ||
|
|
||
|  | ||
|
|
||
| #### 3. Build, test and push container image :arrow_right: [build-push-open-pr.yaml](https://github.com/berkeley-dsep-infra/hub-user-image-template/blob/main/.github/workflows/build-push-image-commit.yaml) | ||
|
|
||
| After a PR is merged to `main`, this workflow builds the image again, pushes to the Artifact Registry and will create a push to the [Datahub repo](https://github.com/berkeley-dsep-infra/datahub) to update the image tag for any hubs that use this image. The PR there will need to be created manually. | ||
| If you just want to see if the image builds, but not automatically launch the server, add `--no-run` to the arguments (before the final `.`). |