Add Earthly configuration and use it to build the spaceros base image. #15

nuclearsandwich · 2022-07-05T22:23:26Z

This PR has been updated so Earthly completely replaces the base docker image.

This work spanned several other significant transitions for Space ROS, including a transition from ROS 2 Rolling as the foundation to ROS 2 Humble, and the more impactful change from Ubuntu 20.04 to Ubuntu 22.04 as the base platform.

The impacts of the latter change present problems building ikos and with availability of the nvidia/cudagl image in limbo right now it's unlikely there will be official 22.04-based images in the near future.

For those reasons the current image does not build or use ikos or bundle nvidia drivers.

I was always planning to use a standard Ubuntu base image for the base space-ros build and take advantage of multi-stage image builds to copy artifacts from that base image to a new demo image based on the nvidia one, however since the nvidia images aren't available for 22.04 using the rocker utility to inject nvidia support at the time its needed is a worthy alternative and I've started updating the README.md in that direction. Since the base Space ROS image doesn't have any included GUI utilities there's not much to add for this README.

Not yet in the README.md is general information on how to use earthly itself rather than just as the docker image build interface. The documentation for that is included as part of space-ros/docs#5 and if anything this README may just link to that document once it is published.

mjeronimo · 2022-07-11T17:02:25Z

@nuclearsandwich I've installed Earthly and have run through the various steps. I have a couple questions:

How does one generate a Docker image that has the build directory output (so that I can run the docker image locally and execute some tests using the static analyzers)? It seems like a docker image is only generated at the +image step which outputs an image with only the install directory.
After the +image step, I run the resulting image with "docker run -it spaceros" and get:

/ros_entrypoint.sh: line 5: /root/src/spaceros_ws/install/setup.bash: No such file or directory

mjeronimo · 2022-07-11T18:49:50Z

Worked great after that fix. I like the concept of Earthly being a combination of makefiles/Docker; it makes it pretty intuitive for me (no pun intended). One thing that wasn't entirely clear to me was when the docker image gets created. I assume that everything lives in the cache until then?

nuclearsandwich · 2022-07-17T22:07:51Z

One thing that wasn't entirely clear to me was when the docker image gets created. I assume that everything lives in the cache until then?

Earthly uses a moby project called buildkitd, which is an improved container image builder for docker (and accessible in recent docker releases as docker buildx. The Earthly buildkit daemon does not use your system's docker image cache or container framework (that is, you won't see earthly containers in docker ps or images in docker images) but it does maintain its own image cache as a docker volume attached to the buildkitd container that it will spawn on startup.

nuclearsandwich · 2022-07-17T22:15:23Z

This PR is still in draft status and one of the reasons is trying to determine whether we should start to use Earthly now or wait until the second demo with Dagger so that we don't migrate the current image build twice. I'm not sure that Dagger's tooling offers the same kind of fundamental replacement for the Dockerfile-based image builds that Earthly does (and the image building was not the Earthly feature that we were actually seeking to evaluate).

I see a few possibilities.

We could make the Earthly configuration and project its own repository and archive it if we don't move forward with it.
We could switch all our Docker image builds to Earthly knowing that it's a nice tool for the job whether or not we continue to use it for other parts of the CI/CD infra for Space ROS
We could leave this PR in draft pending a Dagger demo sometime after 15 August.

nuclearsandwich · 2022-09-16T22:15:36Z

While it is possible to reference Earthly targets in other repositories, I'm not sure how well that fits in our workflow or where the "correct" place for Earthly build files for common reference should be.

I haven't come to any conclusions yet but I have opened a similar PR in the demos repo space-ros/demos#11 which contains some common functionality.

Converting the base docker image (sans Ikos for now) to Earthly targets was fairly straightforward. The targets are chosen someone arbitrarily based on both expected semantics (i.e. `+build` for "build" and `+image` for "install" (by registering a Docker image). The TODO.md has the beginnings of a README as well as some observations from using Earthly.

Does not yet produce test results artifact.

The nvidia:cudagl images are required for doing some GPU work in builds beyond the main build of Space ROS. In the usual layered Docker approach we needed to use this as the base base image for each build but with Earthly it is even easier to move to a multi-stage build and we'll be able to pull in the install directory from this Space ROS build in other later builds which will use the cudagl base image.

There's a branch of the space-ros/space-ros repository which will regenerate and store the ros2.repos file. We can use directly asa remote earthly artifact.

New tutorials will make use of these earthly targets in a workspace that is already created rather than requiring a superdirectory containing the earthly configuration file. This may end up conflicting with building additional docker images with more contents and require some finessing but it's better for the current workflows.

They are all available on Ubuntu Jammy.

This reverts commit 53cb43f. Ikos support work will continue on the nuclearsandwich/ikos-earthly branch.

nuclearsandwich · 2022-11-08T00:10:15Z

spaceros/Earthfile

+  # Install required software development tools and ROS tools (and vim included for convenience)
+  RUN apt-get update && apt-get install -y \
+    bison \
+    build-essential \
+    cmake \
+    git \
+    python3-colcon-common-extensions \
+    python3-flake8 \
+    python3-flake8-blind-except \
+    python3-flake8-builtins \
+    python3-flake8-class-newline \
+    python3-flake8-comprehensions \
+    python3-flake8-deprecated \
+    python3-flake8-docstrings \
+    python3-flake8-import-order \
+    python3-flake8-quotes \
+    python3-pip \
+    python3-pytest \
+    python3-pytest-cov \
+    python3-pytest-repeat \
+    python3-pytest-rerunfailures \
+    python3-rosdep \
+    python3-setuptools \
+    python3-vcstool \
+    wget \
+    vim


We can replace a lot of this with ros-dev-tools and drop much of the rest once ament/ament_lint#420 is released.

nuclearsandwich · 2022-11-08T00:13:06Z

spaceros/README.md

-## Running the Space ROS Unit Tests
+## Rebuilding Space ROS and running unit tests
+
+Space ROS sources and intermediate build artifacts are not included in the docker image.

-The Space ROS unit tests are not built as part of the Docker image build. To run the unit tests in the container, execute the following command from the top of the Space ROS workspace:
+A manifest of the exact sources used to produce the current image is saved as `exact.repos` in the spaceros directory.
+To clone all sources from this manifest you can use the command sequence
+```
+spaceros-user@d10d85c68f0e:~/spaceros$ mkdir src
+spaceros-user@d10d85c68f0e:~/spaceros$ vcs import src < exact.repos
+```

+From there you can run a new build and any additional tests.
 ```
-spaceros-user@d10d85c68f0e:~/spaceros$ colcon test
+spaceros-user@d10d85c68f0e:~/spaceros$ colcon build --cmake-args -DCMAKE_BUILD_TYPE=Debug --DCMAKE_EXPORT_COMPILE_COMMANDS=ON --no-warn-unused-cli
+spaceros-user@d10d85c68f0e:~/spaceros$ colcon test --ctest-args -LE "(ikos|xfail)" --pytest-args -m "not xfail"


@mjeronimo this whole section may be a bit of an overstep for this README. I think moving it to the docs site either in space-ros/docs#5 or another tutorial specifically about the docker image may make sense.

If we do want to leave it here, I updated the colcon test command to match the args used during automated builds (minus retests).

We can work out the proper split when we update the documentation site. I've been thinking about this topic and what information goes where. Good to have it here for now and we can refactor.

space-ros/docker#15) * Add Earthfile for building Space ROS base image. Converting the base docker image (sans Ikos for now) to Earthly targets was fairly straightforward. The targets are chosen someone arbitrarily based on both expected semantics (i.e. `+build` for "build" and `+image` for "install" (by registering a Docker image). * Add build-testing target to build and run colcon tests. Does not yet produce test results artifact. * Use ubuntu:focal as default base image. The nvidia:cudagl images are required for doing some GPU work in builds beyond the main build of Space ROS. In the usual layered Docker approach we needed to use this as the base base image for each build but with Earthly it is even easier to move to a multi-stage build and we'll be able to pull in the install directory from this Space ROS build in other later builds which will use the cudagl base image. * Use a remote artifact for the repos file. There's a branch of the space-ros/space-ros repository which will regenerate and store the ros2.repos file. We can use directly asa remote earthly artifact. * Don't change artifact paths. * Switch base image to Ubuntu Jammy. * Save artifacts directly to current directory instead of subdirectory. New tutorials will make use of these earthly targets in a workspace that is already created rather than requiring a superdirectory containing the earthly configuration file. This may end up conflicting with building additional docker images with more contents and require some finessing but it's better for the current workflows. * Add trailing slash to copy to src directory. * Add Earthly workflow for building the docker image. * Ignore immediate src, install, and log subdirectories. * Add job for uploading build results archive. * Fix earthly download url. * Add Earthfile artifact for bootstrapping. * Fix spelling of COPY directive. * Set up sources locally. * Fix entrypoint name. * Allow sources to supply output for the next step. * Install all dependencies using apt. They are all available on Ubuntu Jammy. * Use the same directory structure as the existing Docker image. * Save the install directory from a build-testing run. * Add labels from base docker image. * Start to add ikos support. * Revert "Start to add ikos support." This reverts commit 3f2709a8eb0ade84d35e8110b30240521722843a. Ikos support work will continue on the nuclearsandwich/ikos-earthly branch. * Update image tag. * Support VCS_REF arg from Docker image build. * Use the current HEAD as the default VCS ref. * Use earthly to build the base Space ROS image. * Omit GPU arguments and update image / repository name. * Remove unused Dockerfile and corresponding GitHub workflow. * Add push step when the build is on main. * Start updating README.md for Earthly workflow. * Clean and re-fetch sources in automatic build script.

… ROS (space-ros/docker#15) * Add system dependencies directly Signed-off-by: Michael Jeronimo <[email protected]> * Create Dockerfiles for MoveIt2 and the MoveIt2 tutorials Signed-off-by: Michael Jeronimo <[email protected]> * Add a few more missing dependencies Signed-off-by: Michael Jeronimo <[email protected]> * Add the clang-tidy tool Signed-off-by: Michael Jeronimo <[email protected]> * Remove a stray .vscode directory and add a firefox dependency (included for testing GUIs from a docker image) Signed-off-by: Michael Jeronimo <[email protected]> * Add a few more missing libraries found when running tests Signed-off-by: Michael Jeronimo <[email protected]> * Add xterm to the MoveIt2 install since it's needed by the demos * Fetch the ros2.repos file from ros2/ros2space-ros/docker#spaceros Signed-off-by: Michael Jeronimo <[email protected]> * Add content to the docker README files Signed-off-by: Michael Jeronimo <[email protected]> * Use rosdep for Space ROS and MoveIt2 system dependencies Signed-off-by: Michael Jeronimo <[email protected]> * Remove the 'joy' repos to avoid a compile-time error with the latest source. The repo isn't needed currently to run the MoveIt2 MoveGroup demo. Signed-off-by: Michael Jeronimo <[email protected]> * Fix typo in README Signed-off-by: Michael Jeronimo <[email protected]> * Build IKOS in the spaceros docker image and add info to the README Signed-off-by: Michael Jeronimo <[email protected]> * Address review feedback * Remove extra WORKDIR statements * Remove --symlink-install Also, * Add new dependencies because of recent source changes * Remove ros_testing as it is now in ros2.repos for spaceros Signed-off-by: Michael Jeronimo <[email protected]>

space-ros/docker#15) * Add Earthfile for building Space ROS base image. Converting the base docker image (sans Ikos for now) to Earthly targets was fairly straightforward. The targets are chosen someone arbitrarily based on both expected semantics (i.e. `+build` for "build" and `+image` for "install" (by registering a Docker image). * Add build-testing target to build and run colcon tests. Does not yet produce test results artifact. * Use ubuntu:focal as default base image. The nvidia:cudagl images are required for doing some GPU work in builds beyond the main build of Space ROS. In the usual layered Docker approach we needed to use this as the base base image for each build but with Earthly it is even easier to move to a multi-stage build and we'll be able to pull in the install directory from this Space ROS build in other later builds which will use the cudagl base image. * Use a remote artifact for the repos file. There's a branch of the space-ros/space-ros repository which will regenerate and store the ros2.repos file. We can use directly asa remote earthly artifact. * Don't change artifact paths. * Switch base image to Ubuntu Jammy. * Save artifacts directly to current directory instead of subdirectory. New tutorials will make use of these earthly targets in a workspace that is already created rather than requiring a superdirectory containing the earthly configuration file. This may end up conflicting with building additional docker images with more contents and require some finessing but it's better for the current workflows. * Add trailing slash to copy to src directory. * Add Earthly workflow for building the docker image. * Ignore immediate src, install, and log subdirectories. * Add job for uploading build results archive. * Fix earthly download url. * Add Earthfile artifact for bootstrapping. * Fix spelling of COPY directive. * Set up sources locally. * Fix entrypoint name. * Allow sources to supply output for the next step. * Install all dependencies using apt. They are all available on Ubuntu Jammy. * Use the same directory structure as the existing Docker image. * Save the install directory from a build-testing run. * Add labels from base docker image. * Start to add ikos support. * Revert "Start to add ikos support." This reverts commit 3f2709a8eb0ade84d35e8110b30240521722843a. Ikos support work will continue on the nuclearsandwich/ikos-earthly branch. * Update image tag. * Support VCS_REF arg from Docker image build. * Use the current HEAD as the default VCS ref. * Use earthly to build the base Space ROS image. * Omit GPU arguments and update image / repository name. * Remove unused Dockerfile and corresponding GitHub workflow. * Add push step when the build is on main. * Start updating README.md for Earthly workflow. * Clean and re-fetch sources in automatic build script.

nuclearsandwich force-pushed the nuclearsandwich/earthly-prototyping branch from 64e6fad to 2df0a05 Compare October 6, 2022 18:31

nuclearsandwich marked this pull request as ready for review October 6, 2022 18:31

nuclearsandwich marked this pull request as draft October 6, 2022 18:40

nuclearsandwich force-pushed the nuclearsandwich/earthly-prototyping branch from eb945b1 to 3c2f093 Compare November 1, 2022 21:24

nuclearsandwich marked this pull request as ready for review November 4, 2022 00:07

nuclearsandwich added 20 commits November 7, 2022 14:49

Add build-testing target to build and run colcon tests.

18f61fd

Does not yet produce test results artifact.

Use a remote artifact for the repos file.

57be1a7

There's a branch of the space-ros/space-ros repository which will regenerate and store the ros2.repos file. We can use directly asa remote earthly artifact.

Don't change artifact paths.

9e7e43f

WIPWIP adding build result archives.

0b45b71

Switch base image to Ubuntu Jammy.

f3167af

Add trailing slash to copy to src directory.

79c319f

Add Earthly workflow for building the docker image.

89d6929

Ignore immediate src, install, and log subdirectories.

5329f28

Add job for uploading build results archive.

f4c453f

Fix earthly download url.

09dddcd

Add Earthfile artifact for bootstrapping.

71194cf

Fix spelling of COPY directive.

b8d7dfd

Set up sources locally.

735185e

Fix entrypoint name.

3972391

Allow sources to supply output for the next step.

cf4c16e

Install all dependencies using apt.

2a28324

They are all available on Ubuntu Jammy.

Use the same directory structure as the existing Docker image.

28568a3

nuclearsandwich added 11 commits November 7, 2022 14:51

Save the install directory from a build-testing run.

0b796f0

Add labels from base docker image.

3afc9e3

Start to add ikos support.

20c7cb7

Revert "Start to add ikos support."

7e04a40

This reverts commit 53cb43f. Ikos support work will continue on the nuclearsandwich/ikos-earthly branch.

Update image tag.

1105ddc

Support VCS_REF arg from Docker image build.

63aeafd

Use the current HEAD as the default VCS ref.

c036caa

Use earthly to build the base Space ROS image.

ecac7f5

Omit GPU arguments and update image / repository name.

9798c56

Remove unused Dockerfile and corresponding GitHub workflow.

5af1296

Add push step when the build is on main.

3f37ef9

nuclearsandwich force-pushed the nuclearsandwich/earthly-prototyping branch from 226142d to 3f37ef9 Compare November 7, 2022 22:51

nuclearsandwich changed the title ~~Add Earthfile prototype config for building Space ROS base image.~~ Add Earthly configuration and use it to build the spaceros base image. Nov 7, 2022

nuclearsandwich added 3 commits November 7, 2022 15:36

Start updating README.md for Earthly workflow.

9f0efb9

This file has been replaced by the README.md and build.sh scripts.

8ba23f5

Clean and re-fetch sources in automatic build script.

5d0b373

nuclearsandwich commented Nov 8, 2022

View reviewed changes

nuclearsandwich requested a review from mjeronimo November 8, 2022 00:14

mjeronimo approved these changes Nov 8, 2022

View reviewed changes

nuclearsandwich merged commit 9f514d7 into main Nov 9, 2022

nuclearsandwich mentioned this pull request May 16, 2023

Port Space ROS CI builds to Ubuntu Jammy space-ros/space-ros#38

Closed

ivanperez-keera deleted the nuclearsandwich/earthly-prototyping branch November 2, 2023 13:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Earthly configuration and use it to build the spaceros base image. #15

Add Earthly configuration and use it to build the spaceros base image. #15

nuclearsandwich commented Jul 5, 2022 •

edited

Loading

mjeronimo commented Jul 11, 2022

mjeronimo commented Jul 11, 2022

nuclearsandwich commented Jul 17, 2022

nuclearsandwich commented Jul 17, 2022

nuclearsandwich commented Sep 16, 2022

nuclearsandwich Nov 8, 2022

nuclearsandwich Nov 8, 2022

mjeronimo Nov 8, 2022

Add Earthly configuration and use it to build the spaceros base image. #15

Add Earthly configuration and use it to build the spaceros base image. #15

Conversation

nuclearsandwich commented Jul 5, 2022 • edited Loading

mjeronimo commented Jul 11, 2022

mjeronimo commented Jul 11, 2022

nuclearsandwich commented Jul 17, 2022

nuclearsandwich commented Jul 17, 2022

nuclearsandwich commented Sep 16, 2022

nuclearsandwich Nov 8, 2022

Choose a reason for hiding this comment

nuclearsandwich Nov 8, 2022

Choose a reason for hiding this comment

mjeronimo Nov 8, 2022

Choose a reason for hiding this comment

nuclearsandwich commented Jul 5, 2022 •

edited

Loading