-
-
Notifications
You must be signed in to change notification settings - Fork 512
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: setup docker_host discovery strategies properly #1161
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mdelapenya
added
the
feature
New functionality or new behaviors on the existing one
label
May 9, 2023
$XDG_RUNTIME_DIR does not contain it
* main: (22 commits) docs: document the Go version (testcontainers#1246) chore(deps): bump github.com/aws/aws-sdk-go-v2/config (testcontainers#1222) chore(deps): bump golang.org/x/sys from 0.7.0 to 0.8.0 (testcontainers#1202) chore(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (testcontainers#1232) chore(deps): bump cloud.google.com/go/spanner in /examples/spanner (testcontainers#1226) chore: Removes the refercence about docker 22.06 from the docker-compose docs and updates the replace directive for the compose module. (testcontainers#1243) chore(deps): bump go.mongodb.org/mongo-driver in /examples/mongodb (testcontainers#1233) chore(deps): bump google.golang.org/api from 0.123.0 to 0.124.0 in /examples (testcontainers#1244) chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 (testcontainers#1241) chore(deps): bump github.com/aws/aws-sdk-go in /modules/localstack (testcontainers#1242) added NATS JetStream example (testcontainers#1190) chore(deps): bump github.com/imdario/mergo from 0.3.12 to 0.3.15 (testcontainers#1204) chore(deps): bump cloud.google.com/go/firestore in /examples/firestore (testcontainers#1216) chore(deps): bump k8s.io/client-go from 0.22.5 to 0.27.2 in /modules/k3s (testcontainers#1211) chore(deps): bump go.mongodb.org/mongo-driver in /examples/mongodb (testcontainers#1092) chore(deps): bump github.com/neo4j/neo4j-go-driver/v5 in /modules/neo4j (testcontainers#1206) chore(deps): bump github.com/twmb/franz-go in /modules/redpanda (testcontainers#1201) chore(deps): bump google.golang.org/api from 0.121.0 to 0.123.0 in /examples (testcontainers#1229) chore(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (testcontainers#1228) ...
…E for the docker socket
Kudos, SonarCloud Quality Gate passed! 0 Bugs No Coverage information |
This was referenced Jun 13, 2023
mdelapenya
added a commit
to mdelapenya/testcontainers-go
that referenced
this pull request
Jun 15, 2023
It's not needed: we added it in an intermediate state of testcontainers#1161
mdelapenya
added a commit
that referenced
this pull request
Jun 15, 2023
* chore: simplify ReadConfig signature, not passing context It's not needed: we added it in an intermediate state of #1161 * fix: more usages
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR does a lot of internal refactoring to accomplish a proper bootstrapping of the Docker client provided by the library.
Docker host and Docker socket path extraction
We have defined an algorithm to locate the Docker host in a well-defined manner, using an array of Go functions that will be executed in a particular order. More concretely, for discovering the Docker host, we are adding functions that extracts the Docker host from the following locations:
The algorithm will execute each of the above functions in that particular order, returning the Docker host location if and only if the function returns a non-empty string and a nil error. In the case a function returns an error, it's wrapped into an outer error, so that it will print all the locations that were checked in the case it's needed.
The first function, the one with most precedence, checks for the
tc.host
key in the testcontainers properties file, which is located at $HOME/.testcontainers.properties.The last function checks for rootless-docker. This PR adds support for rootless Docker, checking for the Docker host in the following locations:
${XDG_RUNTIME_DIR}/.docker/run/docker.sock
.${HOME}/.docker/run/docker.sock
.${HOME}/.docker/desktop/docker.sock
./run/user/${UID}/docker.sock
, where${UID}
is the user ID of the current user.The code will use the same pattern as above: defining functions for each location that will be executed in the aforementioned order, wrapping an outer error if needed. When returned, it will include the
unix://
schema as prefix of the Docker host.At the same time, this PR is adding a way to detect the path of the Docker socket, which will basically use the above mechanism but including certain special checks:
/var/run/docker-alt.sock
unix:///var/run/docker.sock
->/var/run/docker.sock
)/var/run/docker.sock
When returning the Docker socket path, it will remove the
unix://
schema from the path. It's important to mention that at this moment we are not supporting Windows containers at the moment, so nonpipe
schema is added (although we have it in mind).When extracting both the Docker host and the Docker socket path, it happens once: we have defined a sync.Once for each location so that the discovery is not happening for every call, but instead cached during the entire execution of the process.
For both scenarios, we have included a deep battery of unit tests, simulating the conditions each location needs: e.g. setting the env vars for HOME, XDG_RUNTIME_DIR to temporary dirs and creating the socket files in there.
We have also added docs to the website regarding this new discovery mechanism.
Docker rootless
In order to provided a seamless experience when running Testcontainers for Go with Docker rootless, and leveraging the above mechanism to discover the host and the socket, we have added a GH action that configures Docker in rootless-mode for the GH worker. There, we are running the tests to verify the library behaves as expected when it's run in rootless. This pipeline will run for each PR and merge commit.
Side works
Reading TestcontainersConfig
We have pushed certain core functionality to the internal packages so that it's not possible to use it from outside the library. A clear example is the initialisation of the Testcontainers config struct.
This has been needed as part of the refactors to bootstrap the Docker client, which needs a Docker host, which would eventually check the Testcontainers properties file. As a consequence, the public API for reading the config is still present, although will call the internal package to get it. Besides, the
TestcontainersConfig
deprecates all its attributes and defines a new one as an instance of the internal representation of the Config.In the past, reading the properties also calculated the Docker host, now the responsibilities are properly decoupled and it only reads the properties file. The DOCKER_HOST variable is read with the aforementioned discovery mechanism, using this properties as an example.
DockerProvider struct and the Docker host
The struct holds a reference to the host, and it was calculated reading from the properties file (which read from DOCKER_HOST). Now with the consistent discovery mechanism, the provider extracts the Docker host and directly uses it.
Getting the default Docker registry from the Docker client
Now that we have an unified manner of getting the Docker client, the code to extract the default Docker registry does not need to instantiate a Provider struct, but instead simply get a Testcontainers Docker client.
Reaper mounts
Another consequence of having a proper calculation of the Docker socket path is when mounting the Docker socket into certain containers. Ryuk, our resource reaper, needs the Docker socket to talk to the running containers. We are extracting the Docker socket path instead of hardcoding it to
/var/run/docker.sock
Integrated tests detection
While debugging rootless, we have detected a few integrated tests, or tests that were affected by the state produced by other tests that run before. For that reason, we have updated a few tests in order to make it work. As an example, the most illustrative case was a test for checking the Docker Auths with a private registry: testA was using the right auths to connect to the private registry and pull an image, testB should do the same but using wrong auth credentials therefore raising an error. For some reason, in rootfull-mode testB raised an error; but in rootless, testB detected that the image was already cached locally and used it, not raising the expected error. 🤷 We updated the test to always delete the image from local after testA and testB.
Why is it important?
We want to provide a consistent experience when using the library and interacting with the Docker client, for that reason we are unifying how it's bootstrapped and how it discovers the Docker host and the Docker socket from the different alternatives described above.
Related issues
How to test this PR
Well, the CI is defining the scenarios to test this PR, using both rootless and rootfull.