Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: setup docker_host discovery strategies properly #1161

Merged
merged 70 commits into from
May 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
f8b5e2c
chore: add helper function to check if a file exists
mdelapenya May 2, 2023
909cfe5
chore: wrap the extraction of the current docker host into a reusable…
mdelapenya May 3, 2023
927b669
chore: extract part of the algorithm to calculate the docker socket
mdelapenya May 3, 2023
0901e96
chore: use constant for default docker socket in tests
mdelapenya May 3, 2023
bdd8fed
chore: extract docker socket from Go context to a function
mdelapenya May 3, 2023
ccf4410
chore: reuse constant for default docker socket path
mdelapenya May 3, 2023
e1aa90a
chore: detect DOCKER_HOST first
mdelapenya May 3, 2023
6d3c65b
chore: do not fallback to default, return empty string
mdelapenya May 3, 2023
baa0f10
chore: apply extract docker host function
mdelapenya May 3, 2023
89e0440
chore: read properties with context
mdelapenya May 3, 2023
c2c835c
docs: document the docker host detection strategies
mdelapenya May 3, 2023
e514748
feat: look up rootless docker if docker host was not found
mdelapenya May 3, 2023
db1a6eb
chore: extract modules generator pipeline
mdelapenya May 3, 2023
2a64431
chore: add a pipeline for rootless docker
mdelapenya May 3, 2023
e78676b
chore: remove duplicated pipeline
mdelapenya May 4, 2023
31bd8eb
chore: define docekr socket path for unix/windows
mdelapenya May 4, 2023
54ab52c
chore: convert internal constants into vars for testability
mdelapenya May 4, 2023
fa7a94b
chore: use t.Cleanup to restore test state
mdelapenya May 4, 2023
8aba795
feat: support for reading the default docker host from unix/windows
mdelapenya May 4, 2023
a11fefc
chore: move config to internal package
mdelapenya May 5, 2023
1181f43
chore: unify how the docker client is retrieved
mdelapenya May 5, 2023
7fc4510
chore: delegate singleton instance to the internal config
mdelapenya May 5, 2023
7319ffb
chore: remove DOCKER_HOST env var from config
mdelapenya May 5, 2023
bca536a
chore: support resetting the config singleton for testing
mdelapenya May 5, 2023
39c81b4
feat: read docker host from properties
mdelapenya May 5, 2023
cd59f79
Merge branch 'main' into rootless-docker
mdelapenya May 11, 2023
e22c23d
fix: typo
mdelapenya May 11, 2023
280fd37
fix: remove white lines in imports
mdelapenya May 11, 2023
f5fe64e
fix: handle error in tests
mdelapenya May 11, 2023
767a2ce
chore: apply #1160 after merge
mdelapenya May 11, 2023
d91909b
chore: do not go through the provider to get a client
mdelapenya May 11, 2023
4c656e5
fix: honour passed Docker client opts
mdelapenya May 11, 2023
5d5d0bb
fix: read properties does not consider reading DOCKER_HOST env var
mdelapenya May 11, 2023
13f805f
fix: defer client close after docker ping
mdelapenya May 11, 2023
6e289a0
feat: extract the docker host when bootstrapping a docker client
mdelapenya May 12, 2023
b1e661a
chore: honour passed opts for the client
mdelapenya May 12, 2023
212c603
chore: cache docker_host to avoid unnecesary calculations
mdelapenya May 12, 2023
a8094cd
chor: convert socket schema into constants
mdelapenya May 12, 2023
56ee883
chore: support parsing URLs for remote hosts
mdelapenya May 12, 2023
6d07be4
fix: reaper mounts a socket path without schema
mdelapenya May 12, 2023
7889824
fix: proper comment structure for go linting using golangci-lint
mdelapenya May 15, 2023
1d76bb7
chore: get socket mount without schema
mdelapenya May 15, 2023
c93d355
fix: support for disabling rootless for testing
mdelapenya May 15, 2023
c456c56
fix: get provider host from the strategies
mdelapenya May 15, 2023
06b65cf
chore: proper separation of concerns between docker socket and docker…
mdelapenya May 16, 2023
6de78e2
docs: improve docs for Docker host detection
mdelapenya May 16, 2023
c88e11f
fix: comments indentation
mdelapenya May 16, 2023
77f4866
fix: proper list format in Go comments
mdelapenya May 17, 2023
eb8318d
docs: document rootless socket path alternatives in code
mdelapenya May 17, 2023
5d11e71
Merge branch 'main' into rootless-docker
mdelapenya May 19, 2023
66f1682
feat: support for testcontainers host strategy
mdelapenya May 23, 2023
271d73b
docs: include new strategy in go comment
mdelapenya May 23, 2023
804fc2f
fix: use tc.host
mdelapenya May 24, 2023
3bece2e
fix: proper initialisation of baseRunDir
mdelapenya May 24, 2023
8a75d11
fix: unset DOCKER_HOST for tests that does not need it
mdelapenya May 24, 2023
58e58b2
chore: do not print config
mdelapenya May 24, 2023
1cbe5bf
fix: include schema in the rootless docker socket path
mdelapenya May 24, 2023
cfd0aec
fix: add schema properly
mdelapenya May 24, 2023
7f552b4
fix: always prepend the schema
mdelapenya May 24, 2023
e97a308
fix: proper return when sanitising the docker socket path
mdelapenya May 24, 2023
6daff38
chore: proper use of docker socket path
mdelapenya May 24, 2023
ef70624
chore: defensively restore env var in tests
mdelapenya May 25, 2023
dfb06a5
fix: make sure the already built image is removed from the local cache
mdelapenya May 25, 2023
d042b14
fix: use testcontainers docker client when removing image
mdelapenya May 25, 2023
527e490
fix: update docs
mdelapenya May 26, 2023
6ad0f90
chore: rename variable
mdelapenya May 26, 2023
1375dc9
fix: add back module generator tests
mdelapenya May 26, 2023
c45389e
chore: do not break users for the deprecated fields in the tc config
mdelapenya May 26, 2023
222505c
Merge branch 'main' into rootless-docker
mdelapenya May 26, 2023
6d11cde
fix: tc.host takes precedence on TESTCONTAINERS_DOCKER_SOCKET_OVERRID…
mdelapenya May 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/workflows/ci-rootless-docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: Rootless Docker pipeline

on:
push:
paths-ignore:
- 'mkdocs.yml'
- 'docs/**'
- 'README.md'
pull_request:
paths-ignore:
- 'mkdocs.yml'
- 'docs/**'
- 'README.md'

concurrency:
group: "${{ github.workflow }}-${{ github.head_ref || github.sha }}"
cancel-in-progress: true

jobs:
test-rootless-docker:
strategy:
matrix:
go-version: [1.19.x, 1.x]
platform: [ubuntu-latest]
runs-on: ${{ matrix.platform }}
env:
TESTCONTAINERS_RYUK_DISABLED: "false"
steps:

- name: Setup rootless Docker
uses: ScribeMD/[email protected]

- name: Remove Docket root socket
run: sudo rm -rf /var/run/docker.sock

- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
id: go

- name: Check out code into the Go module directory
uses: actions/checkout@v3

- name: modVerify
run: go mod verify

- name: modTidy
run: make tools-tidy

- name: ensure compilation
env:
GOOS: linux
run: go build

- name: gotestsum
# only run tests on linux, there are a number of things that won't allow the tests to run on anything else
# many (maybe, all?) images used can only be build on Linux, they don't have Windows in their manifest, and
# we can't put Windows Server in "Linux Mode" in Github actions
# another, host mode is only available on Linux, and we have tests around that, do we skip them?
if: ${{ matrix.platform == 'ubuntu-latest' }}
run: make test-unit

- name: Run checker
run: |
./scripts/check_environment.sh

- name: Test Summary
uses: test-summary/action@4ee9ece4bca777a38f05c8fc578ac2007fe266f7
with:
paths: "**/TEST-*.xml"
if: always()
105 changes: 21 additions & 84 deletions config.go
Original file line number Diff line number Diff line change
@@ -1,101 +1,38 @@
package testcontainers

import (
"fmt"
"os"
"path/filepath"
"strconv"
"sync"
"context"

"github.com/magiconair/properties"
"github.com/testcontainers/testcontainers-go/internal/config"
)

var tcConfig TestcontainersConfig
var tcConfigOnce *sync.Once = new(sync.Once)

// TestcontainersConfig represents the configuration for Testcontainers
// testcontainersConfig {
type TestcontainersConfig struct {
Host string `properties:"docker.host,default="`
TLSVerify int `properties:"docker.tls.verify,default=0"`
CertPath string `properties:"docker.cert.path,default="`
RyukDisabled bool `properties:"ryuk.disabled,default=false"`
RyukPrivileged bool `properties:"ryuk.container.privileged,default=false"`
Host string `properties:"docker.host,default="` // Deprecated: use Config.Host instead
TLSVerify int `properties:"docker.tls.verify,default=0"` // Deprecated: use Config.TLSVerify instead
CertPath string `properties:"docker.cert.path,default="` // Deprecated: use Config.CertPath instead
RyukDisabled bool `properties:"ryuk.disabled,default=false"` // Deprecated: use Config.RyukDisabled instead
RyukPrivileged bool `properties:"ryuk.container.privileged,default=false"` // Deprecated: use Config.RyukPrivileged instead
Config config.Config
}

// }

// ReadConfig reads from testcontainers properties file, storing the result in a singleton instance
// of the TestcontainersConfig struct
// Deprecated use ReadConfigWithContext instead
func ReadConfig() TestcontainersConfig {
tcConfigOnce.Do(func() {
tcConfig = readConfig()

if tcConfig.RyukDisabled {
ryukDisabledMessage := `
**********************************************************************************************
Ryuk has been disabled for the current execution. This can cause unexpected behavior in your environment.
More on this: https://golang.testcontainers.org/features/garbage_collector/
**********************************************************************************************`
Logger.Printf(ryukDisabledMessage)
Logger.Printf("\n%+v", tcConfig)
}
})

return tcConfig
}

// readConfig reads from testcontainers properties file, if it exists
// it is possible that certain values get overridden when set as environment variables
func readConfig() TestcontainersConfig {
config := TestcontainersConfig{}

applyEnvironmentConfiguration := func(config TestcontainersConfig) TestcontainersConfig {
if dockerHostEnv := os.Getenv("DOCKER_HOST"); dockerHostEnv != "" {
config.Host = dockerHostEnv
}
if config.Host == "" {
config.Host = "unix:///var/run/docker.sock"
}

ryukDisabledEnv := os.Getenv("TESTCONTAINERS_RYUK_DISABLED")
if parseBool(ryukDisabledEnv) {
config.RyukDisabled = ryukDisabledEnv == "true"
}

ryukPrivilegedEnv := os.Getenv("TESTCONTAINERS_RYUK_CONTAINER_PRIVILEGED")
if parseBool(ryukPrivilegedEnv) {
config.RyukPrivileged = ryukPrivilegedEnv == "true"
}

return config
}

home, err := os.UserHomeDir()
if err != nil {
return applyEnvironmentConfiguration(config)
}

tcProp := filepath.Join(home, ".testcontainers.properties")
// init from a file
properties, err := properties.LoadFile(tcProp, properties.UTF8)
if err != nil {
return applyEnvironmentConfiguration(config)
}

if err := properties.Decode(&config); err != nil {
fmt.Printf("invalid testcontainers properties file, returning an empty Testcontainers configuration: %v\n", err)
return applyEnvironmentConfiguration(config)
}

fmt.Printf("Testcontainers properties file has been found: %s\n", tcProp)

return applyEnvironmentConfiguration(config)
return ReadConfigWithContext(context.Background())
}

func parseBool(input string) bool {
if _, err := strconv.ParseBool(input); err == nil {
return true
// ReadConfigWithContext reads from testcontainers properties file, storing the result in a singleton instance
// of the TestcontainersConfig struct
func ReadConfigWithContext(ctx context.Context) TestcontainersConfig {
cfg := config.Read(ctx)
return TestcontainersConfig{
Host: cfg.Host,
TLSVerify: cfg.TLSVerify,
CertPath: cfg.CertPath,
RyukDisabled: cfg.RyukDisabled,
RyukPrivileged: cfg.RyukPrivileged,
Config: cfg,
}
return false
}
Loading