Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

env_docker: make --cpus customizable #68

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,23 @@ In the profiling UI choose a profile type, filter by instances (autocompleted) a

For runnables that are both instrumented and profiled you can use [`e2eobs.AsObservable`](observable/observable.go).

### Debugging flaky tests

Sometimes tests might fail due to timing problems on highly CPU constrained systems such as GitHub actions. To facilitate fixing these issues, `e2e` supports limiting CPU time allocated to Docker containers through `E2E_DOCKER_CPUS` environment variable:

```go mdox-exec="sed -n '285,288p' env_docker.go"
dockerCPUsEnv := os.Getenv(dockerCPUEnvName)
if dockerCPUsEnv != "" {
dockerCPUsParam = dockerCPUsEnv
}
```

You can set it either through command line parameters or `t.Setenv("E2E_DOCKER_CPUS", "...")`.

Alternatively, you could pass `WithCPUs` through environment options so that some e2e test would have permanently reduced available CPU time.

See what values you can pass to the `--cpus` flag on [Docker website](https://docs.docker.com/config/containers/resource_constraints/#configure-the-default-cfs-scheduler).

### Troubleshooting

#### Can't create docker network
Expand Down
7 changes: 7 additions & 0 deletions env.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,13 @@ type environmentOptions struct {
name string

volumes []string
cpus string
}

func WithCPUs(cpus string) EnvironmentOption {
return func(o *environmentOptions) {
o.cpus = cpus
}
}

// WithLogger tells environment to use custom logger to default one (stdout).
Expand Down
18 changes: 18 additions & 0 deletions env_docker.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ type DockerEnvironment struct {

hostAddr string
dockerVolumes []string
cpus string

registered map[string]struct{}
listeners []EnvironmentListener
Expand Down Expand Up @@ -113,6 +114,7 @@ func New(opts ...EnvironmentOption) (_ *DockerEnvironment, err error) {
verbose: e.verbose,
registered: map[string]struct{}{},
dockerVolumes: e.volumes,
cpus: e.cpus,
}

// Force a shutdown in order to cleanup from a spurious situation in case
Expand Down Expand Up @@ -269,12 +271,28 @@ func (e *DockerEnvironment) SharedDir() string {
return e.dir
}

const dockerCPUEnvName = "E2E_DOCKER_CPUS"

func (e *DockerEnvironment) buildDockerRunArgs(name string, ports map[string]int, opts StartOptions) []string {
args := []string{"--rm", "--net=" + e.networkName, "--name=" + dockerNetworkContainerHost(e.networkName, name), "--hostname=" + name}

// Mount the docker env working directory into the container. It's shared across all containers to allow easier scenarios.
args = append(args, "-v", fmt.Sprintf("%s:%s:z", e.dir, e.dir))

// Allow reducing available CPU Time via environment variables or
// environment parameters. The latter takes precedence.
dockerCPUsParam := ""
dockerCPUsEnv := os.Getenv(dockerCPUEnvName)
if dockerCPUsEnv != "" {
dockerCPUsParam = dockerCPUsEnv
}
if e.cpus != "" {
dockerCPUsParam = e.cpus
}
if dockerCPUsParam != "" {
args = append(args, "--cpus", dockerCPUsParam)
}

for _, v := range e.dockerVolumes {
args = append(args, "-v", v)
}
Expand Down