Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor environment variable processing for Docker #49612

Merged
merged 17 commits into from
Dec 16, 2019
Merged

Refactor environment variable processing for Docker #49612

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
133ebf2
Refactor env var processing
pugnascotia Nov 22, 2019
2261af3
Remove outdated comment
pugnascotia Nov 27, 2019
f929105
Revert CronEvalTool superclass change
pugnascotia Nov 27, 2019
3f7d287
Fix test after removing elasticsearch-enve
pugnascotia Nov 27, 2019
fcec60d
Allow -E in MultiCommand tools
pugnascotia Nov 28, 2019
31e4679
Pass along -E arguments to subcommands
pugnascotia Nov 29, 2019
4bf7ba5
Checkstyle
pugnascotia Nov 29, 2019
0d8b8dd
Merge remote-tracking branch 'upstream/master' into 45223-refactor-en…
pugnascotia Nov 29, 2019
9061828
Restrict new packaging test to default distro
pugnascotia Nov 29, 2019
569841a
Merge remote-tracking branch 'upstream/master' into 45223-refactor-en…
pugnascotia Dec 3, 2019
e43283e
Override ES cgroup setting in elasticsearch-env
pugnascotia Dec 3, 2019
35487e9
Remove redundant -E flags from docker exec call
pugnascotia Dec 5, 2019
c25e6ea
Add zip / unzip to the Docker image
pugnascotia Dec 9, 2019
af04f13
Merge remote-tracking branch 'upstream/master' into 45223-refactor-en…
pugnascotia Dec 9, 2019
329d7f9
Merge remote-tracking branch 'upstream/master' into 45223-refactor-en…
pugnascotia Dec 12, 2019
6064ed1
Check that zip and unzip are installed in Docker
pugnascotia Dec 12, 2019
2c61f08
Use rpm to check for package installation
pugnascotia Dec 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions distribution/docker/src/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
FROM ${base_image} AS builder

RUN for iter in {1..10}; do ${package_manager} update --setopt=tsflags=nodocs -y && \
${package_manager} install --setopt=tsflags=nodocs -y gzip shadow-utils tar unzip zip && \
${package_manager} install --setopt=tsflags=nodocs -y gzip shadow-utils tar && \
${package_manager} clean all && exit_code=0 && break || exit_code=\$? && echo "${package_manager} error: retry \$iter in 10s" && sleep 10; done; \
(exit \$exit_code)

Expand Down Expand Up @@ -46,7 +46,7 @@ FROM ${base_image}
ENV ELASTIC_CONTAINER true

RUN for iter in {1..10}; do ${package_manager} update --setopt=tsflags=nodocs -y && \
${package_manager} install --setopt=tsflags=nodocs -y nc shadow-utils && \
${package_manager} install --setopt=tsflags=nodocs -y nc shadow-utils zip unzip && \
${package_manager} clean all && exit_code=0 && break || exit_code=\$? && echo "${package_manager} error: retry \$iter in 10s" && sleep 10; done; \
(exit \$exit_code)

Expand Down
15 changes: 13 additions & 2 deletions qa/os/src/test/java/org/elasticsearch/packaging/util/Docker.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import java.nio.file.attribute.PosixFileAttributes;
import java.nio.file.attribute.PosixFilePermission;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
Expand All @@ -46,10 +47,12 @@
import static org.elasticsearch.packaging.util.FileMatcher.p775;
import static org.elasticsearch.packaging.util.FileUtils.getCurrentVersion;
import static org.elasticsearch.packaging.util.ServerUtils.makeRequest;
import static org.hamcrest.CoreMatchers.containsString;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;

/**
Expand Down Expand Up @@ -276,7 +279,7 @@ public static class DockerShell extends Shell {
protected String[] getScriptCommand(String script) {
assert containerId != null;

return super.getScriptCommand("docker exec " + "--user elasticsearch:root " + "--tty " + containerId + " " + script);
return super.getScriptCommand("docker exec --user elasticsearch:root --tty " + containerId + " " + script);
}
}

Expand Down Expand Up @@ -445,6 +448,14 @@ private static void verifyOssInstallation(Installation es) {
).forEach(executable -> assertPermissionsAndOwnership(es.bin(executable), p755));

Stream.of("LICENSE.txt", "NOTICE.txt", "README.textile").forEach(doc -> assertPermissionsAndOwnership(es.home.resolve(doc), p644));

// These are installed to help users who are working with certificates.
Stream.of("zip", "unzip").forEach(cliTool -> {
// `which` isn't installed in our image, so instead just try to call the command.
final Shell.Result result = dockerShell.runIgnoreExitCode(cliTool + " -h");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In bash you can also rely on type -P <thefile> if you don't have which to ensure the file is in $PATH:

e.g. on the Ubi image:

[elasticsearch@0f6edc7f150a ~]$ type -P zip
[elasticsearch@0f6edc7f150a ~]$ echo $?
1
[elasticsearch@0f6edc7f150a ~]$ type -P ls
/usr/bin/ls
[elasticsearch@0f6edc7f150a ~]$ echo $?
0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type -P would work, but as it's a bash builtin, I'd have to run it through bash. That's a actually a bit cumbersome, because we're already running "bash -c 'docker blah blah commmand'". Instead, I'm now using rpm to check if the package is installed.


assertTrue(cliTool + " ought to be installed. " + result, result.isSuccess());
});
}

private static void verifyDefaultInstallation(Installation es) {
Expand Down