This repository has been archived by the owner on Oct 13, 2023. It is now read-only.
forked from moby/moby
-
Notifications
You must be signed in to change notification settings - Fork 424
[19.03 backport] Testing and Jenkinsfile changes [step 1] #382
Merged
kolyshkin
merged 59 commits into
docker-archive:19.03
from
thaJeztah:19.03_backport_test_fixes
Sep 26, 2019
Merged
[19.03 backport] Testing and Jenkinsfile changes [step 1] #382
kolyshkin
merged 59 commits into
docker-archive:19.03
from
thaJeztah:19.03_backport_test_fixes
Sep 26, 2019
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These tests require a local daemon, and are not built on Windows Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 1e4bd26) Signed-off-by: Sebastiaan van Stijn <[email protected]>
IPVLAN no longer is experimental since 3ab093d, so there's no need to set this option. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit dae9bac) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 93b2867) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 4060a70) Signed-off-by: Sebastiaan van Stijn <[email protected]>
The daemon requires kernel 3.10 or up to start, so there's no need to check if the daemon is kernel 3.8 or up. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 691eb14) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 316e166) Signed-off-by: Sebastiaan van Stijn <[email protected]>
All current versions of Docker support exec, so no need to check for this. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 7204341) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Both Linux and Windows now use busybox, so no need to keep a constant for this. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 27f432c) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Removes some test functions that were unused: - bridgeNfIP6tables - ambientCapabilities (added to support moby#26979, which was reverted in moby#27737) - overlay2Supported Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit c887b09) Signed-off-by: Sebastiaan van Stijn <[email protected]>
This allows our tests, which all share a containerd instance, to be a bit more isolated by setting the containerd namespaces to the generated daemon ID's rather than the default namespaces. This came about because I found in some cases we had test daemons failing to start (really very slow to start) because it was (seemingly) processing events from other tests. Signed-off-by: Brian Goff <[email protected]> (cherry picked from commit 24ad2f4) Signed-off-by: Sebastiaan van Stijn <[email protected]>
The directory used for storage was either changed or new directories were added. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 6a64a4d) Signed-off-by: Sebastiaan van Stijn <[email protected]>
This is just noise due to timing. I picked `> 2` just based on logs from tests I've seen there's always 1 or 2. Signed-off-by: Brian Goff <[email protected]> (cherry picked from commit 15675e2) Signed-off-by: Sebastiaan van Stijn <[email protected]>
A bit too quick on the trigger on some text completion I think... Signed-off-by: Brian Goff <[email protected]> (cherry picked from commit 5d81821) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit c096225) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Deep Debroy <[email protected]> (cherry picked from commit a5c420a) Signed-off-by: Sebastiaan van Stijn <[email protected]>
The session endpoint is no longer experimental since 01c9e70, so we don't need to start an experimental daemon. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit becd29c) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Tonis Tiigi <[email protected]> (cherry picked from commit 8c93628) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Code retrying service update operations when receiving "update out of sequence" errors was removed because of a misunderstanding, which has made tests flaky. This re-adds the "CmdRetryOutOfSequence" method, and uses it in TestSwarmPublishAdd to avoid flaky behavior. Signed-off-by: Drew Erny <[email protected]> (cherry picked from commit 1de9146) Signed-off-by: Sebastiaan van Stijn <[email protected]>
The `DockerDaemonSuite.SetUpTest` already checks for Linux and a local daemon; ``` func (s *DockerDaemonSuite) SetUpTest(c *check.C) { testRequires(c, DaemonIsLinux, testEnv.IsLocalDaemon) s.d = daemon.New(c, dockerBinary, dockerdBinary, testdaemon.WithEnvironment(testEnv.Execution)) } ``` Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 7f37d99) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Call helper for starting swarm agents and add some logging with daemon id's when joining the swarm. Signed-off-by: Brian Goff <[email protected]> (cherry picked from commit b0fe0df) Signed-off-by: Sebastiaan van Stijn <[email protected]>
The inContainer check isn't really useful anymore. Even though it was said that we shouldn't rely on its existence back in 2016, we're now in 2019 and this thing still exists so we should just rely on it now to check whether or not we're in a container. Signed-off-by: Eli Uriegas <[email protected]> (cherry picked from commit f5cd8fd) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: SataQiu <[email protected]> (cherry picked from commit f6226a2) Signed-off-by: Sebastiaan van Stijn <[email protected]>
'Namespace' parallel runs by bind-mounting a different directory in the container, instead of making the tests running inside the container aware of the namespaced location. This makes it transparent to the tests, and slightly reduces complexity. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 3262a69) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit f3be6b3) Signed-off-by: Sebastiaan van Stijn <[email protected]>
…ions Without these options set, test2json does not include a `Time` field in the generated JSON; {"Action":"run","Test":"TestCgroupNamespacesBuild"} {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"=== RUN TestCgroupNamespacesBuild\n"} {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (1.70s)\n"} ... {"Action":"pass","Test":"TestCgroupNamespacesBuild"} As a result, `gotestsum` was not able to calculate test-duration, and reported `time="0.000000"` for all tests; <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="0.000000"></testcase> With this patch applied: {"Time":"2019-08-23T22:42:41.644361357Z","Action":"run","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild"} {"Time":"2019-08-23T22:42:41.644367647Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"=== RUN TestCgroupNamespacesBuild\n"} {"Time":"2019-08-23T22:42:44.926933252Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (3.28s)\n"} ... {"Time":"2019-08-23T22:42:44.927003836Z","Action":"pass","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Elapsed":3.28} Which now correctly reports the test's duration: <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="3.280000"></testcase> Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit d2e00d6) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Generate more unique names, based on architecture and test-suite name. Clean up the path to this integration test to create a useful package name. "$dir" can be either absolute (/go/src/github.com/docker/docker/integration/foo) or relative (./integration/foo). To account for both, first we strip the absolute path, then any leading periods and slashes. For the package-name, we use periods as separator instead of slashes, to be more in-line with Java package names (which is what junit.xml was originally designed for). Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit f007b01) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit d723643) Signed-off-by: Sebastiaan van Stijn <[email protected]>
New features - build CRIU with Android NDK - C/R of - IP RAW sockets - lsm: dump and restore any SELinux process label - support restoring ghost files on readonly mounts Bugfixes - Do not lock network if running in the host network namespace - Fix RPC configuration file handling - util: don't leak file descriptors to third-party tools - small fixes here and there Improvements - travis: switch to the Ubuntu Xenial - travis-ci: Enable ia32 tests - Many improvements and bug fixes in the libcriu - Changes in the API and ABI (SONAME increased from 1 to 2) full diff: checkpoint-restore/criu@v3.11...v3.12 Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 00ad022) Signed-off-by: Sebastiaan van Stijn <[email protected]>
TestBuildMulitStageResetScratch testcase was actually meant to be TestBuildMulitStageResetScratch Signed-off-by: Vitaly Ostrosablin <[email protected]> (cherry picked from commit c266d8f) Signed-off-by: Sebastiaan van Stijn <[email protected]>
- use subtests to make it clearer what the individual test-cases are, and to prevent tests from depending on values set by the previous test(s). - remove redundant messages in assert (gotest.tools already prints a useful message if assertions fail). Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 556d26c) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Trying to see if this helps with the cleanup step exiting in CI, but Jenkins continuing to wait for the script to end afterwards. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 8e8c52c) Signed-off-by: Sebastiaan van Stijn <[email protected]>
…ins runs this script Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 7eb522a) Signed-off-by: Sebastiaan van Stijn <[email protected]>
CI already stores the logs of the test daemon, so we might as well store them as artifacts ``` [2019-09-03T12:49:39.835Z] INFO: Tidying up at end of run [2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.out) to C:\windows\TEMP\CIDUT.out [2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.err) to C:\windows\TEMP\CIDUT.err ``` Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 6ee61f5) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 1fbadd7) Signed-off-by: Sebastiaan van Stijn <[email protected]>
This change will cause Jenkins to only build the latest HEAD of a PR branch, cancelling any previous builds that may already be in progress. This will decrease feedback time and help mitigate resource contention. Signed-off-by: Peter Salvatore <[email protected]> (cherry picked from commit 85bcc52) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Hyper-V isolated containers do not allow file-operations on a running container. This test currently uses `docker cp` to verify that the WORKDIR was automatically created, which cannot be done while the container is running. ``` FAIL: docker_cli_create_test.go:302: DockerSuite.TestCreateWithWorkdir assertion failed: Command: d:\CI-7\CI-f3768a669\binary\docker.exe cp foo:c:\home\foo\bar c:\tmp ExitCode: 1 Error: exit status 1 Stdout: Stderr: Error response from daemon: filesystem operations against a running Hyper-V container are not supported Failures: ExitCode was 1 expected 0 Expected no error ``` Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit ac9ef84) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit b6f596c) Signed-off-by: Sebastiaan van Stijn <[email protected]>
There's already a step in "Nuke Everything", but lets' stop it after stopping the daemon as well Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit e1636ad) Signed-off-by: Sebastiaan van Stijn <[email protected]>
This was an oversight when changing the Dockerfile to use a build-arg; the Windows Dockerfile downloads the Go binaries, which never have a trailing `.0`. This patch makes sure that the trailing zero (if any) is removed. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit c5bd6e3) Signed-off-by: Sebastiaan van Stijn <[email protected]>
The Windows Dockerfile downloads the Go binaries, which (unlike the Golang images) do not have a trailing `.0` in their version. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 61450a6) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Instead of using the locally crafted `serviceContainerCount()` utility Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit f874f8b) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Noticed this test container not exiting correctly while debugging another issue. Before this change, signals were being eaten by bash, now they are hanlded by top. This cuts the test time in half since it doesn't have to wait for docker to SIGKILL it. Old: PASS: docker_cli_swarm_test.go:840: DockerSwarmSuite.TestSwarmServiceTTY 18.997s New: PASS: docker_cli_swarm_test.go:840: DockerSwarmSuite.TestSwarmServiceTTY 6.293s Signed-off-by: Brian Goff <[email protected]> (cherry picked from commit e6c5563) Signed-off-by: Sebastiaan van Stijn <[email protected]>
The daemon was already created and started with the busybox image loaded, so there's no need to load the image again. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 8fc2358) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Starting the daemon should not load the busybox image again in most cases, so add a new `StartNodeWithBusybox` function to be clear that this one loads the busybox image, and use `StartNode()` for cases where loading the busybox image is not needed. Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit ead3f4e) Signed-off-by: Sebastiaan van Stijn <[email protected]>
This test frequently fails on Windows RS1 (mainly), so skipping it for now on Windows; ``` ok github.com/docker/docker/daemon/logger 0.525s coverage: 43.0% of statements time="2019-09-09T20:37:35Z" level=info msg="Trying to get region from EC2 Metadata" time="2019-09-09T20:37:36Z" level=info msg="Log stream already exists" errorCode=ResourceAlreadyExistsException logGroupName= logStreamName= message= origError="<nil>" --- FAIL: TestLogBlocking (0.02s) cloudwatchlogs_test.go:313: Expected to be able to read from stream.messages but was unable to time="2019-09-09T20:37:36Z" level=error msg=Error time="2019-09-09T20:37:36Z" level=error msg="Failed to put log events" errorCode=InvalidSequenceTokenException logGroupName=groupName logStreamName=streamName message="use token token" origError="<nil>" time="2019-09-09T20:37:36Z" level=error msg="Failed to put log events" errorCode=DataAlreadyAcceptedException logGroupName=groupName logStreamName=streamName message="use token token" origError="<nil>" time="2019-09-09T20:37:36Z" level=info msg="Data already accepted, ignoring error" errorCode=DataAlreadyAcceptedException logGroupName=groupName logStreamName=streamName message="use token token" FAIL coverage: 78.2% of statements FAIL github.com/docker/docker/daemon/logger/awslogs 0.630s ``` Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 6c75c86) Signed-off-by: Sebastiaan van Stijn <[email protected]>
This `ENV` was added to the Dockerfile in b96093f, when the repository used per-architecture Dockerfiles, and some architectures needed a different configuration. Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove this `ENV` from the Dockerfile, and set it in CI instead if needed. Also updated the wording and fixed linting issues in hack/validate/gometalinter Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit a464a3d) Signed-off-by: Sebastiaan van Stijn <[email protected]>
I've seen this test fail a number of times recently on RS1 Looking at failures, the test is taking a long time ro run (491.77s, which is more than 8 minutes), so perhaps it's just too slow on RS1, which may be because we switch to a different base image, or because we're now running on different machines. Compared to RS5 (still slow, but a lot faster); ``` --- PASS: Test/DockerSuite/TestAPIImagesSaveAndLoad (146.25s) ``` ``` --- FAIL: Test/DockerSuite/TestAPIImagesSaveAndLoad (491.77s) cli.go:45: assertion failed: Command: d:\CI-5\CI-93d2cf881\binary\docker.exe inspect --format {{.Id}} sha256:69e7c1ff23be5648c494294a3808c0ea3f78616fad67bfe3b10d3a7e2be5ff02 ExitCode: 1 Error: exit status 1 Stdout: Stderr: Error: No such object: sha256:69e7c1ff23be5648c494294a3808c0ea3f78616fad67bfe3b10d3a7e2be5ff02 Failures: ExitCode was 1 expected 0 Expected no error ``` Signed-off-by: Sebastiaan van Stijn <[email protected]> (cherry picked from commit 5adaf52) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Stefan Scherer <[email protected]> (cherry picked from commit 4866207) Signed-off-by: Sebastiaan van Stijn <[email protected]>
TestLogBlocking is intended to test that the Log method blocks by default. It does this by mocking out the internals of the awslogs.logStream and replacing one of its internal channels with one that is controlled by the test. The call to Log occurs inside a goroutine. Go may or may not schedule the goroutine immediately and the blocking may or may not be observed outside the goroutine immediately due to decisions made by the Go runtime. This change adds a small timeout for test failure so that the Go runtime has the opportunity to run the goroutine before the test fails. Signed-off-by: Samuel Karp <[email protected]> (cherry picked from commit fd94bae) Signed-off-by: Sebastiaan van Stijn <[email protected]>
Looking good; only one failure, which I think is still flaky; https://ci.docker.com/public/blue/rest/organizations/jenkins/pipelines/engine/branches/PR-382/runs/1/nodes/195/log/?start=0
yup, still flaky; moby#38885 probably should've cherry-picked moby#39885 as well, so get more debugging on that issue, but can do that in a follow-up |
@thaJeztah you're missing moby#39984 |
ah, good one; think it wasn't merged yet when I created my list of things to backport; added it to #383 |
hm perhaps I need it in this one after all to get CI green (the other one is green now) |
…nded to be Signed-off-by: Tibor Vass <[email protected]> (cherry picked from commit f3d8b8a) Signed-off-by: Sebastiaan van Stijn <[email protected]>
kolyshkin
approved these changes
Sep 26, 2019
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Not yet included:
Backports of:
icmd.RunCmd
insteadicmd.StartCmd
moby/moby#39853 TestRunInteractiveWithRestartPolicy: useicmd.RunCmd
insteadicmd.StartCmd
Cherry-picks were clean; no conflicts