Skip to content
This repository has been archived by the owner on Oct 13, 2023. It is now read-only.

[19.03 backport] Testing and Jenkinsfile changes [step 1] #382

Merged
merged 59 commits into from
Sep 26, 2019

Conversation

thaJeztah
Copy link
Member

@thaJeztah thaJeztah commented Sep 25, 2019

Not yet included:

  • Migration from go-check to go-test
  • Migration from gometalinter to golangci-lint

Backports of:

Cherry-picks were clean; no conflicts

# https://github.com/moby/moby/pull/39430 Integration: remove redundant checks in IPVLAN tests
git cherry-pick -s -S -x 1e4bd2623a37b8083a7c5f8a93068be2c63f491c
git cherry-pick -s -S -x dae9bac6757af3027b06a983b8448ffa962cc071
git cherry-pick -s -S -x 93b28677bffcdaf299969c4ff1603b7d07b45fbc
git cherry-pick -s -S -x 4060a7026c0ffa2983ab1fa41d43da4937ef63c6

# https://github.com/moby/moby/pull/39431 Integration: remove redundant kernel version check for MACVlan
git cherry-pick -s -S -x 691eb142561818b0ca2255e2e340cbea6b91d80c
git cherry-pick -s -S -x 316e16618f8c794b8e56e223ed0bed15e1f4be24

# https://github.com/moby/moby/pull/39468 integration-cli: remove ExecSupport check
git cherry-pick -s -S -x 7204341950c6c8f0a66f9bb0b082217dc0ce6ddb

# https://github.com/moby/moby/pull/39469 integration-cli: remove defaultSleepImage constant
git cherry-pick -s -S -x 27f432ca57fb6d4d0409fc3c5358b74feae228cc

# https://github.com/moby/moby/pull/39470 integration-cli: remove unused requirements utils
git cherry-pick -s -S -x c887b09abc7d4ae149a9c314f74f9c351ac4cba2

# https://github.com/moby/moby/pull/39500 Add (hidden) flags to set containerd namespaces
git cherry-pick -s -S -x 24ad2f486d92681080a8e257760b047f8de2c71c

# https://github.com/moby/moby/pull/39509 integration: fix cleanup of raft data
git cherry-pick -s -S -x 6a64a4deecf5b7519d3748023765f79628059f80

# https://github.com/moby/moby/pull/39510 Don't log test initial test daemon ping failures
git cherry-pick -s -S -x 15675e28f1e3ba25f3abd099c14a7800c8bae2b8

# https://github.com/moby/moby/pull/39517 Fix Microsecond -> Millisecond.
git cherry-pick -s -S -x 5d818213ff3b9f8cda8e4fb3b071bef8fab782ad

# https://github.com/moby/moby/pull/39516 DockerSwarmSuite lock portIndex to work around race
git cherry-pick -s -S -x c096225e8ee6cb36e363ac38de4e4a566cf09918

# https://github.com/moby/moby/pull/39552 Be more conservative for Windows in TestFrequency for Splunk 
git cherry-pick -s -S -x a5c420ac54f07dbc84bcdea91b83364a25f1c921

# https://github.com/moby/moby/pull/39554 integration: run build session tests on non-experimental
git cherry-pick -s -S -x becd29c6651dffc253027e15f903ae7e7c918594

# https://github.com/moby/moby/pull/39557 integration-cli: increase healthcheck timeout
git cherry-pick -s -S -x 8c9362857f352548b94bc39c675d558b4da0d3b3

# https://github.com/moby/moby/pull/39560 Retry service updates on out of sequence errors
git cherry-pick -s -S -x 1de914695b7d0c9affc97a6da4198548da2f5f78

# https://github.com/moby/moby/pull/39569 integration-cli: remove redundant "testrequires"
git cherry-pick -s -S -x 7f37d99ef50b4046284ecb3c3b290319acf11405

# https://github.com/moby/moby/pull/39603 Better logging for swarm tests
git cherry-pick -s -S -x b0fe0dff7a5b7f70e2d08e59f4773c40e48e5c0d

# https://github.com/moby/moby/pull/39604 hack: Remove inContainer check, it wasn't useful
git cherry-pick -s -S -x f5cd8fdd446750a749182f3410adc6f1968053ca

# https://github.com/moby/moby/pull/39734 Fix some spelling mistakes
git cherry-pick -s -S -x f6226a2a56c38b790b59393ff4ce7f88f790c811

# https://github.com/moby/moby/pull/39802 integration: simplify parallel run destination
git cherry-pick -s -S -x 3262a69be6c582e4543199ddc54ebe2e306e3637

# https://github.com/moby/moby/pull/39724 Jenkinsfile: create junit.xml for integration tests
git cherry-pick -s -S -x f3be6b346f5300150dfa1de927880ea84440bb5b
git cherry-pick -s -S -x d2e00d62e244b293b5b2459942b3a55447c191f3
git cherry-pick -s -S -x f007b0150a1559ab3ba34cd81832c7c9b4e8f78f
git cherry-pick -s -S -x d723643dc38c18556ec5f7f0e97a8a594df545f7

# https://github.com/moby/moby/pull/39743 Dockerfile: update CRIU to v3.12
git cherry-pick -s -S -x 00ad0222cef8024f19394e2e9d5dc0e05f0ec982

# https://github.com/moby/moby/pull/39774 Fix testcase name
git cherry-pick -s -S -x c266d8fe563d4b1b65fb14a90da1b72f61b6310c

# https://github.com/moby/moby/pull/39800 pkg/term: refactor TestEscapeProxyRead
git cherry-pick -s -S -x 556d26c07d068d92fd896428ab4ac890554239d7

# https://github.com/moby/moby/pull/39804 integration: windows.ps1: turn defender error into a warning
git cherry-pick -s -S -x 31885181fcc0ca0cacd6c12f6c64ac553ff2bc2b

# https://github.com/moby/moby/pull/39817 TestDispatch: refactor to use subtests again, and fix linting (structcheck)
git cherry-pick -s -S -x a3f9cb5b635a76484a926289bfa4fc6feee1763a

# https://github.com/moby/moby/pull/39819 integration-cli: getContainerCount() fix trimming prefix
git cherry-pick -s -S -x 02c9b0674fe2cf43def3a1e625cab74a0db8136a

# https://github.com/moby/moby/pull/39831 Fix docker inspect for dutimgVersion
git cherry-pick -s -S -x 52a53e2587e34d45b1d4a9b7b7d307ad51c5b0b0

# https://github.com/moby/moby/pull/39836 hack/make/binary-daemon: fix some linting issues
git cherry-pick -s -S -x 70d36778252124d7cec0489ed98d1bac475d9727

# https://github.com/moby/moby/pull/39844 Fix some bashisms/non-standard comparisons in integration tests
git cherry-pick -s -S -x dbde4786e48531f095f9c3ecaff0f57b838abefc
git cherry-pick -s -S -x 32f1c651623421ee1ac480b200d34025a74436bb

# https://github.com/moby/moby/pull/39848 docker-py: skip flaky AttachContainerTest::test_attach_no_stream
git cherry-pick -s -S -x ce77a804b86886489fae47aa15a3ee5a0a8815e0

# https://github.com/moby/moby/pull/39853 TestRunInteractiveWithRestartPolicy: use `icmd.RunCmd` instead `icmd.StartCmd`
git cherry-pick -s -S -x e6fce00ec83df2f23523b836f647b8f3df97953f

# https://github.com/moby/moby/pull/39854 hack/ci/windows.ps1: explicitly set exit code to result of tests
git cherry-pick -s -S -x 8e8c52c4abe011a4cf3334da0726ef1fc0d17b14
git cherry-pick -s -S -x 7eb522a2350d759cf6a9aad493ac1b8ffc3d3335

# https://github.com/moby/moby/pull/39858 Jenkinsfile: create bundles for Windows stages
git cherry-pick -s -S -x 6ee61f54935d77d3d5667b4a3ba8f0be6c3de23d
git cherry-pick -s -S -x 1fbadd76b7ceab6e5bb683a7471f5cae8e715495

# https://github.com/moby/moby/pull/39863 Jenkinsfile hack for auto-cancellation
git cherry-pick -s -S -x 85bcc524eaa86ea5dde0c7214d15658587863a4c

# https://github.com/moby/moby/pull/39876 integration-cli: update TestCreateWithWorkdir for Hyper-V isolation
git cherry-pick -s -S -x ac9ef840ef94ff66266d3d8b9d32caf570d3b93f

# https://github.com/moby/moby/pull/39877 hack/ci/windows.ps1: add support for DOCKER_STORAGE_OPTS
git cherry-pick -s -S -x b6f596c4112818109441c84d313cf38fa06d6768

# https://github.com/moby/moby/pull/39878 hack/ci/windows.ps1: stop tailing logs after stopping the daemon
git cherry-pick -s -S -x e1636ad5fa1351c3edc20dd9006fb4d6f63c9f69

# https://github.com/moby/moby/pull/39585 Dockerfile.windows: trim .0 from Go versions
git cherry-pick -s -S -x c5bd6e3dc7680d6c683496f63dafb1f30f87eaa7

# https://github.com/moby/moby/pull/39879 hack/ci/windows.ps1: fix Go version check (due to trailing .0)
git cherry-pick -s -S -x 61450a651ba8fe4ef3cac284482a9495fc0b761d

# https://github.com/moby/moby/pull/39881 integration: TestInspect(): use swarm.RunningTasksCount
git cherry-pick -s -S -x f874f8b6fd10a180cc47d7a9829dff63233ee20c

# https://github.com/moby/moby/pull/39884 Fix Service TTY test so signal handlers work
git cherry-pick -s -S -x e6c5563ae9d083cc72fcf84796c542047ac0a299

# https://github.com/moby/moby/pull/39886 integration-cli: don't load busybox image on every daemon start
git cherry-pick -s -S -x 8fc23588f1aa14054f1134e60dd996220c595363
git cherry-pick -s -S -x ead3f4e7c8b7add7beb2de44649b38f41947180f

# https://github.com/moby/moby/pull/39887 Windows: skip flaky TestLogBlocking
git cherry-pick -s -S -x 6c75c862403c98138ab0f7811f6ba9113f9e5d61

# https://github.com/moby/moby/pull/39900 Dockerfile: remove GOMETALINTER_OPTS
git cherry-pick -s -S -x a464a3d51f89566b3e480d66903056004576cc85

# https://github.com/moby/moby/pull/39910 integration-cli: Skip TestAPIImagesSaveAndLoad on RS3 and older
git cherry-pick -s -S -x 5adaf52953476bf0ff612c8d78500e35821035bb

# https://github.com/moby/moby/pull/39917 Zap a fixed windows folder, add build number to folder inside
git cherry-pick -s -S -x 48662075434da5a13b1316df1ac7488a81e4cf0a

# https://github.com/moby/moby/pull/39968 awslogs: fix flaky TestLogBlocking unit test
git cherry-pick -s -S -x fd94bae0b8acc451a79667dd1cef4c583d532187

# https://github.com/moby/moby/pull/39984 Jenkinsfile: move integration step cleanup to amd64 where it was intended to be
git cherry-pick -s -S -x f3d8b8ae74804ba6a0386186c1c65bd1d556075a

thaJeztah and others added 30 commits September 25, 2019 17:21
These tests require a local daemon, and are not built on Windows

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 1e4bd26)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
IPVLAN no longer is experimental since 3ab093d,
so there's no need to set this option.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit dae9bac)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 93b2867)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 4060a70)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The daemon requires kernel 3.10 or up to start, so there's no need
to check if the daemon is kernel 3.8 or up.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 691eb14)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 316e166)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
All current versions of Docker support exec, so no need
to check for this.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 7204341)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Both Linux and Windows now use busybox, so no need to keep a
constant for this.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 27f432c)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Removes some test functions that were unused:

- bridgeNfIP6tables
- ambientCapabilities (added to support moby#26979, which was reverted in moby#27737)
- overlay2Supported

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit c887b09)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This allows our tests, which all share a containerd instance, to be a
bit more isolated by setting the containerd namespaces to the generated
daemon ID's rather than the default namespaces.

This came about because I found in some cases we had test daemons
failing to start (really very slow to start) because it was (seemingly)
processing events from other tests.

Signed-off-by: Brian Goff <[email protected]>
(cherry picked from commit 24ad2f4)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The directory used for storage was either changed or new directories
were added.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 6a64a4d)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This is just noise due to timing. I picked `> 2` just based on
logs from tests I've seen there's always 1 or 2.

Signed-off-by: Brian Goff <[email protected]>
(cherry picked from commit 15675e2)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
A bit too quick on the trigger on some text completion I think...

Signed-off-by: Brian Goff <[email protected]>
(cherry picked from commit 5d81821)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit c096225)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Deep Debroy <[email protected]>
(cherry picked from commit a5c420a)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The session endpoint is no longer experimental since
01c9e70, so we don't
need to start an experimental daemon.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit becd29c)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Tonis Tiigi <[email protected]>
(cherry picked from commit 8c93628)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Code retrying service update operations when receiving "update out of
sequence" errors was removed because of a misunderstanding, which has
made tests flaky. This re-adds the "CmdRetryOutOfSequence" method, and
uses it in TestSwarmPublishAdd to avoid flaky behavior.

Signed-off-by: Drew Erny <[email protected]>
(cherry picked from commit 1de9146)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The `DockerDaemonSuite.SetUpTest` already checks for Linux and a local daemon;

```
func (s *DockerDaemonSuite) SetUpTest(c *check.C) {
	testRequires(c, DaemonIsLinux, testEnv.IsLocalDaemon)
	s.d = daemon.New(c, dockerBinary, dockerdBinary, testdaemon.WithEnvironment(testEnv.Execution))
}
```

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 7f37d99)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Call helper for starting swarm agents and add some logging with daemon
id's when joining the swarm.

Signed-off-by: Brian Goff <[email protected]>
(cherry picked from commit b0fe0df)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The inContainer check isn't really useful anymore.

Even though it was said that we shouldn't rely on its existence back in
2016, we're now in 2019 and this thing still exists so we should just
rely on it now to check whether or not we're in a container.

Signed-off-by: Eli Uriegas <[email protected]>
(cherry picked from commit f5cd8fd)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: SataQiu <[email protected]>
(cherry picked from commit f6226a2)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
'Namespace' parallel runs by bind-mounting a different directory
in the container, instead of making the tests running inside
the container aware of the namespaced location.

This makes it transparent to the tests, and slightly reduces
complexity.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 3262a69)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit f3be6b3)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
…ions

Without these options set, test2json does not include a `Time`
field in the generated JSON;

    {"Action":"run","Test":"TestCgroupNamespacesBuild"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (1.70s)\n"}
    ...
    {"Action":"pass","Test":"TestCgroupNamespacesBuild"}

As a result, `gotestsum` was not able to calculate test-duration, and
reported `time="0.000000"` for all tests;

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="0.000000"></testcase>

With this patch applied:

    {"Time":"2019-08-23T22:42:41.644361357Z","Action":"run","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild"}
    {"Time":"2019-08-23T22:42:41.644367647Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Time":"2019-08-23T22:42:44.926933252Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (3.28s)\n"}
    ...
    {"Time":"2019-08-23T22:42:44.927003836Z","Action":"pass","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Elapsed":3.28}

Which now correctly reports the test's duration:

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="3.280000"></testcase>

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit d2e00d6)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Generate more unique names, based on architecture and test-suite name.

Clean up the path to this integration test to create a useful package name.
"$dir" can be either absolute (/go/src/github.com/docker/docker/integration/foo)
or relative (./integration/foo). To account for both, first we strip the
absolute path, then any leading periods and slashes.

For the package-name, we use periods as separator instead of slashes, to be more
in-line with Java package names (which is what junit.xml was originally designed
for).

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit f007b01)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit d723643)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
New features

- build CRIU with Android NDK
- C/R of
  - IP RAW sockets
  - lsm: dump and restore any SELinux process label
  - support restoring ghost files on readonly mounts

Bugfixes

 - Do not lock network if running in the host network namespace
- Fix RPC configuration file handling
- util: don't leak file descriptors to third-party tools
- small fixes here and there

Improvements

- travis: switch to the Ubuntu Xenial
- travis-ci: Enable ia32 tests
- Many improvements and bug fixes in the libcriu
  - Changes in the API and ABI (SONAME increased from 1 to 2)

full diff: checkpoint-restore/criu@v3.11...v3.12

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 00ad022)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
TestBuildMulitStageResetScratch testcase was actually meant to be
TestBuildMulitStageResetScratch

Signed-off-by: Vitaly Ostrosablin <[email protected]>
(cherry picked from commit c266d8f)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
- use subtests to make it clearer what the individual test-cases
  are, and to prevent tests from depending on values set by the
  previous test(s).
- remove redundant messages in assert (gotest.tools already prints
  a useful message if assertions fail).

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 556d26c)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
thaJeztah and others added 19 commits September 25, 2019 21:36
Trying to see if this helps with the cleanup step exiting in CI, but
Jenkins continuing to wait for the script to end afterwards.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 8e8c52c)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
…ins runs this script

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 7eb522a)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
CI already stores the logs of the test daemon, so we might as well
store them as artifacts

```
[2019-09-03T12:49:39.835Z] INFO: Tidying up at end of run
[2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.out) to C:\windows\TEMP\CIDUT.out
[2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.err) to C:\windows\TEMP\CIDUT.err
```

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 6ee61f5)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 1fbadd7)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This change will cause Jenkins to only build the
latest HEAD of a PR branch, cancelling any
previous builds that may already be in progress.
This will decrease feedback time and help mitigate
resource contention.

Signed-off-by: Peter Salvatore <[email protected]>
(cherry picked from commit 85bcc52)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Hyper-V isolated containers do not allow file-operations on a
running container. This test currently uses `docker cp` to verify
that the WORKDIR was automatically created, which cannot be done
while the container is running.

```
FAIL: docker_cli_create_test.go:302: DockerSuite.TestCreateWithWorkdir

assertion failed:
Command:  d:\CI-7\CI-f3768a669\binary\docker.exe cp foo:c:\home\foo\bar c:\tmp
ExitCode: 1
Error:    exit status 1
Stdout:
Stderr:   Error response from daemon: filesystem operations against a running Hyper-V container are not supported

Failures:
ExitCode was 1 expected 0
Expected no error
```

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit ac9ef84)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit b6f596c)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
There's already a step in  "Nuke Everything", but lets' stop it
after stopping the daemon as well

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit e1636ad)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This was an oversight when changing the Dockerfile to use a build-arg;
the Windows Dockerfile downloads the Go binaries, which never have a
trailing `.0`.

This patch makes sure that the trailing zero (if any) is removed.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit c5bd6e3)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The Windows Dockerfile downloads the Go binaries, which (unlike
the Golang images) do not have a trailing `.0` in their version.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 61450a6)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Instead of using the locally crafted `serviceContainerCount()` utility

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit f874f8b)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Noticed this test container not exiting correctly while debugging
another issue. Before this change, signals were being eaten by bash, now
they are hanlded by top. This cuts the test time in half since it
doesn't have to wait for docker to SIGKILL it.

Old:
PASS: docker_cli_swarm_test.go:840: DockerSwarmSuite.TestSwarmServiceTTY	18.997s

New:
PASS: docker_cli_swarm_test.go:840: DockerSwarmSuite.TestSwarmServiceTTY	6.293s

Signed-off-by: Brian Goff <[email protected]>
(cherry picked from commit e6c5563)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
The daemon was already created and started with the busybox
image loaded, so there's no need to load the image again.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 8fc2358)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Starting the daemon should not load the busybox image again
in most cases, so add a new `StartNodeWithBusybox` function
to be clear that this one loads the busybox image, and use
`StartNode()` for cases where loading the busybox image is
not needed.

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit ead3f4e)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This test frequently fails on Windows RS1 (mainly), so skipping it
for now on Windows;

```
ok  	github.com/docker/docker/daemon/logger	0.525s	coverage: 43.0% of statements
time="2019-09-09T20:37:35Z" level=info msg="Trying to get region from EC2 Metadata"
time="2019-09-09T20:37:36Z" level=info msg="Log stream already exists" errorCode=ResourceAlreadyExistsException logGroupName= logStreamName= message= origError="<nil>"
--- FAIL: TestLogBlocking (0.02s)
    cloudwatchlogs_test.go:313: Expected to be able to read from stream.messages but was unable to
time="2019-09-09T20:37:36Z" level=error msg=Error
time="2019-09-09T20:37:36Z" level=error msg="Failed to put log events" errorCode=InvalidSequenceTokenException logGroupName=groupName logStreamName=streamName message="use token token" origError="<nil>"
time="2019-09-09T20:37:36Z" level=error msg="Failed to put log events" errorCode=DataAlreadyAcceptedException logGroupName=groupName logStreamName=streamName message="use token token" origError="<nil>"
time="2019-09-09T20:37:36Z" level=info msg="Data already accepted, ignoring error" errorCode=DataAlreadyAcceptedException logGroupName=groupName logStreamName=streamName message="use token token"
FAIL
coverage: 78.2% of statements
FAIL	github.com/docker/docker/daemon/logger/awslogs	0.630s
```

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 6c75c86)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
This `ENV` was added to the Dockerfile in b96093f,
when the repository used per-architecture Dockerfiles, and some architectures needed
a different configuration.

Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove
this `ENV` from the Dockerfile, and set it in CI instead if needed.

Also updated the wording and fixed linting issues in hack/validate/gometalinter

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit a464a3d)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
I've seen this test fail a number of times recently on RS1

Looking at failures, the test is taking a long time ro run (491.77s, which is
more than 8 minutes), so perhaps it's just too slow on RS1, which may be
because we switch to a different base image, or because we're now running
on different machines.

Compared to RS5 (still slow, but a lot faster);

```
--- PASS: Test/DockerSuite/TestAPIImagesSaveAndLoad (146.25s)
```

```
 --- FAIL: Test/DockerSuite/TestAPIImagesSaveAndLoad (491.77s)
     cli.go:45: assertion failed:
         Command:  d:\CI-5\CI-93d2cf881\binary\docker.exe inspect --format {{.Id}} sha256:69e7c1ff23be5648c494294a3808c0ea3f78616fad67bfe3b10d3a7e2be5ff02
         ExitCode: 1
         Error:    exit status 1
         Stdout:

         Stderr:   Error: No such object: sha256:69e7c1ff23be5648c494294a3808c0ea3f78616fad67bfe3b10d3a7e2be5ff02

         Failures:
         ExitCode was 1 expected 0
         Expected no error
```

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit 5adaf52)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Signed-off-by: Stefan Scherer <[email protected]>
(cherry picked from commit 4866207)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
TestLogBlocking is intended to test that the Log method blocks by
default.  It does this by mocking out the internals of the
awslogs.logStream and replacing one of its internal channels with one
that is controlled by the test.  The call to Log occurs inside a
goroutine.  Go may or may not schedule the goroutine immediately and the
blocking may or may not be observed outside the goroutine immediately
due to decisions made by the Go runtime.  This change adds a small
timeout for test failure so that the Go runtime has the opportunity to
run the goroutine before the test fails.

Signed-off-by: Samuel Karp <[email protected]>
(cherry picked from commit fd94bae)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah thaJeztah added this to the 19.03.3 milestone Sep 25, 2019
@thaJeztah
Copy link
Member Author

Looking good; only one failure, which I think is still flaky; https://ci.docker.com/public/blue/rest/organizations/jenkins/pipelines/engine/branches/PR-382/runs/1/nodes/195/log/?start=0

[2019-09-25T20:45:37.193Z] FAIL: docker_cli_swarm_test.go:1333: DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
[2019-09-25T20:45:37.193Z] [df3b3ef329025] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [df3b3ef329025] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [df3b3ef329025] daemon started
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
[2019-09-25T20:45:37.193Z] [dac804ef559d1] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [dac804ef559d1] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [dac804ef559d1] daemon started
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] [dac804ef559d1] joining swarm manager [df3b3ef329025]@0.0.0.0:2477, swarm listen addr 0.0.0.0:2478
[2019-09-25T20:45:37.193Z] Creating a new daemon at: /go/src/github.com/docker/docker/bundles/test-integration/DockerSwarmSuite.TestSwarmClusterRotateUnlockKey
[2019-09-25T20:45:37.193Z] [da2168d1efc90] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [da2168d1efc90] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [da2168d1efc90] daemon started
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] [da2168d1efc90] joining swarm manager [df3b3ef329025]@0.0.0.0:2477, swarm listen addr 0.0.0.0:2479
[2019-09-25T20:45:37.193Z] [dac804ef559d1] Stopping daemon
[2019-09-25T20:45:37.193Z] [dac804ef559d1] exiting daemon
[2019-09-25T20:45:37.193Z] [dac804ef559d1] Daemon stopped
[2019-09-25T20:45:37.193Z] [dac804ef559d1] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [dac804ef559d1] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [dac804ef559d1] daemon started
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] [da2168d1efc90] Stopping daemon
[2019-09-25T20:45:37.193Z] [da2168d1efc90] exiting daemon
[2019-09-25T20:45:37.193Z] [da2168d1efc90] Daemon stopped
[2019-09-25T20:45:37.193Z] [da2168d1efc90] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [da2168d1efc90] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [da2168d1efc90] daemon started
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] [dac804ef559d1] Stopping daemon
[2019-09-25T20:45:37.193Z] [dac804ef559d1] exiting daemon
[2019-09-25T20:45:37.193Z] [dac804ef559d1] Daemon stopped
[2019-09-25T20:45:37.193Z] [dac804ef559d1] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [dac804ef559d1] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [dac804ef559d1] daemon started
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] [da2168d1efc90] Stopping daemon
[2019-09-25T20:45:37.193Z] [da2168d1efc90] exiting daemon
[2019-09-25T20:45:37.193Z] [da2168d1efc90] Daemon stopped
[2019-09-25T20:45:37.193Z] [da2168d1efc90] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [da2168d1efc90] waiting for daemon to start
[2019-09-25T20:45:37.193Z] [da2168d1efc90] daemon started
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] docker_cli_swarm_test.go:1413:
[2019-09-25T20:45:37.193Z]     c.Assert(err, checker.IsNil, check.Commentf("%s", outs))
[2019-09-25T20:45:37.193Z] ... value *exec.ExitError = &exec.ExitError{ProcessState:(*os.ProcessState)(0xc000937420), Stderr:[]uint8(nil)} ("exit status 1")
[2019-09-25T20:45:37.193Z] ... Error response from daemon: rpc error: code = Unknown desc = The swarm does not have a leader. It's possible that too few managers are online. Make sure more than half of the managers are online.
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] 
[2019-09-25T20:45:37.193Z] [df3b3ef329025] Stopping daemon
[2019-09-25T20:45:37.193Z] [df3b3ef329025] exiting daemon
[2019-09-25T20:45:37.193Z] [df3b3ef329025] Daemon stopped
[2019-09-25T20:45:37.193Z] [dac804ef559d1] Stopping daemon
[2019-09-25T20:45:37.193Z] [dac804ef559d1] exiting daemon
[2019-09-25T20:45:37.193Z] [dac804ef559d1] Daemon stopped
[2019-09-25T20:45:37.193Z] [da2168d1efc90] Stopping daemon
[2019-09-25T20:45:37.193Z] [da2168d1efc90] exiting daemon
[2019-09-25T20:45:37.193Z] [da2168d1efc90] Daemon stopped

yup, still flaky; moby#38885

probably should've cherry-picked moby#39885 as well, so get more debugging on that issue, but can do that in a follow-up

@tiborvass
Copy link

@thaJeztah you're missing moby#39984

@thaJeztah
Copy link
Member Author

ah, good one; think it wasn't merged yet when I created my list of things to backport; added it to #383

@thaJeztah
Copy link
Member Author

hm perhaps I need it in this one after all to get CI green (the other one is green now)

…nded to be

Signed-off-by: Tibor Vass <[email protected]>
(cherry picked from commit f3d8b8a)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
Copy link

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kolyshkin kolyshkin merged commit 9eec36e into docker-archive:19.03 Sep 26, 2019
@thaJeztah thaJeztah deleted the 19.03_backport_test_fixes branch September 26, 2019 17:43
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.