Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] GoogleCloudStorageBlobStoreRepositoryTests.testSnapshotAndRestore failure #42519

Closed
matriv opened this issue May 24, 2019 · 2 comments
Closed
Assignees
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test-failure Triaged test failures from CI

Comments

@matriv
Copy link
Contributor

matriv commented May 24, 2019

Example build failure

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+7.1+intake/87/console

Reproduction line

Doesn't reproduces locally

./gradlew :plugins:repository-gcs:test --tests "org.elasticsearch.repositories.gcs.GoogleCloudStorageBlobStoreRepositoryTests.testSnapshotAndRestore" \
  -Dtests.seed=2B52BC99A06777E8 \
  -Dtests.security.manager=true \
  -Dtests.locale=ja-JP-u-ca-japanese-x-lvariant-JP \
  -Dtests.timezone=America/Creston \
  -Dcompiler.java=12 \
  -Druntime.java=8

Example relevant log:

14:11:09   1> [0001-05-24T05:11:06,556][INFO ][o.e.r.g.GoogleCloudStorageBlobStoreRepositoryTests] [testSnapshotAndRestore] -->  create random index tzbqmdctx with 465 records
14:11:09   1> [0001-05-24T05:11:06,562][INFO ][o.e.r.g.GoogleCloudStorageBlobStoreRepositoryTests] [testSnapshotAndRestore] Index [465] docs async: [false] bulk: [true] partitions [6]
14:11:09   1> [0001-05-24T05:11:06,565][INFO ][o.e.c.m.MetaDataCreateIndexService] [node_s1] [tzbqmdctx] creating index, cause [auto(bulk api)], templates [random_index_template], shards [2]/[1], mappings []
14:11:09   1> [0001-05-24T05:11:06,643][INFO ][o.e.c.m.MetaDataMappingService] [node_s1] [tzbqmdctx/I4Nzg9T7S-C3ptAChlJ3Lw] create_mapping [tzbqmdctx]
14:11:09   1> [0001-05-24T05:11:06,828][INFO ][o.e.r.g.GoogleCloudStorageBlobStoreRepositoryTests] [testSnapshotAndRestore] -->  create snapshot x:o
14:11:09   1> [0001-05-24T05:11:06,840][WARN ][o.e.s.SnapshotsService   ] [node_s1] failed to create snapshot [o/SCE0LbxMTJGiWKYoMazshA]
14:11:09   1> org.elasticsearch.snapshots.InvalidSnapshotNameException: [x:o] Invalid snapshot name [o], snapshot with the same name already exists
14:11:09   1> 	at org.elasticsearch.repositories.blobstore.BlobStoreRepository.initializeSnapshot(BlobStoreRepository.java:387) ~[elasticsearch-7.1.1-SNAPSHOT.jar:7.1.1-SNAPSHOT]
14:11:09   1> 	at org.elasticsearch.snapshots.SnapshotsService$2.doRun(SnapshotsService.java:413) [elasticsearch-7.1.1-SNAPSHOT.jar:7.1.1-SNAPSHOT]
14:11:09   1> 	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-7.1.1-SNAPSHOT.jar:7.1.1-SNAPSHOT]
14:11:09   1> 	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.1.1-SNAPSHOT.jar:7.1.1-SNAPSHOT]
14:11:09   1> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
14:11:09   1> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
14:11:09   1> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

Not muting for the moment, I found just one more failure like that.

@matriv matriv added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test-failure Triaged test failures from CI labels May 24, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@original-brownbear
Copy link
Member

The fix for this is incoming in #42511

ywelsch added a commit that referenced this issue May 24, 2019
Fixes an issue where repositories are unintentionally shared among tests (given that the repo contents is captured in a static variable on the test class, to allow "sharing" among nodes) and two tests randomly chose the same snapshot name, leading to a conflict.

Closes #42519
gurkankaymak pushed a commit to gurkankaymak/elasticsearch that referenced this issue May 27, 2019
Fixes an issue where repositories are unintentionally shared among tests (given that the repo contents is captured in a static variable on the test class, to allow "sharing" among nodes) and two tests randomly chose the same snapshot name, leading to a conflict.

Closes elastic#42519
ywelsch added a commit that referenced this issue Jun 12, 2019
Fixes an issue where repositories are unintentionally shared among tests (given that the repo contents is captured in a static variable on the test class, to allow "sharing" among nodes) and two tests randomly chose the same snapshot name, leading to a conflict.

Closes #42519
ywelsch added a commit that referenced this issue Jun 12, 2019
Fixes an issue where repositories are unintentionally shared among tests (given that the repo contents is captured in a static variable on the test class, to allow "sharing" among nodes) and two tests randomly chose the same snapshot name, leading to a conflict.

Closes #42519
ywelsch added a commit that referenced this issue Jun 12, 2019
Fixes an issue where repositories are unintentionally shared among tests (given that the repo contents is captured in a static variable on the test class, to allow "sharing" among nodes) and two tests randomly chose the same snapshot name, leading to a conflict.

Closes #42519
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test-failure Triaged test failures from CI
Projects
None yet
Development

No branches or pull requests

4 participants