Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky CQL test: testReindexingForEdgeIndex #3142

Open
FlorianHockmann opened this issue Jul 12, 2022 · 1 comment
Open

Flaky CQL test: testReindexingForEdgeIndex #3142

FlorianHockmann opened this issue Jul 12, 2022 · 1 comment
Labels
storage/cassandra testing/flaky

Comments

@FlorianHockmann
Copy link
Member

Flaky Test

Stack Trace

The error is:

java.lang.IllegalArgumentException: Update action [ENABLE_INDEX] cannot be invoked for index with status [INSTALLED]

and there are several stack traces like this one logged just before:

org.janusgraph.core.JanusGraphException: Could not commit transaction due to exception during persistence
	at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1538)
	at org.janusgraph.graphdb.JanusGraphTest$4.run(JanusGraphTest.java:3192)
Caused by: org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention
	at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:324)
	at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.acquireLock(ExpectedValueCheckingStore.java:102)
	at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.acquireLock(KCVSProxy.java:52)
	at org.janusgraph.diskstorage.BackendTransaction.acquireIndexLock(BackendTransaction.java:258)
	at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:636)
	at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:776)
	at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1527)
	... 1 more
org.janusgraph.core.JanusGraphException: Could not commit transaction due to exception during persistence
	at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1538)
	at org.janusgraph.graphdb.JanusGraphTest$4.run(JanusGraphTest.java:3192)
Caused by: org.janusgraph.diskstorage.locking.PermanentLockingException: Local lock contention
	at org.janusgraph.diskstorage.locking.AbstractLocker.writeLock(AbstractLocker.java:324)
	at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.acquireLock(ExpectedValueCheckingStore.java:102)
	at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.acquireLock(KCVSProxy.java:52)
	at org.janusgraph.diskstorage.BackendTransaction.acquireIndexLock(BackendTransaction.java:258)
	at org.janusgraph.graphdb.database.StandardJanusGraph.prepareCommit(StandardJanusGraph.java:636)
	at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:776)
	at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1527)
	... 1 more
@rngcntr rngcntr mentioned this issue Dec 14, 2022
Merged
9 tasks
@FlorianHockmann
Copy link
Member Author

Also flaky for CQLGraphCacheTest:

Error:  Errors: 
Error:    CQLGraphCacheTest>JanusGraphTest.testReindexingForEdgeIndex:8778 » IllegalArgument Update action [ENABLE_INDEX] cannot be invoked for index with status [INSTALLED]

FlorianHockmann added a commit to GDATASoftwareAG/janusgraph that referenced this issue Sep 6, 2023
@FlorianHockmann
Mark more tests as flaky
c7f1414 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959

Signed-off-by: Florian Hockmann <[email protected]>
FlorianHockmann added a commit to GDATASoftwareAG/janusgraph that referenced this issue Sep 6, 2023
@FlorianHockmann
Mark more tests as flaky
0cf2afb 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

Signed-off-by: Florian Hockmann <[email protected]>
FlorianHockmann added a commit to FlorianHockmann/janusgraph that referenced this issue Sep 7, 2023
@FlorianHockmann
Mark more tests as flaky
1f8a634 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

Signed-off-by: Florian Hockmann <[email protected]>
@FlorianHockmann FlorianHockmann mentioned this issue Sep 7, 2023
Merged
FlorianHockmann added a commit to GDATASoftwareAG/janusgraph that referenced this issue Sep 7, 2023
@FlorianHockmann
Mark more tests as flaky
80ca920 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#1498
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: JanusGraph#3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

Signed-off-by: Florian Hockmann <[email protected]>
FlorianHockmann added a commit to GDATASoftwareAG/janusgraph that referenced this issue Sep 13, 2023
@FlorianHockmann
Mark more tests as flaky
87911fd 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#1498
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: JanusGraph#3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
FlorianHockmann added a commit that referenced this issue Sep 14, 2023
@FlorianHockmann
Mark more tests as flaky
2ba933c 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* #1457
* #1459
* #1462
* #1464
* #1465
* #1497
* #1498
* #2272
* #3142
* #3356
* #3392
* #3393
* #3651
* #3931
* #3959
* #3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: #3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
FlorianHockmann added a commit to FlorianHockmann/janusgraph that referenced this issue Sep 14, 2023
@FlorianHockmann
Mark more tests as flaky
b7386bc 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* JanusGraph#1457
* JanusGraph#1459
* JanusGraph#1462
* JanusGraph#1464
* JanusGraph#1465
* JanusGraph#1497
* JanusGraph#1498
* JanusGraph#2272
* JanusGraph#3142
* JanusGraph#3356
* JanusGraph#3392
* JanusGraph#3393
* JanusGraph#3651
* JanusGraph#3931
* JanusGraph#3959
* JanusGraph#3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: JanusGraph#3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
(cherry picked from commit 2ba933c)

# Conflicts:
#	janusgraph-cql/src/test/java/org/janusgraph/graphdb/cql/CQLGraphTest.java
#	janusgraph-solr/src/test/java/org/janusgraph/diskstorage/solr/CQLSolrTest.java
#	janusgraph-test/src/test/java/org/janusgraph/diskstorage/cache/ExpirationCacheTest.java
FlorianHockmann added a commit that referenced this issue Sep 14, 2023
@FlorianHockmann
Mark more tests as flaky
9553819 
This adds the necessary annotation to repeat flaky tests if necessary
for tests described in these issues:
* #1457
* #1459
* #1462
* #1464
* #1465
* #1497
* #1498
* #2272
* #3142
* #3356
* #3392
* #3393
* #3651
* #3931
* #3959
* #3960

I went through the recent builds on `master` and checked all failed
builds. I think this should handle all flaky tests that occurred in
these builds, apart from the TinkerPop test where I just don't know how
to deal with it: #3841, but we can of course just address that in
another PR.
We however also sometimes get failed jobs because of some intermediate
issues where a backend cannot be properly set up or teared down for some
reason or downloading some artifact just fails. I don't know what we can
do to improve this, but I would also consider that out of scope of this
PR where I just wanted to deal with failing jobs due to specific tests
being flaky.

Hopefully, this will already lead to significantly fewer cases where we
have to manually restart failed jobs (which I had to do a lot lately for
all these Dependabot PRs).

I also added a link to the relevant issue to all flaky tests.

Signed-off-by: Florian Hockmann <[email protected]>
(cherry picked from commit 2ba933c)

# Conflicts:
#	janusgraph-cql/src/test/java/org/janusgraph/graphdb/cql/CQLGraphTest.java
#	janusgraph-solr/src/test/java/org/janusgraph/diskstorage/solr/CQLSolrTest.java
#	janusgraph-test/src/test/java/org/janusgraph/diskstorage/cache/ExpirationCacheTest.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
storage/cassandra testing/flaky
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@FlorianHockmann