Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fallback queryNumber() if there are uncommitted records #2060

Merged
merged 3 commits into from
Dec 29, 2022

Conversation

javeme
Copy link
Contributor

@javeme javeme commented Dec 22, 2022

fixed sub-issue 1 of #2058

@javeme javeme mentioned this pull request Dec 22, 2022
1 task
@imbajin
Copy link
Member

imbajin commented Dec 23, 2022

image

@imbajin imbajin added ci-cd Build or deploy tests Add or improve test cases labels Dec 23, 2022
@imbajin imbajin added this to the 1.0.0 milestone Dec 23, 2022
@codecov
Copy link

codecov bot commented Dec 26, 2022

Codecov Report

Merging #2060 (973e6b0) into master (6c3170c) will decrease coverage by 0.01%.
The diff coverage is 75.86%.

@@             Coverage Diff              @@
##             master    #2060      +/-   ##
============================================
- Coverage     68.36%   68.35%   -0.02%     
- Complexity      977      979       +2     
============================================
  Files           481      481              
  Lines         39807    39821      +14     
  Branches       5592     5598       +6     
============================================
+ Hits          27216    27221       +5     
- Misses         9952     9957       +5     
- Partials       2639     2643       +4     
Impacted Files Coverage Δ
.../apache/hugegraph/backend/tx/GraphTransaction.java 79.87% <74.07%> (-0.27%) ⬇️
...java/org/apache/hugegraph/backend/query/Query.java 86.56% <100.00%> (+0.10%) ⬆️
...hugegraph/backend/store/raft/RaftBackendStore.java 73.11% <0.00%> (-2.16%) ⬇️
.../hugegraph/backend/store/rocksdb/RocksDBTable.java 67.11% <0.00%> (-2.02%) ⬇️
...hugegraph/backend/store/raft/rpc/RpcForwarder.java 69.49% <0.00%> (-1.70%) ⬇️
...ugegraph/backend/store/raft/StoreStateMachine.java 64.86% <0.00%> (-0.91%) ⬇️
...a/org/apache/hugegraph/util/collection/IntSet.java 73.72% <0.00%> (-0.85%) ⬇️
...g/apache/hugegraph/backend/store/BackendTable.java 85.11% <0.00%> (+0.59%) ⬆️
...n/java/org/apache/hugegraph/schema/IndexLabel.java 77.27% <0.00%> (+1.13%) ⬆️
...e/hugegraph/backend/tx/SchemaIndexTransaction.java 79.54% <0.00%> (+6.81%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@javeme
Copy link
Contributor Author

javeme commented Dec 26, 2022

cassandra can't run on mac m1:

bin/cassandraapache-cassandra-3.11.14 [0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:bin/../logs/gc.log instead.
intx ThreadPriorityPolicy=42 is outside the allowed range [ 0 ... 1 ]
Improperly specified VM option 'ThreadPriorityPolicy=42'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
bin/cassandra            
                                                                                                                                                                      
The stack size specified is too small, Specify at least 640k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
➜  apache-cassandra-3.11.14 vim conf/jvm.options
ERROR [main] 2022-12-26 23:40:03,898 NativeLibraryDarwin.java:64 - Failed to link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /private/var/folders/3q/9d9hmtzd6c397wsb1fchsvyh0000gn/T/jna-101015171/jna6875563382134887427.tmp: dlopen(/private/var/folders/3q/9d9hmtzd6c397wsb1fchsvyh0000gn/T/jna-101015171/jna6875563382134887427.tmp, 0x0001): tried: '/private/var/folders/3q/9d9hmtzd6c397wsb1fchsvyh0000gn/T/jna-101015171/jna6875563382134887427.tmp' (fat file, but missing compatible architecture (have (i386,x86_64), need (arm64e)))
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832)
	at java.lang.Runtime.load0(Runtime.java:783)
	at java.lang.System.load(System.java:1100)
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851)
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826)
	at com.sun.jna.Native.<clinit>(Native.java:140)
	at com.sun.jna.NativeLibrary.<clinit>(NativeLibrary.java:84)
	at org.apache.cassandra.utils.NativeLibraryDarwin.<clinit>(NativeLibraryDarwin.java:55)
	at org.apache.cassandra.utils.NativeLibrary.<clinit>(NativeLibrary.java:90)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:212)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:633)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:786)
WARN  [main] 2022-12-26 23:40:03,899 StartupChecks.java:136 - jemalloc shared library could not be preloaded to speed up memory allocations
WARN  [main] 2022-12-26 23:40:03,899 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
ERROR [main] 2022-12-26 23:40:03,899 CassandraDaemon.java:803 - The native library could not be initialized properly.

@imbajin
Copy link
Member

imbajin commented Dec 27, 2022

some simple ways to handle this problem:

  1. replace a JNA jar in 3.x - refer how-to-start-cassandra-on-a-m1-macbook
  2. use 4.x directly (seems no compatible problem)
  3. use 3.x with apple transfer compile

@imbajin
Copy link
Member

imbajin commented Dec 27, 2022

cassandra can't run on mac m1:

bin/cassandraapache-cassandra-3.11.14 [0.001s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:bin/../logs/gc.log instead.
intx ThreadPriorityPolicy=42 is outside the allowed range [ 0 ... 1 ]
Improperly specified VM option 'ThreadPriorityPolicy=42'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
bin/cassandra            
                                                                                                                                                                      
The stack size specified is too small, Specify at least 640k
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
➜  apache-cassandra-3.11.14 vim conf/jvm.options
ERROR [main] 2022-12-26 23:40:03,898 NativeLibraryDarwin.java:64 - Failed to link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /private/var/folders/3q/9d9hmtzd6c397wsb1fchsvyh0000gn/T/jna-101015171/jna6875563382134887427.tmp: dlopen(/private/var/folders/3q/9d9hmtzd6c397wsb1fchsvyh0000gn/T/jna-101015171/jna6875563382134887427.tmp, 0x0001): tried: '/private/var/folders/3q/9d9hmtzd6c397wsb1fchsvyh0000gn/T/jna-101015171/jna6875563382134887427.tmp' (fat file, but missing compatible architecture (have (i386,x86_64), need (arm64e)))
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1950)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1832)
	at java.lang.Runtime.load0(Runtime.java:783)
	at java.lang.System.load(System.java:1100)
	at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851)
	at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826)
	at com.sun.jna.Native.<clinit>(Native.java:140)
	at com.sun.jna.NativeLibrary.<clinit>(NativeLibrary.java:84)
	at org.apache.cassandra.utils.NativeLibraryDarwin.<clinit>(NativeLibraryDarwin.java:55)
	at org.apache.cassandra.utils.NativeLibrary.<clinit>(NativeLibrary.java:90)
	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:212)
	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:633)
	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:786)
WARN  [main] 2022-12-26 23:40:03,899 StartupChecks.java:136 - jemalloc shared library could not be preloaded to speed up memory allocations
WARN  [main] 2022-12-26 23:40:03,899 StartupChecks.java:169 - JMX is not enabled to receive remote connections. Please see cassandra-env.sh for more info.
ERROR [main] 2022-12-26 23:40:03,899 CassandraDaemon.java:803 - The native library could not be initialized properly.

some simple ways to handle this problem:

  1. replace a JNA jar in 3.x - refer how-to-start-cassandra-on-a-m1-macbook
  2. use 4.x directly (seems no compatible problem)
  3. use 3.x with apple transfer compile

@zyxxoo
Copy link
Contributor

zyxxoo commented Dec 28, 2022

这里 select count(*) 的结果,在解析的时候 count 被视为column,但是我们在 hugeKey 中没有定义

row2Entry:429, CassandraTable (org.apache.hugegraph.backend.store.cassandra)
lambda$results2Entries$5:418, CassandraTable (org.apache.hugegraph.backend.store.cassandra)
apply:-1, 646504864 (org.apache.hugegraph.backend.store.cassandra.CassandraTable$$Lambda$192)
fetch:115, CassandraEntryIterator (org.apache.hugegraph.backend.store.cassandra)
hasNext:62, BackendEntryIterator (org.apache.hugegraph.backend.store)
fetch:86, ExtendableIterator (org.apache.hugegraph.iterator)
hasNext:40, WrappedIterator (org.apache.hugegraph.iterator)
lambda$flatMap$0:226, QueryResults (org.apache.hugegraph.backend.query)
apply:-1, 253709867 (org.apache.hugegraph.backend.query.QueryResults$$Lambda$249)
fetch:62, FlatMapperIterator (org.apache.hugegraph.iterator)
hasNext:40, WrappedIterator (org.apache.hugegraph.iterator)
fetch:42, MapperIterator (org.apache.hugegraph.iterator)
hasNext:40, WrappedIterator (org.apache.hugegraph.iterator)
keepInputOrderIfNeeded:103, QueryResults (org.apache.hugegraph.backend.query)
queryVerticesFromBackend:821, GraphTransaction (org.apache.hugegraph.backend.tx)
queryVerticesFromBackend:243, CachedGraphTransaction (org.apache.hugegraph.backend.cache)
queryVertices:801, GraphTransaction (org.apache.hugegraph.backend.tx)
lambda$queryNumber$1:583, GraphTransaction (org.apache.hugegraph.backend.tx)
apply:-1, 984832924 (org.apache.hugegraph.backend.tx.GraphTransaction$$Lambda$266)
iterator:161, QueryList$OptimizedQuery (org.apache.hugegraph.backend.page)
apply:-1, 1055453694 (org.apache.hugegraph.backend.page.QueryList$$Lambda$248)
lambda$flatMap$0:225, QueryResults (org.apache.hugegraph.backend.query)
apply:-1, 253709867 (org.apache.hugegraph.backend.query.QueryResults$$Lambda$249)
fetch:62, FlatMapperIterator (org.apache.hugegraph.iterator)
hasNext:40, WrappedIterator (org.apache.hugegraph.iterator)
reduce:93, Aggregate$AggregateFunc (org.apache.hugegraph.backend.query)
reduce:50, Aggregate (org.apache.hugegraph.backend.query)
queryNumber:593, GraphTransaction (org.apache.hugegraph.backend.tx)
queryNumber:703, StandardHugeGraph (org.apache.hugegraph)
verticesCount:91, HugeGraphStep (org.apache.hugegraph.traversal.optimize)
count:80, HugeGraphStep (org.apache.hugegraph.traversal.optimize)
processNextStart:77, HugeCountStep (org.apache.hugegraph.traversal.optimize)
next:135, AbstractStep (org.apache.tinkerpop.gremlin.process.traversal.step.util)
next:40, AbstractStep (org.apache.tinkerpop.gremlin.process.traversal.step.util)
next:230, DefaultTraversal (org.apache.tinkerpop.gremlin.process.traversal.util)
testQueryCountWithOptimizeByIndex:5793, VertexCoreTest (org.apache.hugegraph.core)
invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
invoke:566, Method (java.lang.reflect)
runReflectiveCall:50, FrameworkMethod$1 (org.junit.runners.model)
run:12, ReflectiveCallable (org.junit.internal.runners.model)
invokeExplosively:47, FrameworkMethod (org.junit.runners.model)
evaluate:17, InvokeMethod (org.junit.internal.runners.statements)
evaluate:26, RunBefores (org.junit.internal.runners.statements)
evaluate:27, RunAfters (org.junit.internal.runners.statements)
runLeaf:325, ParentRunner (org.junit.runners)
runChild:78, BlockJUnit4ClassRunner (org.junit.runners)
runChild:57, BlockJUnit4ClassRunner (org.junit.runners)
run:290, ParentRunner$3 (org.junit.runners)
schedule:71, ParentRunner$1 (org.junit.runners)
runChildren:288, ParentRunner (org.junit.runners)
access$000:58, ParentRunner (org.junit.runners)
evaluate:268, ParentRunner$2 (org.junit.runners)
evaluate:26, RunBefores (org.junit.internal.runners.statements)
evaluate:27, RunAfters (org.junit.internal.runners.statements)
run:363, ParentRunner (org.junit.runners)
runChild:128, Suite (org.junit.runners)
runChild:27, Suite (org.junit.runners)
run:290, ParentRunner$3 (org.junit.runners)
schedule:71, ParentRunner$1 (org.junit.runners)
runChildren:288, ParentRunner (org.junit.runners)
access$000:58, ParentRunner (org.junit.runners)
evaluate:268, ParentRunner$2 (org.junit.runners)
run:363, ParentRunner (org.junit.runners)
run:137, JUnitCore (org.junit.runner)
startRunnerWithArgs:69, JUnit4IdeaTestRunner (com.intellij.junit4)
execute:38, IdeaTestRunner$Repeater$1 (com.intellij.rt.junit)
repeat:11, TestsRepeater (com.intellij.rt.execution.junit)
startRunnerWithArgs:35, IdeaTestRunner$Repeater (com.intellij.rt.junit)
prepareStreamsAndStart:235, JUnitStarter (com.intellij.rt.junit)
main:54, JUnitStarter (com.intellij.rt.junit)

@zyxxoo
Copy link
Contributor

zyxxoo commented Dec 28, 2022

以前是走 queryNumber:

lambda$queryNumber$1:121, CassandraTable (org.apache.hugegraph.backend.store.cassandra)
apply:-1, 417160915 (org.apache.hugegraph.backend.store.cassandra.CassandraTable$$Lambda$266)
query:152, CassandraTable (org.apache.hugegraph.backend.store.cassandra)
queryNumber:117, CassandraTable (org.apache.hugegraph.backend.store.cassandra)
queryNumber:333, CassandraStore (org.apache.hugegraph.backend.store.cassandra)
queryNumber:131, AbstractTransaction (org.apache.hugegraph.backend.tx)
lambda$queryNumber$1:559, GraphTransaction (org.apache.hugegraph.backend.tx)
apply:-1, 265556987 (org.apache.hugegraph.backend.tx.GraphTransaction$$Lambda$269)
iterator:161, QueryList$OptimizedQuery (org.apache.hugegraph.backend.page)
apply:-1, 1962610818 (org.apache.hugegraph.backend.page.QueryList$$Lambda$247)
lambda$flatMap$0:225, QueryResults (org.apache.hugegraph.backend.query)
apply:-1, 1695173607 (org.apache.hugegraph.backend.query.QueryResults$$Lambda$248)
fetch:62, FlatMapperIterator (org.apache.hugegraph.iterator)
hasNext:40, WrappedIterator (org.apache.hugegraph.iterator)
reduce:93, Aggregate$AggregateFunc (org.apache.hugegraph.backend.query)
reduce:50, Aggregate (org.apache.hugegraph.backend.query)
queryNumber:583, GraphTransaction (org.apache.hugegraph.backend.tx)
queryNumber:703, StandardHugeGraph (org.apache.hugegraph)
verticesCount:91, HugeGraphStep (org.apache.hugegraph.traversal.optimize)
count:80, HugeGraphStep (org.apache.hugegraph.traversal.optimize)
processNextStart:77, HugeCountStep (org.apache.hugegraph.traversal.optimize)
next:135, AbstractStep (org.apache.tinkerpop.gremlin.process.traversal.step.util)
next:40, AbstractStep (org.apache.tinkerpop.gremlin.process.traversal.step.util)
next:230, DefaultTraversal (org.apache.tinkerpop.gremlin.process.traversal.util)
testQueryCountWithOptimizeByIndex:5783, VertexCoreTest (org.apache.hugegraph.core)
invoke0:-1, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:62, NativeMethodAccessorImpl (jdk.internal.reflect)
invoke:43, DelegatingMethodAccessorImpl (jdk.internal.reflect)
invoke:566, Method (java.lang.reflect)
runReflectiveCall:50, FrameworkMethod$1 (org.junit.runners.model)
run:12, ReflectiveCallable (org.junit.internal.runners.model)
invokeExplosively:47, FrameworkMethod (org.junit.runners.model)
evaluate:17, InvokeMethod (org.junit.internal.runners.statements)
evaluate:26, RunBefores (org.junit.internal.runners.statements)
evaluate:27, RunAfters (org.junit.internal.runners.statements)
runLeaf:325, ParentRunner (org.junit.runners)
runChild:78, BlockJUnit4ClassRunner (org.junit.runners)
runChild:57, BlockJUnit4ClassRunner (org.junit.runners)
run:290, ParentRunner$3 (org.junit.runners)
schedule:71, ParentRunner$1 (org.junit.runners)
runChildren:288, ParentRunner (org.junit.runners)
access$000:58, ParentRunner (org.junit.runners)
evaluate:268, ParentRunner$2 (org.junit.runners)
evaluate:26, RunBefores (org.junit.internal.runners.statements)
evaluate:27, RunAfters (org.junit.internal.runners.statements)
run:363, ParentRunner (org.junit.runners)
runChild:128, Suite (org.junit.runners)
runChild:27, Suite (org.junit.runners)
run:290, ParentRunner$3 (org.junit.runners)
schedule:71, ParentRunner$1 (org.junit.runners)
runChildren:288, ParentRunner (org.junit.runners)
access$000:58, ParentRunner (org.junit.runners)
evaluate:268, ParentRunner$2 (org.junit.runners)
run:363, ParentRunner (org.junit.runners)
run:137, JUnitCore (org.junit.runner)
startRunnerWithArgs:69, JUnit4IdeaTestRunner (com.intellij.junit4)
execute:38, IdeaTestRunner$Repeater$1 (com.intellij.rt.junit)
repeat:11, TestsRepeater (com.intellij.rt.execution.junit)
startRunnerWithArgs:35, IdeaTestRunner$Repeater (com.intellij.rt.junit)
prepareStreamsAndStart:235, JUnitStarter (com.intellij.rt.junit)
main:54, JUnitStarter (com.intellij.rt.junit)

@javeme javeme force-pushed the fallback-queryNumber-with-uncommitted branch from e769b85 to a022486 Compare December 28, 2022 13:29
Change-Id: Icefb8089c9ef5231a9bf4b1e5e6e359935feda51
Change-Id: I4a08d6ee3cdc4c914e02d27396bda488c424f136
@javeme javeme force-pushed the fallback-queryNumber-with-uncommitted branch from a022486 to 973e6b0 Compare December 28, 2022 13:33
@imbajin imbajin merged commit eb7a65c into master Dec 29, 2022
@imbajin imbajin deleted the fallback-queryNumber-with-uncommitted branch December 29, 2022 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-cd Build or deploy tests Add or improve test cases
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants