Large size project export gets stuck at 10% #1071

qiuwei · 2019-03-28T04:43:11Z

Describe the bug
We have an annotation project which consists of around 6000 documents.
No matter which export format we choose, the export process gets stuck at 10%.
We checked the server log, but couldn't find any error message.
To Reproduce
Steps to reproduce the behavior:

Go to settings page
choose a format such as conll2009
click on export
wait

Expected behavior
The export should finish in a reasonable time.

Screenshots
If applicable, add screenshots to help explain your problem.

Please complete the following information:

Version and build ID: INCEpTION -- 0.7.1 (2019-01-24 20:14:18, build e944075)
OS: Linux
Browser: chrome

Additional context
Add any other context about the problem here.

The text was updated successfully, but these errors were encountered:

reckart · 2019-03-28T06:09:28Z

Could you use the jstack command on the server command line to produce a thread dump which would show us where it gets stuck?

reckart · 2019-03-28T08:42:13Z

It might also be that your annotations are not compatible with the chosen format. Does the export also hang if you choose "binary CAS" as the format?

qiuwei · 2019-03-29T07:49:43Z

@reckart thanks for the quick response.
Indeed, it is due to that the annotations are not compatible. After choosing the binary cas, I got OutOfMemoryError. I suppose that could be solved by increasing the max heap size.

qiuwei · 2019-04-03T09:14:32Z

Sorry to bring this up again. After I increased the max heap size, exporting binary cas format still causes a hang at 19%.

the following is the jstack output:

2019-04-03 09:08:25
Full thread dump OpenJDK 64-Bit Server VM (11.0.1+13-Debian-2bpo91 mixed mode, sharing):

Threads class SMR info:
_java_thread_list=0x00007fb058001c50, length=32, elements={
0x00007fb0a4231800, 0x00007fb0a4233800, 0x00007fb0a423a800, 0x00007fb0a423c800,
0x00007fb0a423e800, 0x00007fb0a4240800, 0x00007fb0a429f000, 0x00007fb0a42a3800,
0x00007fb01465b000, 0x00007fb0a5e84800, 0x00007fb0a52ef800, 0x00007fb0a4fe4800,
0x00007fb0a6892800, 0x00007fb0a46ab800, 0x00007fb0a4f52800, 0x00007fb0a4f53800,
0x00007fb0a5599800, 0x00007fb0a559a800, 0x00007fb0a5743800, 0x00007fb0a5744800,
0x00007fb0a6303000, 0x00007fb0a6304000, 0x00007fb0a5c88000, 0x00007fb0a5c89000,
0x00007fb0a6974000, 0x00007fb0a6974800, 0x00007fb0a514e000, 0x00007fb0a4011800,
0x00007faff01fc800, 0x00007faff0203000, 0x00007fafe0022800, 0x00007fb058001000
}

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=95.55ms elapsed=8847.90s tid=0x00007fb0a4231800 nid=0xf waiting on condition  [0x00007fb07c7e5000]
   java.lang.Thread.State: RUNNABLE
	at java.lang.ref.Reference.waitForReferencePendingList([email protected]/Native Method)
	at java.lang.ref.Reference.processPendingReferences([email protected]/Reference.java:241)
	at java.lang.ref.Reference$ReferenceHandler.run([email protected]/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=158.68ms elapsed=8847.90s tid=0x00007fb0a4233800 nid=0x10 in Object.wait()  [0x00007fb07c6e4000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <no object reference available>
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000006ffe01490> (a java.lang.ref.ReferenceQueue$Lock)
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:176)
	at java.lang.ref.Finalizer$FinalizerThread.run([email protected]/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=2.41ms elapsed=8847.89s tid=0x00007fb0a423a800 nid=0x11 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 cpu=79209.02ms elapsed=8847.89s tid=0x00007fb0a423c800 nid=0x12 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #8 daemon prio=9 os_prio=0 cpu=14954.48ms elapsed=8847.89s tid=0x00007fb0a423e800 nid=0x13 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #9 daemon prio=9 os_prio=0 cpu=5710.56ms elapsed=8847.89s tid=0x00007fb0a4240800 nid=0x14 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #10 daemon prio=9 os_prio=0 cpu=0.16ms elapsed=8847.87s tid=0x00007fb0a429f000 nid=0x15 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #11 daemon prio=8 os_prio=0 cpu=167.80ms elapsed=8847.87s tid=0x00007fb0a42a3800 nid=0x17 in Object.wait()  [0x00007fb04fdfc000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <no object reference available>
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x00000006ffe01ab8> (a java.lang.ref.ReferenceQueue$Lock)
	at jdk.internal.ref.CleanerImpl.run([email protected]/CleanerImpl.java:148)
	at java.lang.Thread.run([email protected]/Thread.java:834)
	at jdk.internal.misc.InnocuousThread.run([email protected]/InnocuousThread.java:134)

"Abandoned connection cleanup thread" #18 daemon prio=5 os_prio=0 cpu=299.95ms elapsed=8838.16s tid=0x00007fb01465b000 nid=0x29 in Object.wait()  [0x00007fb04cabc000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait([email protected]/Native Method)
	- waiting on <no object reference available>
	at java.lang.ref.ReferenceQueue.remove([email protected]/ReferenceQueue.java:155)
	- waiting to re-lock in wait() <0x000000070771ec48> (a java.lang.ref.ReferenceQueue$Lock)
	at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"ContainerBackgroundProcessor[StandardEngine[Tomcat]]" #19 daemon prio=5 os_prio=0 cpu=936.70ms elapsed=8835.33s tid=0x00007fb0a5e84800 nid=0x2a waiting on condition  [0x00007fb04c5bb000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep([email protected]/Native Method)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1359)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"container-0" #20 prio=5 os_prio=0 cpu=42.19ms elapsed=8835.33s tid=0x00007fb0a52ef800 nid=0x2b waiting on condition  [0x00007fb04c4ba000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep([email protected]/Native Method)
	at org.apache.catalina.core.StandardServer.await(StandardServer.java:427)
	at org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainer$2.run(TomcatEmbeddedServletContainer.java:192)

"Index task consumer" #22 prio=1 os_prio=0 cpu=0.31ms elapsed=8827.42s tid=0x00007fb0a4fe4800 nid=0x39 waiting on condition  [0x00007fb04c1b9000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703026738> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.ArrayBlockingQueue.take([email protected]/ArrayBlockingQueue.java:417)
	at de.tudarmstadt.ukp.inception.search.scheduling.TaskConsumer.run(TaskConsumer.java:57)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"NioBlockingSelector.BlockPoller-1" #23 daemon prio=5 os_prio=0 cpu=604.89ms elapsed=8820.38s tid=0x00007fb0a6892800 nid=0x3b runnable  [0x00007fb04cdbd000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPoll.wait([email protected]/Native Method)
	at sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
	- locked <0x0000000703017e80> (a sun.nio.ch.Util$2)
	- locked <0x0000000703017e28> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:136)
	at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.run(NioBlockingSelector.java:298)

"http-nio-8080-exec-1" #24 daemon prio=5 os_prio=0 cpu=1305.15ms elapsed=8820.37s tid=0x00007fb0a46ab800 nid=0x3c waiting on condition  [0x00007fb04cebe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-2" #25 daemon prio=5 os_prio=0 cpu=981.62ms elapsed=8820.37s tid=0x00007fb0a4f52800 nid=0x3d waiting on condition  [0x00007fb005738000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-3" #26 daemon prio=5 os_prio=0 cpu=2197.15ms elapsed=8820.37s tid=0x00007fb0a4f53800 nid=0x3e waiting on condition  [0x00007fb005637000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-4" #27 daemon prio=5 os_prio=0 cpu=1691.52ms elapsed=8820.37s tid=0x00007fb0a5599800 nid=0x3f waiting on condition  [0x00007fb005536000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-5" #28 daemon prio=5 os_prio=0 cpu=6341.24ms elapsed=8820.37s tid=0x00007fb0a559a800 nid=0x40 waiting on condition  [0x00007fb005435000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-6" #29 daemon prio=5 os_prio=0 cpu=917.56ms elapsed=8820.37s tid=0x00007fb0a5743800 nid=0x41 waiting on condition  [0x00007fb005334000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-7" #30 daemon prio=5 os_prio=0 cpu=1318.95ms elapsed=8820.37s tid=0x00007fb0a5744800 nid=0x42 waiting on condition  [0x00007fb005233000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-8" #31 daemon prio=5 os_prio=0 cpu=1597.11ms elapsed=8820.37s tid=0x00007fb0a6303000 nid=0x43 waiting on condition  [0x00007fb005132000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-9" #32 daemon prio=5 os_prio=0 cpu=926.17ms elapsed=8820.37s tid=0x00007fb0a6304000 nid=0x44 waiting on condition  [0x00007fb005031000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-exec-10" #33 daemon prio=5 os_prio=0 cpu=1043.91ms elapsed=8820.37s tid=0x00007fb0a5c88000 nid=0x45 waiting on condition  [0x00007fb004f30000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035120> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:194)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:2081)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:433)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:103)
	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:31)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1054)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1114)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-ClientPoller-0" #34 daemon prio=5 os_prio=0 cpu=715.71ms elapsed=8820.36s tid=0x00007fb0a5c89000 nid=0x46 runnable  [0x00007fb004e2f000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPoll.wait([email protected]/Native Method)
	at sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
	- locked <0x0000000703035aa8> (a sun.nio.ch.Util$2)
	- locked <0x0000000703035a50> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:136)
	at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:825)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-ClientPoller-1" #35 daemon prio=5 os_prio=0 cpu=744.12ms elapsed=8820.36s tid=0x00007fb0a6974000 nid=0x47 runnable  [0x00007fb004d2e000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.EPoll.wait([email protected]/Native Method)
	at sun.nio.ch.EPollSelectorImpl.doSelect([email protected]/EPollSelectorImpl.java:120)
	at sun.nio.ch.SelectorImpl.lockAndDoSelect([email protected]/SelectorImpl.java:124)
	- locked <0x00000007030186a0> (a sun.nio.ch.Util$2)
	- locked <0x0000000703018648> (a sun.nio.ch.EPollSelectorImpl)
	at sun.nio.ch.SelectorImpl.select([email protected]/SelectorImpl.java:136)
	at org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:825)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-Acceptor-0" #36 daemon prio=5 os_prio=0 cpu=21.70ms elapsed=8820.36s tid=0x00007fb0a6974800 nid=0x48 runnable  [0x00007fb004c2d000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.ServerSocketChannelImpl.accept0([email protected]/Native Method)
	at sun.nio.ch.ServerSocketChannelImpl.accept([email protected]/ServerSocketChannelImpl.java:533)
	at sun.nio.ch.ServerSocketChannelImpl.accept([email protected]/ServerSocketChannelImpl.java:285)
	at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:482)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"http-nio-8080-AsyncTimeout" #37 daemon prio=5 os_prio=0 cpu=548.48ms elapsed=8820.36s tid=0x00007fb0a514e000 nid=0x49 waiting on condition  [0x00007fb004b2c000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep([email protected]/Native Method)
	at org.apache.coyote.AbstractProtocol$AsyncTimeout.run(AbstractProtocol.java:1149)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"DestroyJavaVM" #39 prio=5 os_prio=0 cpu=23260.24ms elapsed=8820.27s tid=0x00007fb0a4011800 nid=0x8 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Wicket-AsyncDataStore-PageSavingThread" #40 daemon prio=5 os_prio=0 cpu=866.61ms elapsed=7210.01s tid=0x00007faff01fc800 nid=0x4e waiting on condition  [0x00007fb00482b000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703035f68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
	at java.util.concurrent.LinkedBlockingQueue.poll([email protected]/LinkedBlockingQueue.java:458)
	at org.apache.wicket.pageStore.AsynchronousDataStore$PageSavingRunnable.run(AsynchronousDataStore.java:321)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"Wicket-AsyncPageStore-PageSavingThread" #41 daemon prio=5 os_prio=0 cpu=4000.66ms elapsed=7209.99s tid=0x00007faff0203000 nid=0x4f waiting on condition  [0x00007fb00472a000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x0000000703043cd8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:234)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos([email protected]/AbstractQueuedSynchronizer.java:2123)
	at java.util.concurrent.LinkedBlockingQueue.poll([email protected]/LinkedBlockingQueue.java:458)
	at org.apache.wicket.pageStore.AsynchronousPageStore$PageSavingRunnable.run(AsynchronousPageStore.java:198)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"Thread-3" #42 daemon prio=5 os_prio=0 cpu=7055275.75ms elapsed=7088.48s tid=0x00007fafe0022800 nid=0x5b runnable  [0x00007fb04f8d1000]
   java.lang.Thread.State: RUNNABLE
	at org.hibernate.type.AbstractStandardBasicType.isEqual(AbstractStandardBasicType.java:198)
	at org.hibernate.type.AbstractStandardBasicType.isSame(AbstractStandardBasicType.java:186)
	at org.hibernate.type.AbstractStandardBasicType.isDirty(AbstractStandardBasicType.java:229)
	at org.hibernate.type.AbstractStandardBasicType.isDirty(AbstractStandardBasicType.java:225)
	at org.hibernate.type.TypeHelper.findDirty(TypeHelper.java:309)
	at org.hibernate.persister.entity.AbstractEntityPersister.findDirty(AbstractEntityPersister.java:4404)
	at org.hibernate.event.internal.DefaultFlushEntityEventListener.dirtyCheck(DefaultFlushEntityEventListener.java:569)
	at org.hibernate.event.internal.DefaultFlushEntityEventListener.isUpdateNecessary(DefaultFlushEntityEventListener.java:226)
	at org.hibernate.event.internal.DefaultFlushEntityEventListener.onFlushEntity(DefaultFlushEntityEventListener.java:153)
	at org.hibernate.event.internal.AbstractFlushingEventListener.flushEntities(AbstractFlushingEventListener.java:235)
	at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:94)
	at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:44)
	at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1415)
	at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1501)
	at org.hibernate.query.internal.AbstractProducedQuery.doList(AbstractProducedQuery.java:1537)
	at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1505)
	at org.hibernate.query.Query.getResultList(Query.java:135)
	at de.tudarmstadt.ukp.clarin.webanno.security.UserDaoImpl.exists(UserDaoImpl.java:54)
	at de.tudarmstadt.ukp.clarin.webanno.security.UserDaoImpl$$FastClassBySpringCGLIB$$ef901b5c.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionInterceptor$$Lambda$302/0x0000000840b4f040.proceedWithInvocation(Unknown Source)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
	at de.tudarmstadt.ukp.clarin.webanno.security.UserDaoImpl$$EnhancerBySpringCGLIB$$dba9c54a.exists(<generated>)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.DocumentServiceImpl.getAllAnnotators(DocumentServiceImpl.java:876)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.DocumentServiceImpl.listAnnotationDocuments(DocumentServiceImpl.java:721)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.DocumentServiceImpl$$FastClassBySpringCGLIB$$7e8e7e07.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionInterceptor$$Lambda$302/0x0000000840b4f040.proceedWithInvocation(Unknown Source)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.DocumentServiceImpl$$EnhancerBySpringCGLIB$$38185b2b.listAnnotationDocuments(<generated>)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.export.exporters.AnnotationDocumentExporter.exportAnnotationDocuments(AnnotationDocumentExporter.java:105)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.export.exporters.AnnotationDocumentExporter.exportData(AnnotationDocumentExporter.java:92)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.export.ProjectExportServiceImpl.exportProject(ProjectExportServiceImpl.java:185)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.export.ProjectExportServiceImpl.exportProject(ProjectExportServiceImpl.java:130)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.export.ProjectExportServiceImpl$$FastClassBySpringCGLIB$$3d43e16f.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.transaction.interceptor.TransactionInterceptor$$Lambda$302/0x0000000840b4f040.proceedWithInvocation(Unknown Source)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
	at de.tudarmstadt.ukp.clarin.webanno.api.dao.export.ProjectExportServiceImpl$$EnhancerBySpringCGLIB$$2cdbfb63.exportProject(<generated>)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0([email protected]/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke([email protected]/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke([email protected]/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke([email protected]/Method.java:566)
	at org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxyFactory.java:521)
	at com.sun.proxy.$Proxy176.exportProject(Unknown Source)
	at de.tudarmstadt.ukp.clarin.webanno.ui.project.export.ProjectExportPanel$FileGenerator.run(ProjectExportPanel.java:401)
	at java.lang.Thread.run([email protected]/Thread.java:834)

"Attach Listener" #43 daemon prio=9 os_prio=0 cpu=1.23ms elapsed=65.72s tid=0x00007fb058001000 nid=0xd2 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=6320.98ms elapsed=8847.90s tid=0x00007fb0a422f000 nid=0xe runnable

"GC Thread#0" os_prio=0 cpu=20784.12ms elapsed=8847.92s tid=0x00007fb0a4027800 nid=0x9 runnable

"GC Thread#1" os_prio=0 cpu=20746.27ms elapsed=8847.56s tid=0x00007fb06c001000 nid=0x1a runnable

"GC Thread#2" os_prio=0 cpu=20973.89ms elapsed=8847.56s tid=0x00007fb06c002800 nid=0x1b runnable

"GC Thread#3" os_prio=0 cpu=20738.37ms elapsed=8847.56s tid=0x00007fb06c004000 nid=0x1c runnable

"GC Thread#4" os_prio=0 cpu=20777.10ms elapsed=8847.56s tid=0x00007fb06c006000 nid=0x1d runnable

"GC Thread#5" os_prio=0 cpu=20849.97ms elapsed=8847.56s tid=0x00007fb06c008000 nid=0x1e runnable

"GC Thread#6" os_prio=0 cpu=20723.53ms elapsed=8845.26s tid=0x00007fb06c03d800 nid=0x20 runnable

"GC Thread#7" os_prio=0 cpu=20831.84ms elapsed=8843.42s tid=0x00007fb06c079000 nid=0x23 runnable

"G1 Main Marker" os_prio=0 cpu=32.10ms elapsed=8847.92s tid=0x00007fb0a4088000 nid=0xa runnable

"G1 Conc#0" os_prio=0 cpu=27529.21ms elapsed=8847.92s tid=0x00007fb0a4089800 nid=0xb runnable

"G1 Conc#1" os_prio=0 cpu=27529.37ms elapsed=8845.14s tid=0x00007fb078001000 nid=0x21 runnable

"G1 Refine#0" os_prio=0 cpu=987.05ms elapsed=8847.92s tid=0x00007fb0a418a000 nid=0xc runnable

"G1 Refine#1" os_prio=0 cpu=84.56ms elapsed=7087.15s tid=0x00007fb074003000 nid=0x5e runnable

"G1 Refine#2" os_prio=0 cpu=72.04ms elapsed=7086.72s tid=0x00007fb03c0bf000 nid=0x60 runnable

"G1 Refine#3" os_prio=0 cpu=51.48ms elapsed=7086.72s tid=0x00007fafb0001000 nid=0x61 runnable

"G1 Refine#4" os_prio=0 cpu=47.33ms elapsed=7086.71s tid=0x00007fafb4001000 nid=0x62 runnable

"G1 Refine#5" os_prio=0 cpu=29.57ms elapsed=7086.71s tid=0x00007fafa8001000 nid=0x63 runnable

"G1 Refine#6" os_prio=0 cpu=20.18ms elapsed=7086.71s tid=0x00007fafac001000 nid=0x64 runnable

"G1 Refine#7" os_prio=0 cpu=8.13ms elapsed=7086.71s tid=0x00007fafa0001000 nid=0x65 runnable

"G1 Young RemSet Sampling" os_prio=0 cpu=5406.61ms elapsed=8847.92s tid=0x00007fb0a418c000 nid=0xd runnable
"VM Periodic Task Thread" os_prio=0 cpu=7604.13ms elapsed=8847.87s tid=0x00007fb0a42a1800 nid=0x16 waiting on condition

JNI global refs: 22, weak refs: 0

reckart · 2019-04-03T10:19:28Z

@qiuwei how many users and documents do you have in your project?

qiuwei · 2019-04-03T11:03:11Z

I have around 30 users, but they are not active when the export is executed.
There are 6000 documents, each of which has 20~300 words.

qiuwei · 2019-04-03T11:04:25Z

Exporting a single document has no problem. I am wondering wether there is an api for exporting a single document, so that I can just iterate through the project.

reckart · 2019-04-03T11:10:17Z

I assume that the system hangs because it is hammering the DB - but I need to investigate this in more detail.

reckart · 2019-04-03T12:11:10Z

@qiuwei Well, you can export documents via the remote API.

qiuwei · 2019-04-03T12:15:37Z

@reckart thanks. I will try that out.
My current workaround is to read the binary cas saved in the filesystem. Is there any potential problem with this approach? thanks!

reckart · 2019-04-03T12:44:09Z

If you are comfortable with that - no. You may note that it contains additional internal annotations such as a CASMetadata annotation which the exported documents would not contain because it is filtered out before export. You may also note that the DocumentMetadata values may differ between the files on disk and the exported files. If you do not rely on these, you should be fine.

reckart · 2019-11-08T20:17:25Z

There were recently a couple of upstream improvements regarding the export process - it should now be more efficient. I'm closing this therefore. If somebody continues to find the export too slow, please open a new issue.

qiuwei closed this as completed Mar 29, 2019

qiuwei reopened this Apr 3, 2019

reckart added 🐛Bug Something isn't working Requires upstream changes labels May 14, 2019

reckart modified the milestones: 0.9.1, 0.9.2 May 14, 2019

reckart modified the milestones: 0.9.2, 0.9.3 Jun 6, 2019

reckart modified the milestones: 0.10.1, 0.10.2 Jul 9, 2019

reckart modified the milestones: 0.10.3, 0.11.1 Aug 14, 2019

reckart modified the milestones: 0.11.1, Bug backlog Aug 27, 2019

reckart closed this as completed Nov 8, 2019

reckart modified the milestones: Bug backlog, 0.13.0 Nov 8, 2019

reckart self-assigned this Nov 8, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Large size project export gets stuck at 10% #1071

Large size project export gets stuck at 10% #1071

qiuwei commented Mar 28, 2019

reckart commented Mar 28, 2019

reckart commented Mar 28, 2019

qiuwei commented Mar 29, 2019

qiuwei commented Apr 3, 2019 •

edited by reckart

Loading

reckart commented Apr 3, 2019

qiuwei commented Apr 3, 2019

qiuwei commented Apr 3, 2019 •

edited

Loading

reckart commented Apr 3, 2019

reckart commented Apr 3, 2019

qiuwei commented Apr 3, 2019

reckart commented Apr 3, 2019

reckart commented Nov 8, 2019

Large size project export gets stuck at 10% #1071

Large size project export gets stuck at 10% #1071

Comments

qiuwei commented Mar 28, 2019

reckart commented Mar 28, 2019

reckart commented Mar 28, 2019

qiuwei commented Mar 29, 2019

qiuwei commented Apr 3, 2019 • edited by reckart Loading

reckart commented Apr 3, 2019

qiuwei commented Apr 3, 2019

qiuwei commented Apr 3, 2019 • edited Loading

reckart commented Apr 3, 2019

reckart commented Apr 3, 2019

qiuwei commented Apr 3, 2019

reckart commented Apr 3, 2019

reckart commented Nov 8, 2019

qiuwei commented Apr 3, 2019 •

edited by reckart

Loading

qiuwei commented Apr 3, 2019 •

edited

Loading