add document not working well #311
Assignees
Labels
Comments
|
Hi Arthcras, thanks a lot for using Datashare and for your feedback. I can't reproduce the bug so I have more questions for you: "For some reason new documents added to the datashare folder are not shown with search document. I believe the issue relates to that the indexing is not working when adding new documents" "Even after reinstalling datashare conform manual and removing the whole application, still an old document is visible in the data share application; while it is not in the datashare folder anymore; also the new documents added to the datashare folder are not feasible" Please let us know if this answer your questions or if there is a bug that we haven't seen. Thanks a lot. |
|
Hi
Thanks for the clarification
Ok will remove the index
However it seems that there is something with the index function because new documentents added to the folder are not shown thus or not indexed
Can you look into that posted also the scrip error message on github
Rgds
Arthur
Verstuurd vanaf mijn iPhone
… Op 18 nov. 2019 om 06:55 heeft Soliine ***@***.***> het volgende geschreven:
Hi Arthcras, thanks a lot for using Datashare and for your feedback. I can't reproduce the bug so I have more questions for you:
"For some reason new documents added to the datashare folder are not shown with search document. I believe the issue relates to that the indexing is not working when adding new documents"
-> indeed, every time you add documents in the Datashare folder, you need to 'analyze documents' again so the new documents are indexed in Datashare and you can search them: https://icij.gitbook.io/datashare/all/analyze-documents. Does this answer your question?
"Even after reinstalling datashare conform manual and removing the whole application, still an old document is visible in the data share application; while it is not in the datashare folder anymore; also the new documents added to the datashare folder are not feasible"
-> if you want to remove documents from Datashare, you need to delete them from Datashare https://icij.gitbook.io/datashare/faq/can-i-remove-a-document-from-datashare
If you don't do this, they will remain indexed even if you uninstall the application and reinstall it later. Does this work on your side?
Please let us know if this answer your questions or if there is a bug that we haven't seen. Thanks a lot.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
It seems that the issue relates to the index function; as I did it; any old documents remains remain still in database and no new indexes are generated
Verstuurd vanaf mijn iPhone
… Op 18 nov. 2019 om 06:55 heeft Soliine ***@***.***> het volgende geschreven:
Hi Arthcras, thanks a lot for using Datashare and for your feedback. I can't reproduce the bug so I have more questions for you:
"For some reason new documents added to the datashare folder are not shown with search document. I believe the issue relates to that the indexing is not working when adding new documents"
-> indeed, every time you add documents in the Datashare folder, you need to 'analyze documents' again so the new documents are indexed in Datashare and you can search them: https://icij.gitbook.io/datashare/all/analyze-documents. Does this answer your question?
"Even after reinstalling datashare conform manual and removing the whole application, still an old document is visible in the data share application; while it is not in the datashare folder anymore; also the new documents added to the datashare folder are not feasible"
-> if you want to remove documents from Datashare, you need to delete them from Datashare https://icij.gitbook.io/datashare/faq/can-i-remove-a-document-from-datashare
If you don't do this, they will remain indexed even if you uninstall the application and reinstall it later. Does this work on your side?
Please let us know if this answer your questions or if there is a bug that we haven't seen. Thanks a lot.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
OK I am going to test with the Saudi Aramco document. Can you tell us which OS do you use and how much RAM you have? |
|
Thanks
Will let you know
Rgds
Arthur
Verstuurd vanaf mijn iPhone
… Op 18 nov. 2019 om 10:41 heeft Soliine ***@***.***> het volgende geschreven:
OK I am going to test with the Saudi Aramco document. Can you tell us which OS do you use and how much RAM you have?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Hi Datashare Team
Did you already have any news about the testing:
Best rgds
Arthur
Verstuurd vanaf mijn iPhone
… Op 18 nov. 2019 om 15:41 heeft Soliine ***@***.***> het volgende geschreven:
OK I am going to test with the Saudi Aramco document. Can you tell us which OS do you use and how much RAM you have?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Hello Arthur, |
|
Hi Team
Thanks for your reply
One of points i had that the documents were also not indexed; i just reduced the document added to a smaller document (copy attached),but also this document was not indexed.; could this be related to the same issue, or is there something else not working good. I believe their is something not going well with the index function.
Kindly let me know
Best regards
Arthur
… On Nov 21, 2019, at 11:37 AM, Soliine ***@***.***> wrote:
Hello Arthur,
We confirm that this document in particular is too big and complex to work on some machines right now. The PDF is 11,8 MB and is complex (columns, etc). The indexing of this document didn't work on my computer (Mac OS Mojave 10.14.6 with 16GB of RAM) but did work on my colleague's machine (Linux Ubuntu Core i7 10th generation, 16GB of RAM). It did work on my colleague's machine also because we use an ElasticSearch cluster. ElasticSearch went down on my machine. It is something we are going to improve, notably through a project of creating a Datashare Light #317 <#317>.
At the moment, you can try indexing documents that are less big and/or less complex.
Thanks a lot for your interest.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub <#311?email_source=notifications&email_token=AIF7NU3S75GHQY63TMOYUV3QUZQFXA5CNFSM4JOGDB5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEZYK3Q#issuecomment-557024622>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIF7NU4HDJKMCM4QQRFA423QUZQFXANCNFSM4JOGDB5A>.
|
|
Hello Arthur, |
|
Hi team
can send you the datashare model and datashare index files
Will That be ok
Rgds
Arthur
Verstuurd vanaf mijn iPhone
… Op 21 nov. 2019 om 17:10 heeft Soliine ***@***.***> het volgende geschreven:
Hello Arthur,
I cannot reproduce the bug on my side. I put 25 documents in my Datashare folder, analyze them and I get 25 documents in Datashare. So we will need more information in order to understand why some of your documents were not indexed. If it is not confidential, can you please share the logs of your terminal or the documents (you said 'copy attached' but I cannot see any screenshot at the moment)? Thanks a lot.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Just send the data with we transfer
Rgds
Arthur
Verstuurd vanaf mijn iPhone
… Op 21 nov. 2019 om 17:10 heeft Soliine ***@***.***> het volgende geschreven:
Hello Arthur,
I cannot reproduce the bug on my side. I put 25 documents in my Datashare folder, analyze them and I get 25 documents in Datashare. So we will need more information in order to understand why some of your documents were not indexed. If it is not confidential, can you please share the logs of your terminal or the documents (you said 'copy attached' but I cannot see any screenshot at the moment)? Thanks a lot.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Hi Team
Have also attached some logs, hope this clarifies, the issue i encounter.
noteL got also an error message “safari can not open”, which remains after several rei-nstallations
Best rgds
Arthur
On Nov 21, 2019, at 5:10 PM, Soliine ***@***.***> wrote:
Hello Arthur,
I cannot reproduce the bug on my side. I put 25 documents in my Datashare folder, analyze them and I get 25 documents in Datashare. So we will need more information in order to understand why some of your documents were not indexed. If it is not confidential, can you please share the logs of your terminal or the documents (you said 'copy attached' but I cannot see any screenshot at the moment)? Thanks a lot.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub <#311?email_source=notifications&email_token=AIF7NU3VLZA6J4IPP4PQFQLQU2XHDA5CNFSM4JOGDB5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE2YH7Y#issuecomment-557155327>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIF7NU734GU32MMSY3SV5LTQU2XHDANCNFSM4JOGDB5A>.
1:M 21 Nov 21:35:25.142 * Running mode=standalone, port=6379.
1:M 21 Nov 21:35:25.142 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 21 Nov 21:35:25.142 # Server initialized
1:M 21 Nov 21:35:25.142 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 21 Nov 21:35:25.149 * DB loaded from disk: 0.007 seconds
1:M 21 Nov 21:35:25.149 * Ready to accept connections
1:signal-handler (1574372355) Received SIGTERM scheduling shutdown...
1:C 21 Nov 21:39:17.626 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 21 Nov 21:39:17.628 # Redis version=4.0.1, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 21 Nov 21:39:17.628 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 21 Nov 21:39:17.651 * Running mode=standalone, port=6379.
1:M 21 Nov 21:39:17.653 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 21 Nov 21:39:17.653 # Server initialized
1:M 21 Nov 21:39:17.655 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
1:M 21 Nov 21:39:17.662 * DB loaded from disk: 0.006 seconds
1:M 21 Nov 21:39:17.662 * Ready to accept connections
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[2019-11-21T21:40:07,459][INFO ][o.e.n.Node ] [] initializing ...
[2019-11-21T21:40:07,862][INFO ][o.e.e.NodeEnvironment ] [tqCfowL] using [1] data paths, mounts [[/usr/share/elasticsearch/data (osxfs)]], net usable_space [12.5gb], net total_space [233.5gb], types [fuse.osxfs]
[2019-11-21T21:40:07,867][INFO ][o.e.e.NodeEnvironment ] [tqCfowL] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-11-21T21:40:08,194][INFO ][o.e.n.Node ] [tqCfowL] node name derived from node ID [tqCfowLYQZO_xusHdMoPUw]; set [node.name] to override
[2019-11-21T21:40:08,195][INFO ][o.e.n.Node ] [tqCfowL] version[6.3.0], pid[1], build[default/tar/424e937/2018-06-11T23:38:03.357887Z], OS[Linux/4.9.184-linuxkit/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/10.0.1/10.0.1+10]
[2019-11-21T21:40:08,196][INFO ][o.e.n.Node ] [tqCfowL] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.FVFspyqd, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -Des.cgroups.hierarchy.override=/, -Xmx2048m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2019-11-21T21:40:13,022][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [aggs-matrix-stats]
[2019-11-21T21:40:13,022][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [analysis-common]
[2019-11-21T21:40:13,023][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [ingest-common]
[2019-11-21T21:40:13,023][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [lang-expression]
[2019-11-21T21:40:13,024][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [lang-mustache]
[2019-11-21T21:40:13,025][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [lang-painless]
[2019-11-21T21:40:13,025][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [mapper-extras]
[2019-11-21T21:40:13,026][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [parent-join]
[2019-11-21T21:40:13,026][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [percolator]
[2019-11-21T21:40:13,026][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [rank-eval]
[2019-11-21T21:40:13,026][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [reindex]
[2019-11-21T21:40:13,026][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [repository-url]
[2019-11-21T21:40:13,027][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [transport-netty4]
[2019-11-21T21:40:13,027][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [tribe]
[2019-11-21T21:40:13,027][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-core]
[2019-11-21T21:40:13,028][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-deprecation]
[2019-11-21T21:40:13,028][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-graph]
[2019-11-21T21:40:13,029][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-logstash]
[2019-11-21T21:40:13,031][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-ml]
[2019-11-21T21:40:13,032][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-monitoring]
[2019-11-21T21:40:13,032][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-rollup]
[2019-11-21T21:40:13,033][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-security]
[2019-11-21T21:40:13,034][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-sql]
[2019-11-21T21:40:13,034][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-upgrade]
[2019-11-21T21:40:13,034][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded module [x-pack-watcher]
[2019-11-21T21:40:13,036][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded plugin [ingest-geoip]
[2019-11-21T21:40:13,036][INFO ][o.e.p.PluginsService ] [tqCfowL] loaded plugin [ingest-user-agent]
[2019-11-21T21:40:21,312][INFO ][o.e.x.s.a.s.FileRolesStore] [tqCfowL] parsed [0] roles from file [/usr/share/elasticsearch/config/roles.yml]
[2019-11-21T21:40:23,293][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/77] [Main.cc@109] controller (64 bit): Version 6.3.0 (Build 0f0a34c67965d7) Copyright (c) 2018 Elasticsearch BV
[2019-11-21T21:40:25,130][INFO ][o.e.d.DiscoveryModule ] [tqCfowL] using discovery type [single-node]
[2019-11-21T21:40:27,066][INFO ][o.e.n.Node ] [tqCfowL] initialized
[2019-11-21T21:40:27,068][INFO ][o.e.n.Node ] [tqCfowL] starting ...
[2019-11-21T21:40:27,597][INFO ][o.e.t.TransportService ] [tqCfowL] publish_address {172.18.0.2:9300}, bound_addresses {0.0.0.0:9300}
[2019-11-21T21:40:27,719][WARN ][o.e.b.BootstrapChecks ] [tqCfowL] initial heap size [1073741824] not equal to maximum heap size [2147483648]; this can cause resize pauses and prevents mlockall from locking the entire heap
[2019-11-21T21:40:27,833][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [tqCfowL] publish_address {172.18.0.2:9200}, bound_addresses {0.0.0.0:9200}
[2019-11-21T21:40:27,839][INFO ][o.e.n.Node ] [tqCfowL] started
[2019-11-21T21:40:29,466][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [tqCfowL] Failed to clear cache for realms [[]]
[2019-11-21T21:40:29,583][INFO ][o.e.l.LicenseService ] [tqCfowL] license [e9004ab6-3979-4fc2-82e0-ce5462d58452] mode [basic] - valid
[2019-11-21T21:40:29,681][INFO ][o.e.g.GatewayService ] [tqCfowL] recovered [1] indices into cluster_state
[2019-11-21T21:40:32,279][INFO ][o.e.c.r.a.AllocationService] [tqCfowL] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[local-datashare][3]] ...]).
[2019-11-21T21:40:57,934][WARN ][o.e.c.r.a.DiskThresholdMonitor] [tqCfowL] high disk watermark [90%] exceeded on [tqCfowLYQZO_xusHdMoPUw][tqCfowL][/usr/share/elasticsearch/data/nodes/0] free: 12.4gb[5.3%], shards will be relocated away from this node
[2019-11-21T21:40:57,936][INFO ][o.e.c.r.a.DiskThresholdMonitor] [tqCfowL] rerouting shards: [high disk watermark exceeded on one or more nodes]
[2019-11-21T21:41:28,006][WARN ][o.e.c.r.a.DiskThresholdMonitor] [tqCfowL] high disk watermark [90%] exceeded on [tqCfowLYQZO_xusHdMoPUw][tqCfowL][/usr/share/elasticsearch/data/nodes/0] free: 12.4gb[5.3%], shards will be relocated away from this node
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.5-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-mitie-4.21.5-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-ixapipe-4.21.5-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-corenlp-4.21.5-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-db-4.21.5-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-app-4.21.5-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
21:41:53,521 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
21:41:53,522 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
21:41:53,523 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.5-jar-with-dependencies.jar!/logback.xml]
21:41:53,525 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
21:41:53,525 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.5-jar-with-dependencies.jar!/logback.xml]
21:41:53,525 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-ixapipe-4.21.5-jar-with-dependencies.jar!/logback.xml]
21:41:53,525 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-app-4.21.5-jar-with-dependencies.jar!/logback.xml]
21:41:53,525 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-corenlp-4.21.5-jar-with-dependencies.jar!/logback.xml]
21:41:53,525 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-mitie-4.21.5-jar-with-dependencies.jar!/logback.xml]
21:41:53,569 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@6979e8cb - URL [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.5-jar-with-dependencies.jar!/logback.xml] is not of type file
21:41:53,747 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
21:41:53,764 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
21:41:53,788 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
21:41:53,804 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:41:53,948 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
21:41:53,953 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
21:41:53,988 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1983747920 - No compression will be used
21:41:53,990 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1983747920 - Will use the pattern ./logs/datashare_opennlp.%d{yyyy-MM-dd}.log for the active file
21:41:53,995 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern './logs/datashare_opennlp.%d{yyyy-MM-dd}.log'.
21:41:53,995 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
21:41:54,006 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Thu Nov 21 21:28:27 GMT 2019
21:41:54,009 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
21:41:54,012 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./logs/datashare.log
21:41:54,012 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./logs/datashare.log]
21:41:54,013 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
21:41:54,013 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
21:41:54,014 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
21:41:54,014 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
21:41:54,015 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@5c0369c4 - Registering current configuration as safe fallback point
2019-11-21T21:41:54.024035870Z
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
2019-11-21 21:41:54,333 [main] INFO Main - Running datashare web server
2019-11-21 21:41:54,361 [main] WARN PropertiesProvider - no null file found, using default values
2019-11-21 21:41:54,364 [main] INFO PropertiesProvider - properties set to {}
2019-11-21 21:41:54,364 [main] INFO PropertiesProvider - merged properties (without override) with {tcpListenPort=8080, stages=SCAN,INDEX,NLP, elasticsearchAddress=http://elasticsearch:9200, queueName=extract:queue, sessionTtlSeconds=43200, protectedUriPrefix=/api/, cors=no-cors, mode=LOCAL, redisAddress=redis://redis:6379, defaultUserName=local, ocr=true, nlpParallelism=1, parserParallelism=1, dataSourceUrl=jdbc:sqlite:/home/datashare/dist/database.sqlite, dataDir=/home/datashare/data, defaultProject=local-datashare, clusterName=datashare, messageBusAddress=redis, parallelism=2}
2019-11-21 21:41:56,170 [main] INFO HikariDataSource - HikariPool-1 - Starting...
2019-11-21 21:41:56,468 [main] INFO HikariDataSource - HikariPool-1 - Start completed.
2019-11-21 21:41:56,534 [main] INFO HikariDataSource - HikariPool-2 - Starting...
2019-11-21 21:41:56,544 [main] INFO HikariDataSource - HikariPool-2 - Start completed.
2019-11-21 21:41:56,842 [main] INFO HikariDataSource - HikariPool-3 - Starting...
2019-11-21 21:41:56,853 [main] INFO HikariDataSource - HikariPool-3 - Start completed.
2019-11-21 21:42:02,656 [main] INFO JdbcExecutor - SELECT COUNT(*) FROM DATABASECHANGELOGLOCK
2019-11-21 21:42:02,734 [main] INFO JdbcExecutor - SELECT COUNT(*) FROM DATABASECHANGELOGLOCK
2019-11-21 21:42:02,783 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:42:02,785 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:42:12,792 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:42:12,799 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:42:22,803 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:42:22,808 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:42:32,790 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:42:32,795 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:42:42,799 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:42:42,802 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:42:52,808 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:42:52,812 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:43:02,795 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:43:02,798 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:43:12,802 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:43:12,806 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:43:22,808 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:43:22,811 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:43:32,795 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:43:32,799 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:43:42,802 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:43:42,805 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:43:52,807 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:43:52,810 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:44:02,792 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:44:02,797 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:44:12,798 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:44:12,801 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:44:22,804 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:44:22,807 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:44:32,787 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:44:32,791 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:44:42,794 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:44:42,800 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:44:52,803 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:44:52,808 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:45:02,783 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:45:03,790 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:45:13,796 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:45:13,800 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:45:23,802 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:45:23,806 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:45:33,793 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:45:33,801 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:45:43,804 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:45:43,807 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:45:53,808 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:45:53,816 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:46:03,797 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:46:03,800 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:46:13,803 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:46:13,829 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:46:23,836 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:46:23,845 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:46:33,827 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:46:33,830 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:46:43,832 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:46:43,836 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:46:53,839 [main] INFO JdbcExecutor - SELECT LOCKED FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:46:53,842 [main] INFO StandardLockService - Waiting for changelog lock....
2019-11-21 21:47:03,828 [main] INFO JdbcExecutor - SELECT ID,LOCKED,LOCKGRANTED,LOCKEDBY FROM DATABASECHANGELOGLOCK WHERE ID=1
2019-11-21 21:47:03,837 [main] INFO HikariDataSource - HikariPool-3 - Shutdown initiated...
2019-11-21 21:47:03,886 [main] INFO HikariDataSource - HikariPool-3 - Shutdown completed.
Nov 21, 2019 9:47:03 PM com.google.inject.internal.MessageProcessor visit
INFO: An exception was caught and reported. Message: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
java.lang.RuntimeException: liquibase.exception.LockException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
at org.icij.datashare.db.RepositoryFactoryImpl.initDatabase(RepositoryFactoryImpl.java:50)
at org.icij.datashare.db.RepositoryFactoryImpl.initDatabase(RepositoryFactoryImpl.java:56)
at org.icij.datashare.mode.CommonMode.configurePersistence(CommonMode.java:91)
at org.icij.datashare.mode.LocalMode.configure(LocalMode.java:20)
at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
at com.google.inject.spi.Elements.getElements(Elements.java:110)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
at com.google.inject.Guice.createInjector(Guice.java:99)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at net.codestory.http.injection.GuiceAdapter.<init>(GuiceAdapter.java:28)
at org.icij.datashare.mode.CommonMode.defaultRoutes(CommonMode.java:95)
at org.icij.datashare.mode.CommonMode.lambda$createWebConfiguration$c4402707$1(CommonMode.java:82)
at net.codestory.http.routes.RouteCollection.configure(RouteCollection.java:88)
at net.codestory.http.reload.FixedRoutesProvider.<init>(FixedRoutesProvider.java:27)
at net.codestory.http.reload.RoutesProvider.fixed(RoutesProvider.java:29)
at net.codestory.http.AbstractWebServer.configure(AbstractWebServer.java:62)
at org.icij.datashare.WebApp.start(WebApp.java:22)
at org.icij.datashare.Main.main(Main.java:18)
Caused by: liquibase.exception.LockException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
at liquibase.lockservice.StandardLockService.listLocks(StandardLockService.java:399)
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:220)
at liquibase.Liquibase.update(Liquibase.java:184)
at liquibase.Liquibase.update(Liquibase.java:179)
at liquibase.Liquibase.update(Liquibase.java:175)
at org.icij.datashare.db.RepositoryFactoryImpl.initDatabase(RepositoryFactoryImpl.java:48)
... 20 more
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
at liquibase.lockservice.StandardLockService.listLocks(StandardLockService.java:388)
... 25 more
2019-11-21T21:47:03.974469234Z
Exception in thread "main" com.google.inject.CreationException: Unable to create injector, see the following errors:
2019-11-21T21:47:04.430242122Z
1) An exception was caught and reported. Message: liquibase.exception.LockException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
2019-11-21T21:47:04.430288623Z
1 error
at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:470)
at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:155)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:107)
at com.google.inject.Guice.createInjector(Guice.java:99)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at net.codestory.http.injection.GuiceAdapter.<init>(GuiceAdapter.java:28)
at org.icij.datashare.mode.CommonMode.defaultRoutes(CommonMode.java:95)
at org.icij.datashare.mode.CommonMode.lambda$createWebConfiguration$c4402707$1(CommonMode.java:82)
at net.codestory.http.routes.RouteCollection.configure(RouteCollection.java:88)
at net.codestory.http.reload.FixedRoutesProvider.<init>(FixedRoutesProvider.java:27)
at net.codestory.http.reload.RoutesProvider.fixed(RoutesProvider.java:29)
at net.codestory.http.AbstractWebServer.configure(AbstractWebServer.java:62)
at org.icij.datashare.WebApp.start(WebApp.java:22)
at org.icij.datashare.Main.main(Main.java:18)
Caused by: java.lang.RuntimeException: liquibase.exception.LockException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
at org.icij.datashare.db.RepositoryFactoryImpl.initDatabase(RepositoryFactoryImpl.java:50)
at org.icij.datashare.db.RepositoryFactoryImpl.initDatabase(RepositoryFactoryImpl.java:56)
at org.icij.datashare.mode.CommonMode.configurePersistence(CommonMode.java:91)
at org.icij.datashare.mode.LocalMode.configure(LocalMode.java:20)
at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
at com.google.inject.spi.Elements.getElements(Elements.java:110)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
... 12 more
Caused by: liquibase.exception.LockException: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
at liquibase.lockservice.StandardLockService.listLocks(StandardLockService.java:399)
at liquibase.lockservice.StandardLockService.waitForLock(StandardLockService.java:220)
at liquibase.Liquibase.update(Liquibase.java:184)
at liquibase.Liquibase.update(Liquibase.java:179)
at liquibase.Liquibase.update(Liquibase.java:175)
at org.icij.datashare.db.RepositoryFactoryImpl.initDatabase(RepositoryFactoryImpl.java:48)
... 20 more
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.util.Date
at liquibase.lockservice.StandardLockService.listLocks(StandardLockService.java:388)
... 25 more
|
|
Ok I think there is a database initialization issue here, I'm trying to reproduce. thanks for reporting @arthcras |
|
@arthcras I can't reproduce it with the latest version 4.21.6 on mac 10.14 Mojave either with an existent database or with a new one. If you don't have valuable stars and tags, I would recommend to remove the database file that is located in |
|
See comment #299 (comment) |
|
@arthcras Did you try to remove the database file before the fresh install like @bamthomas said ? |
|
Yes I did and the application is up and running however as said no documents are indexed; while new documents are in the folder
Verstuurd vanaf mijn iPhone
… Op 27 nov. 2019 om 16:22 heeft annelhote ***@***.***> het volgende geschreven:
@arthcras Did you try to remove the database file before the fresh install like @bamthomas said ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
I you wish we can said up a call so that I can show you the issue
Verstuurd vanaf mijn iPhone
… Op 27 nov. 2019 om 16:22 heeft annelhote ***@***.***> het volgende geschreven:
@arthcras Did you try to remove the database file before the fresh install like @bamthomas said ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
A call is planned on November 29th, 2019. |
|
After the call, it appears that the "forbidden" in ElasticSearch appears as a self protection when ElasticSearch does not have enough ressources (here due to arthcras' machine which is a Macbook Air). |
|
Thanks to the Datashare Team
For you support for this issue
Rgds
Arthur
… On Nov 29, 2019, at 4:10 PM, Soliine ***@***.***> wrote:
Closed #311 <#311>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#311?email_source=notifications&email_token=AIF7NU7SPMOCN2KNZTT3CSLQWEV2RA5CNFSM4JOGDB5KYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOVFQO5WI#event-2841702105>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIF7NU7UQK3QDNCSHUC5YFDQWEV2RANCNFSM4JOGDB5A>.
|
|
ElasticSearch wrongly halting indexing because of supposedly low disk free space <https://stackoverflow.com/questions/58998215/elasticsearch-wrongly-halting-indexing-because-of-supposedly-low-disk-free-space>
https://stackoverflow.com/questions/58998215/elasticsearch-wrongly-halting-indexing-because-of-supposedly-low-disk-free-space <https://stackoverflow.com/questions/58998215/elasticsearch-wrongly-halting-indexing-because-of-supposedly-low-disk-free-space>
Could me the issue
… On Nov 29, 2019, at 4:10 PM, Soliine ***@***.***> wrote:
Closed #311 <#311>.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#311?email_source=notifications&email_token=AIF7NU7SPMOCN2KNZTT3CSLQWEV2RA5CNFSM4JOGDB5KYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOVFQO5WI#event-2841702105>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AIF7NU7UQK3QDNCSHUC5YFDQWEV2RANCNFSM4JOGDB5A>.
|
|
Dear Team
This is to inform you that datashare is up and running; after freeing up space (above 15%) of total hard disk
Thanks for the effort and support very much appreciated
Rgds
Arthur
Verstuurd vanaf mijn iPhone
… Op 29 nov. 2019 om 16:10 heeft Soliine ***@***.***> het volgende geschreven:
Closed #311.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
That's good news, thanks Arthur! We take note of this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The issue is as follows
For some reason new documents added to the datashare folder are not shown with search document. I believe the issue relates to that the indexing is not working when adding new documents
Even after reinstalling datashare conform manual and removing the whole application, still an old document is visible in the data share application; while it is not in the datashare folder anymore; also the new documents added to the datashare folder are not feasible
2019-11-17 11:58:21,032 [pool-12-thread-2] ERROR DocumentConsumer - Exception while consuming file: "/home/datashare/data/saudi-aramco-prospectus-en.pdf".
java.net.ConnectException: Connection refused
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:949)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:229)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1593)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1563)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1546)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1512)
at org.elasticsearch.client.RestHighLevelClient.index(RestHighLevelClient.java:858)
at org.icij.datashare.text.indexing.elasticsearch.ElasticsearchSpewer.writeDocument(ElasticsearchSpewer.java:62)
at org.icij.spewer.Spewer.write(Spewer.java:56)
at org.icij.extract.extractor.Extractor.extract(Extractor.java:275)
at org.icij.extract.extractor.DocumentConsumer.lambda$accept$0(DocumentConsumer.java:125)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
... 1 common frames omitted
2019-11-17 11:58:21,215 [pool-2-thread-2] INFO DocumentConsumer - Terminated.
2019-11-17 11:58:22,448 [pool-2-thread-2] INFO IndexTask - exiting
2019-11-17 11:58:43,788 [pool-13-thread-2] ERROR DocumentConsumer - Exception while consuming file: "/home/datashare/data/saudi-aramco-prospectus-en.pdf".
java.net.ConnectException: Connection refused
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:949)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:229)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1593)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1563)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1546)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1512)
at org.elasticsearch.client.RestHighLevelClient.index(RestHighLevelClient.java:858)
at org.icij.datashare.text.indexing.elasticsearch.ElasticsearchSpewer.writeDocument(ElasticsearchSpewer.java:62)
at org.icij.spewer.Spewer.write(Spewer.java:56)
at org.icij.extract.extractor.Extractor.extract(Extractor.java:275)
at org.icij.extract.extractor.DocumentConsumer.lambda$accept$0(DocumentConsumer.java:125)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171)
at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
... 1 common frames omitted
2019-11-17 11:58:43,836 [pool-2-thread-1] INFO DocumentConsumer - Terminated.
2019-11-17 11:58:44,232 [pool-2-thread-1] INFO IndexTask - exiting
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-mitie-4.21.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-ixapipe-4.21.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-nlp-corenlp-4.21.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-db-4.21.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/datashare/lib/datashare-app-4.21.0-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
15:03:07,995 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
15:03:07,997 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
15:03:07,998 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.0-jar-with-dependencies.jar!/logback.xml]
15:03:08,002 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs multiple times on the classpath.
15:03:08,006 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-app-4.21.0-jar-with-dependencies.jar!/logback.xml]
15:03:08,006 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-mitie-4.21.0-jar-with-dependencies.jar!/logback.xml]
15:03:08,006 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-corenlp-4.21.0-jar-with-dependencies.jar!/logback.xml]
15:03:08,006 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-ixapipe-4.21.0-jar-with-dependencies.jar!/logback.xml]
15:03:08,006 |-WARN in ch.qos.logback.classic.LoggerContext[default] - Resource [logback.xml] occurs at [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.0-jar-with-dependencies.jar!/logback.xml]
15:03:08,086 |-INFO in ch.qos.logback.core.joran.spi.ConfigurationWatchList@6979e8cb - URL [jar:file:/home/datashare/lib/datashare-nlp-opennlp-4.21.0-jar-with-dependencies.jar!/logback.xml] is not of type file
15:03:08,508 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
15:03:08,537 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
15:03:08,569 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [STDOUT]
15:03:08,594 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:09,036 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
15:03:09,048 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [FILE]
15:03:09,467 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1983747920 - No compression will be used
15:03:09,476 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy@1983747920 - Will use the pattern ./logs/datashare_opennlp.%d{yyyy-MM-dd}.log for the active file
15:03:09,504 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern './logs/datashare_opennlp.%d{yyyy-MM-dd}.log'.
15:03:09,504 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
15:03:09,524 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Sun Nov 17 15:01:58 GMT 2019
15:03:09,534 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.classic.encoder.PatternLayoutEncoder] for [encoder] property
15:03:09,566 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - Active log file name: ./logs/datashare.log
15:03:09,566 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[FILE] - File property is set to [./logs/datashare.log]
15:03:09,575 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
15:03:09,576 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
15:03:09,578 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE] to Logger[ROOT]
15:03:09,578 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
15:03:09,581 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@5c0369c4 - Registering current configuration as safe fallback point
2019-11-17T15:03:09.600813300Z
2019-11-17 15:03:10,570 [main] INFO Main - Running datashare web server
2019-11-17 15:03:10,588 [main] INFO Main - with properties: {defaultUserName=local, nlpParallelism=1, parserParallelism=1, stages=SCAN,INDEX,NLP, elasticsearchAddress=http://elasticsearch:9200, defaultProject=local-datashare, messageBusAddress=redis, dataSourceUrl=jdbc:sqlite:/home/datashare/dist/database.sqlite, queueName=extract:queue, parallelism=2, mode=LOCAL, cors=no-cors, ocr=true, redisAddress=redis://redis:6379, dataDir=/home/datashare/data, clusterName=datashare}
2019-11-17 15:03:10,626 [main] INFO PropertiesProvider - reading properties from jar:file:/home/datashare/lib/datashare-app-4.21.0-jar-with-dependencies.jar!/datashare.properties
2019-11-17 15:03:10,776 [main] INFO PropertiesProvider - adding properties from env vars {mountedDataDir=/Users/arthurnamiasdecrasto/Datashare}
The text was updated successfully, but these errors were encountered: