Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IngestAttachmentClientYamlTestSuiteIT fails on jdk 11 #31456

Closed
alpar-t opened this issue Jun 20, 2018 · 7 comments
Closed

IngestAttachmentClientYamlTestSuiteIT fails on jdk 11 #31456

alpar-t opened this issue Jun 20, 2018 · 7 comments
Labels
blocker :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP :Delivery/Build Build or test infrastructure jdk11 Team:Delivery Meta label for Delivery team >test-failure Triaged test failures from CI

Comments

@alpar-t
Copy link
Contributor

alpar-t commented Jun 20, 2018 •
edited
Loading

Reproducible with:

./gradlew :plugins:ingest-attachment:integTestRunner \
  -Dtests.seed=60D84CAF3939D163 \
  -Dtests.class=org.elasticsearch.ingest.attachment.IngestAttachmentClientYamlTestSuiteIT \
  -Dtests.security.manager=true -Dtests.locale=en-IO -Dtests.timezone=Europe/Minsk

on the feature/31230_gradle48_jdk11 feature branch.
(Make sure to re-enable the integTest task in Gradle, since there's a single test, couldn't use @AwaitsFix)

From the failure log:

   >                 \ Your document contained more than 30 characters, and so your requested\
   >                 \ limit has been reached. To receive the full text of the document,\
   >                 \ increase your limit. (Text up to the limit is however available).\n\
@alpar-t alpar-t added :Delivery/Build Build or test infrastructure :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP labels Jun 20, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@ywelsch ywelsch added the >test-failure Triaged test failures from CI label Jun 20, 2018
alpar-t added a commit that referenced this issue Jun 20, 2018
@alpar-t
Disable failing tests #31456
8b9b449
@alpar-t
Copy link
Contributor Author

alpar-t commented Jun 20, 2018

Failure in CI: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+31230_gradle48_jdk11+matrix-java-feature-branch/2/ES_BUILD_JAVA=java11,ES_RUNTIME_JAVA=java10,nodes=virtual&&linux/

@dadoonet
Copy link
Member

@tballison Does this ring a bell?

1> Caused by: java.lang.IllegalStateException: Can't overwrite cause with org.apache.tika.sax.WriteOutContentHandler$WriteLimitReachedException: Your document contained more than 30 characters, and so your requested limit has been reached. To receive the full text of the document, increase your limit. (Text up to the limit is however available).
  1> 	at java.base/java.lang.Throwable.initCause(Throwable.java:462)
  1> 	at org.apache.tika.sax.TaggedSAXException.<init>(TaggedSAXException.java:41)
  1> 	at org.apache.tika.sax.TaggedContentHandler.handleException(TaggedContentHandler.java:113)
  1> 	at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:148)
  1> 	at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
  1> 	at org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)
  1> 	at org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)
  1> 	at org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)
  1> 	at org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)
  1> 	at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:279)
  1> 	at org.apache.tika.parser.txt.TXTParser.parse(TXTParser.java:104)
  1> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
  1> 	... 16 more
  1> Caused by: org.apache.tika.sax.TaggedSAXException: Your document contained more than 30 characters, and so your requested limit has been reached. To receive the full text of the document, increase your limit. (Text up to the limit is however available).
  1> org.apache.tika.sax.WriteOutContentHandler$WriteLimitReachedException: Your document contained more than 30 characters, and so your requested limit has been reached. To receive the full text of the document, increase your limit. (Text up to the limit is however available).
  1> 	... 26 more
  1> Caused by: org.apache.tika.sax.WriteOutContentHandler$WriteLimitReachedException: Your document contained more than 30 characters, and so your requested limit has been reached. To receive the full text of the document, increase your limit. (Text up to the limit is however available).
  1> 	at org.apache.tika.sax.WriteOutContentHandler.characters(WriteOutContentHandler.java:141)
  1> 	at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
  1> 	at org.apache.tika.sax.xpath.MatchingContentHandler.characters(MatchingContentHandler.java:85)
  1> 	at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
  1> 	at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
  1> 	at org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)
  1> 	at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
  1> 	... 24 more

@tballison
Copy link

tballison commented Jun 20, 2018 •
edited
Loading

Y. This is the real culprit:

1> Caused by: java.lang.IllegalStateException: Can't overwrite cause with org.apache.tika.sax.WriteOutContentHandler$WriteLimitReachedException:

I fixed this in TIKA-2668.

I think we're coming up on a release of Tika 1.19 which will solve this. I want to fix this before that release:
TIKA-2674 -- update deps for Java 11

@alpar-t
Copy link
Contributor Author

alpar-t commented Jun 20, 2018

Some addition test failures with the same reason:

@jasontedor jasontedor mentioned this issue Jun 21, 2018
Closed
10 tasks
alpar-t added a commit to alpar-t/elasticsearch that referenced this issue Jun 28, 2018
@alpar-t
Disable failing tests elastic#31456
e433bce
@alpar-t alpar-t added the blocker label Jul 5, 2018
alpar-t added a commit that referenced this issue Jul 5, 2018
@alpar-t
Add JDK11 support and enable in CI (#31644)
cf2295b 
* Upgrade bouncycastle

Required to fix
`bcprov-jdk15on-1.55.jar; invalid manifest format `
on jdk 11

* Downgrade bouncycastle to avoid invalid manifest

* Add checksum for new jars

* Update tika permissions for jdk 11

* Mute test failing on jdk 11

* Add JDK11 to CI

* Thread#stop(Throwable) was removed

http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html

* Disable failing tests #31456

* Temprorarily disable doc tests

To see if there are other failures on JDK11

* Only blacklist specific doc tests

* Disable only failing tests in ingest attachment plugin

* Mute failing HDFS tests #31498

* Mute failing lang-painless tests #31500

* Fix backwards compatability builds

Fix JAVA version to 10 for ES 6.3

* Add 6.x to bwx -> java10

* Prefix out and err from buildBwcVersion for readability

```
> Task :distribution:bwc:next-bugfix-snapshot:buildBwcVersion
  [bwc] :buildSrc:compileJava
  [bwc] WARNING: An illegal reflective access operation has occurred
  [bwc] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/home/alpar/.gradle/wrapper/dists/gradle-4.5-all/cg9lyzfg3iwv6fa00os9gcgj4/gradle-4.5/lib/groovy-all-2.4.12.jar) to method java.lang.Object.finalize()
  [bwc] WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
  [bwc] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
  [bwc] WARNING: All illegal access operations will be denied in a future release
  [bwc] :buildSrc:compileGroovy
  [bwc] :buildSrc:writeVersionProperties
  [bwc] :buildSrc:processResources
  [bwc] :buildSrc:classes
  [bwc] :buildSrc:jar

```

* Also set RUNTIME_JAVA_HOME for bwcBuild

So that we can make sure it's not too new for the build to understand.

* Align bouncycastle dependency

* fix painles array tets

closes #31500

* Update jar checksums

* Keep 8/10 runtime/compile untill consensus builds on 11

* Only skip failing tests if running on Java 11

* Failures are dependent of compile java version not runtime

* Condition doc test exceptions on compiler java version as well

* Disable hdfs tests based on runtime java

* Set runtime java to minimum supported for bwc

* PR review

* Add comment with ticket for forbidden apis
alpar-t added a commit that referenced this issue Jul 5, 2018
@alpar-t
Add JDK11 support without enabling in CI (#31644)
1a8771e 
* Upgrade bouncycastle

Required to fix
`bcprov-jdk15on-1.55.jar; invalid manifest format `
on jdk 11

* Downgrade bouncycastle to avoid invalid manifest

* Add checksum for new jars

* Update tika permissions for jdk 11

* Mute test failing on jdk 11

* Add JDK11 to CI

* Thread#stop(Throwable) was removed

http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html

* Disable failing tests #31456

* Temprorarily disable doc tests

To see if there are other failures on JDK11

* Only blacklist specific doc tests

* Disable only failing tests in ingest attachment plugin

* Mute failing HDFS tests #31498

* Mute failing lang-painless tests #31500

* Fix backwards compatability builds

Fix JAVA version to 10 for ES 6.3

* Add 6.x to bwx -> java10

* Prefix out and err from buildBwcVersion for readability

```
> Task :distribution:bwc:next-bugfix-snapshot:buildBwcVersion
  [bwc] :buildSrc:compileJava
  [bwc] WARNING: An illegal reflective access operation has occurred
  [bwc] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/home/alpar/.gradle/wrapper/dists/gradle-4.5-all/cg9lyzfg3iwv6fa00os9gcgj4/gradle-4.5/lib/groovy-all-2.4.12.jar) to method java.lang.Object.finalize()
  [bwc] WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
  [bwc] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
  [bwc] WARNING: All illegal access operations will be denied in a future release
  [bwc] :buildSrc:compileGroovy
  [bwc] :buildSrc:writeVersionProperties
  [bwc] :buildSrc:processResources
  [bwc] :buildSrc:classes
  [bwc] :buildSrc:jar

```

* Also set RUNTIME_JAVA_HOME for bwcBuild

So that we can make sure it's not too new for the build to understand.

* Align bouncycastle dependency

* fix painles array tets

closes #31500

* Update jar checksums

* Keep 8/10 runtime/compile untill consensus builds on 11

* Only skip failing tests if running on Java 11

* Failures are dependent of compile java version not runtime

* Condition doc test exceptions on compiler java version as well

* Disable hdfs tests based on runtime java

* Set runtime java to minimum supported for bwc

* PR review

* Add comment with ticket for forbidden apis
@alpar-t alpar-t added the jdk11 label Jul 6, 2018
@tballison
Copy link

Tika 1.19 is out. Give it a try.

@jaymode jaymode mentioned this issue Sep 28, 2018
Merged
jkakavas added a commit that referenced this issue Oct 12, 2018
@jkakavas
[WIP] Ingest Attachement: Upgrade tika to v1.19.1 (#33896)
91f0b96 
Upgrades Tika to 1.19.1 and relevant transitive dependencies

Resolves: #31456, #31305
@tballison
Copy link

w00t! Thank you @jkakavas and team!

jkakavas added a commit that referenced this issue Oct 12, 2018
@jkakavas
[WIP] Ingest Attachement: Upgrade tika to v1.19.1 (#33896)
081597f 
Upgrades Tika to 1.19.1 and relevant transitive dependencies

Resolves: #31456, #31305
kcm pushed a commit that referenced this issue Oct 30, 2018
@jkakavas @kcm
[WIP] Ingest Attachement: Upgrade tika to v1.19.1 (#33896)
e4f8c43 
Upgrades Tika to 1.19.1 and relevant transitive dependencies

Resolves: #31456, #31305
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
blocker :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP :Delivery/Build Build or test infrastructure jdk11 Team:Delivery Meta label for Delivery team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@dadoonet @alpar-t @ywelsch @mark-vieira @tballison @elasticmachine