Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NO_REVIEW] JAMES-2717 Run everything on docker ES #2343

Conversation

trantienduchn
Copy link

Just for testing, It's not meant to be merged

@trantienduchn trantienduchn added this to the Sprint 7 - Robusta Bean milestone May 8, 2019
@trantienduchn trantienduchn self-assigned this May 8, 2019
@chibenwa chibenwa added the no_review Don't review this! label May 8, 2019
@chibenwa chibenwa changed the title [Part-1] JAMES-2717 Run all Jmap intergration test on docker ES [NO_REVIEW] [Part-1] JAMES-2717 Run all Jmap intergration test on docker ES May 8, 2019
@trantienduchn
Copy link
Author

[8366c2236c5ea127a50a1c7273005ca811392b2c] [ERROR]   CassandraMessageIdToImapUidDAOTest.updateShouldUpdateModSeq:203 

test this please

@trantienduchn trantienduchn force-pushed the remove-embeddedES-full-tests branch 2 times, most recently from b1e93e3 to 1332dec Compare May 8, 2019 10:59
@trantienduchn trantienduchn changed the title [NO_REVIEW] [Part-1] JAMES-2717 Run all Jmap intergration test on docker ES [NO_REVIEW] [Part-1][IT Tests] JAMES-2717 Run all Jmap intergration test on docker ES May 8, 2019
@trantienduchn trantienduchn force-pushed the remove-embeddedES-full-tests branch from 1332dec to dd13d1e Compare May 8, 2019 12:11
@trantienduchn trantienduchn changed the title [NO_REVIEW] [Part-1][IT Tests] JAMES-2717 Run all Jmap intergration test on docker ES [NO_REVIEW] JAMES-2717 Run everything on docker ES May 9, 2019
@trantienduchn trantienduchn force-pushed the remove-embeddedES-full-tests branch 2 times, most recently from 14aecc2 to 0631a35 Compare May 9, 2019 06:49
@trantienduchn
Copy link
Author

[0631a359951e7e08bc340bcc42ef35e63aeef938] 2019-05-09 10:03:19,917 WARN  [elasticsearch[Thermo][transport_client_worker][T#7]{New I/O worker #28863}] - [org.elasticsearch.netty.channel.socket.nio.AbstractNioSelector]- Unexpected exception in the selector loop.
[0631a359951e7e08bc340bcc42ef35e63aeef938] java.lang.OutOfMemoryError: Direct buffer memory
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at java.nio.Bits.reserveMemory(Bits.java:695)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.channel.socket.nio.SocketReceiveBufferAllocator.newBuffer(SocketReceiveBufferAllocator.java:64)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.channel.socket.nio.SocketReceiveBufferAllocator.get(SocketReceiveBufferAllocator.java:41)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:62)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[0631a359951e7e08bc340bcc42ef35e63aeef938] 	at java.lang.Thread.run(Thread.java:748)

Not so good

@chibenwa
Copy link
Member

chibenwa commented May 9, 2019

Can you try to restart ES container let's say every 100 test, just to see?

@chibenwa
Copy link
Member

chibenwa commented May 9, 2019

Or try closing the ES ClientProvider when shutting down James server - this is some kind of leak related to connection not being reused like it was done with the Embedded ES

@trantienduchn trantienduchn force-pushed the remove-embeddedES-full-tests branch from 99e45cb to 4bd41b8 Compare May 9, 2019 11:24
@trantienduchn
Copy link
Author

trantienduchn commented May 10, 2019

[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR] Failures: 
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   CassandraBulkOperationTest.setMessagesShouldFailForHugeNumberOfEmailsToTrashWhenChunksConfigurationAreTooBig:118 1 expectation failed.
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] Expected status code <400> but was <200>.
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] 
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR] Errors: 
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   DeletedMessagesVaultTest.setup:130 » Creation Unable to create injector, see t...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [ERROR]   SetVacationResponseTest.setup:70 » Creation Unable to create injector, see the...



[4bd41b82bd7007626f4b43c70eb3429998e9ab04] [INFO] Apache James :: Server :: JMAP :: Cassandra Integration testing FAILURE [  02:00 h]

I tested and saw that the CassandraBulkOperationTest.setMessagesShouldFailForHugeNumberOfEmailsToTrashWhenChunksConfigurationAreTooBig() is already failed with embedded ES. seems it's be like that for a while. I'm investigating

Update1: seems like the docker cassandra has been not available, and this log is generated which I can't see in other builds

[4bd41b82bd7007626f4b43c70eb3429998e9ab04] 2019-05-09 13:50:33,158 ERROR [cluster311-nio-worker-4] - [reactor.core.publisher.Operators]- Operator called default onErrorDropped
[4bd41b82bd7007626f4b43c70eb3429998e9ab04] com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /192.168.0.1:48975 (com.datastax.driver.core.exceptions.TransportException: [/192.168.0.1:48975] Connection has been closed))

@trantienduchn trantienduchn force-pushed the remove-embeddedES-full-tests branch from 4bd41b8 to 425debc Compare May 10, 2019 03:28
@trantienduchn
Copy link
Author

[425debcb08865f428cc2c5e9dd7f9df20785df60] [ERROR] shouldSendANoticeWhenThresholdExceeded  Time elapsed: 3.115 s  <<< ERROR!
[425debcb08865f428cc2c5e9dd7f9df20785df60] java.lang.OutOfMemoryError: GC overhead limit exceeded

@trantienduchn trantienduchn force-pushed the remove-embeddedES-full-tests branch from 435ba5e to df30131 Compare May 10, 2019 10:44
@trantienduchn
Copy link
Author

remember to create a ticket about the cassandra bulk operation bug

@trantienduchn
Copy link
Author

trantienduchn commented May 13, 2019

remember to create a ticket about the cassandra bulk operation bug

https://issues.apache.org/jira/browse/JAMES-2758

@trantienduchn trantienduchn force-pushed the remove-embeddedES-full-tests branch from df30131 to 0832c30 Compare May 13, 2019 04:50
@trantienduchn
Copy link
Author

[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [ERROR] Failures: 
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [ERROR]   RabbitMQEventBusTest$LifeCycleTest$SingleEventBus.dispatchShouldWorkAfterNetworkIssuesForOldRegistration:323 
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] Wanted but not invoked:
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] mailboxListener.event(
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496]     org.apache.james.mailbox.events.MailboxListener$MailboxAdded@bf1a1130
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] );
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] -> at org.apache.james.mailbox.events.RabbitMQEventBusTest$LifeCycleTest$SingleEventBus.dispatchShouldWorkAfterNetworkIssuesForOldRegistration(RabbitMQEventBusTest.java:323)

test this please

@trantienduchn
Copy link
Author

good new :)

[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [INFO] Apache James :: Server :: JMAP :: Cassandra Integration testing SUCCESS [28:04 min]
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [ERROR] Errors: 
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [ERROR]   SpamAssassinContract.movingAMailToTrashShouldNotImpactSpamassassinLearning:370 » ConditionTimeout
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [INFO] 
[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [ERROR] Tests run: 718, Failures: 0, Errors: 1, Skipped: 3

test this please

@trantienduchn
Copy link
Author

[0832c30b6a4dd359d6de5b23ca9f550ea55f6496] [ERROR]   MailRepositoryContract.storingAndRemovingMessagesConcurrentlyShouldLeadToConsistentResult:475 » Execution

test this please

@trantienduchn
Copy link
Author

the goal is achieved

vttranlina pushed a commit to vttranlina/james-project that referenced this pull request Jul 15, 2024
…er side data (linagora#2343)

This could help in the case the operator reverses James to an older version that does not support the recently introduced TypeName.
- PushSubscription/get and PushListener would not fail when TypeStateFactory parsing the unknown typename from DB
- Deserialize StateChangeEvent would not fail leads to resilient handling of StateChangeEvent

Meanwhile, parsing user input, we need to be strict.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no_review Don't review this!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants