Improve EarlyDeprecationindexingIT test reliability #105696
Conversation
this test intends to test the bulkProcessor2's request consumer before the templates are initialised. This requires the flush to happen before the templates are loaded. To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now) This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured. It also adds additional trace logging to help with troubleshooting. relates elastic#104716
pgomulka
added
>test
elasticsearchmachine
added
v8.14.0
Team:Core/Infra
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
| @@ -70,6 +70,7 @@ public void testEarlyDeprecationIsIndexedAfterTemplateIsLoaded() throws Exceptio | |||
| ) | |||
| ); | |||
| }, 30, TimeUnit.SECONDS); | |||
| fail(); | |||
There was a problem hiding this comment.
Looks like a left over from testing?
There was a problem hiding this comment.
ah yes :) at least it would be failing reliably lol :D
.../src/main/java/org/elasticsearch/xpack/deprecation/logging/DeprecationIndexingComponent.java
Outdated
Show resolved
Hide resolved
| setting 'xpack.security.enabled', 'false' | ||
| setting 'xpack.license.self_generated.type', 'trial' | ||
| setting 'cluster.deprecation_indexing.enabled', 'true' | ||
| setting 'cluster.deprecation_indexing.flush_interval', '1ms' |
There was a problem hiding this comment.
This seems infeasibly short. Would 100ms be a bit closer to real-world usage?
There was a problem hiding this comment.
the smallest the interval the better. We want the flush happen as soon as possible (to make sure it will be before the templates are loaded)
On my laptop it takes <500ms from the indexRequest being added into bulk processor to loading the templates. Could be that on CI machine it would be faster?
There was a problem hiding this comment.
My only concern is that 1ms might disrupt normal ES working, but if it works - fine
There was a problem hiding this comment.
disrupt normal ES working
this is the goal. indexing should not happen that early in startup so there is a queue of those too early requests to process later
one redundant log line
|
Could we put a note on #104716 to remove the trace logging once it is resolved |
|
@elasticmachine update branch |
this test intends to test the bulkProcessor2's request consumer (see DeprecationIndexingComponent#getBulkProcessor) scheduling requests before the startup is completed (flush is enabled). To verify this behaviour the flush has to happen before the templates are loaded. To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now) This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured. It also adds additional trace logging to help with troubleshooting. relates elastic#104716
…107540 this test intends to test the bulkProcessor2's request consumer (see DeprecationIndexingComponent#getBulkProcessor) scheduling requests before the startup is completed (flush is enabled). To verify this behaviour the flush has to happen before the templates are loaded. To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now) This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured. It also adds additional trace logging to help with troubleshooting. relates #104716
this test intends to test the bulkProcessor2's request consumer (see DeprecationIndexingComponent#getBulkProcessor) scheduling requests before the startup is completed (flush is enabled). To verify this behaviour the flush has to happen before the templates are loaded. To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now)
This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured. It also adds additional trace logging to help with troubleshooting.
relates #104716