Script: Configuration driven validation for CtxMap #6
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently when sorting on incompatible types, we get class_cast_exception error (code 500). This patch improves the error to explain that the problem is because of incompatible sort types for the field across different shards and returns user error (code 400). Closes elastic#73146
Adds FieldProperties record that configures how to validate fields. Fields have a type, are writeable or read-only, and nullable or not and may have an additional validation useful for Set/Enum validation. Splits IngestMetadata from Metadata in preparation for new Metdata subclasses.
… into ingest_ctx_map_field_prop
…search into ingest_ctx_map_field_prop
This commit stops adding the geo_shape field mapper by default and adds the mapper only when it is needed.
This change replcase 2 hash map operations with a single one and a null check.
This class's maps are used very hot in the disk threshold allocation decider. Moving them from hppc maps to unmodifiable map wrapping `HashMap` has led to a measurable slowdown in the many-shards benchmark bootstrapping. Lets use immutable map copies here exclusively to make performance outright better and more predictable via a single implementation.
We need the metadata in a number of allocation deciders and pass it to other allocation methods. Passing it here avoids redundant lookups across deciders.
Some polishing of reworked LoggedExec task
This PR adds a new audit trail event for when API keys are updated.
…iveSizeErrors (elastic#88457) -1 is handled differently by the xcontent code path so this test will fail when `randomIntBetween` lands on -1. To fix, we add another integer for the xcontent test which starts at -2.
The API key ID generation is handled by the Request class since elastic#63221. This makes it possible to audit it when creating or granting API keys. This PR makes the necessary changes for it to happen. Relates: elastic#63221
1. Add trace log guards to avoid high cost method 2. Log the time it took to rollup a shard
…astic#88460) Instead of registering the plugin by default, implementations that need it are responsible on registering the plugin.
It's faster and easier to reason about if we always have an immutable collections map here and not have the type depend on what the last operation on the index service was.
* Corrected an incomplete sentence. * Update docs/reference/aggregations/metrics/avg-aggregation.asciidoc Co-authored-by: Christos Soulios <[email protected]> Co-authored-by: David Kilfoyle <[email protected]> Co-authored-by: Christos Soulios <[email protected]>
This PR adds a noop check for API key updates. If we detect a noop update, i.e., an update that does not result in any changes to the existing doc, we skip the index step and return updated = false in the response. This PR also extends test coverage around various corner cases.
But DEBUG (silent) logging of snapshot restore/completion when done in the context of CCR or searchable snapshots.
…ermark (elastic#88452) Fix this test unexpectedly being off by one by increasing the accuracy of the fp division (better to have a larger dividend and divisor) a little. I could easily reproduce the failure without the fix but with it, the test cases we use at least run accurate with the change. closes elastic#88433
…#88476) Currently, an abort (especially when triggered an index delete) can manifest as either an aborted snapshot exception, a missing index exception or an NPE. The latter two show up as noise in logs. This change catches effectively all of these cleanly as aborted snapshot exceptions so they don't get logged as warnings and avoids the NPE if a shard was removed from the index service concurrently by using the API that throws on missing shards to look it up.
…gest_ctx_map_field_prop
…apshot (elastic#88398) Add tracking for the number of invocations that have passed between a successful SLM snapshot and the most recent failure. These stats would be helpful for reporting on SLM policy health.
…elastic#88488) Two runtime fields script classes were placed into the package for the scripting fields api. These have been moved to the script package where the rest of the runtime fields script classes live. This is a purely mechanical change.
…ic#88401) Pull out the implementation of `Metadata` from `IngestSourceAndMetadata`. `Metadata` will become a base class extended by the update contexts: ingest, update, update by query and reindex. `Metadata` implements a map-like interface, making it easy for a class containing `Metadata` to implement the full `Map` interface.
…gest_ctx_map_field_prop
…x validate javadoc
This reverts commit a08afb8.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds FieldProperties record that configures how to validate
fields. Fields have a type, are writeable or read-only, and nullable
or not and may have an additional validation useful for Set/Enum
validation.
Splits IngestMetadata from Metadata in preparation for new Metdata
subclasses.