Sync from elastic master #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We've been trying to shrink the big, mutable `SearchContext`. I'm doing my part by removing `BigArrays` from it. Doing that required reworking how we test `Aggregator`s to not need `SearchContext`. So I did that too. Mostly. `top_hits` still needs a `SubSearchContext` which we can still build, but it is now quite contained.
This adds a default `inference_config` object to the stored lang ident model. This doesn't change behavior as there are no default settings set in the config. Additionally, when serializing over the wire in older versions, we handle bwc. This commit also slightly changes the error message when a model stored as a resource is attempted to be deleted. relates: elastic/kibana#85179
Removed assertion that is flaky in CI and not really necessary to test HLRC integration. Relates to #60461
This commit sends a parent-task ban for each connection so that we can reply on channel disconnect instead of node leave events to remove bans.
After backporting #65707 we can now run it in our backwards compatibility tests.
This attempts to fix the flakiness in the LifecycleLicenseIT tests. This waits for an active license to be available in the cluster (REST layer comes up before) and adds busy waiting for setting/checking the licenses with logging of responses. Co-authored-by: Elastic Machine <[email protected]>
…al` (#65874) (#66031) Co-authored-by: James Rodewig <[email protected]> Co-authored-by: Elastic Machine <[email protected]> Co-authored-by: Ernesto Reig <[email protected]>
Looks like we've hit another "funny" eclipse compiler error. I filed https://bugs.eclipse.org/bugs/show_bug.cgi?id=569557 , but short version is that the following code compiles with javac and not eclipse: ``` package test; import java.util.function.Supplier; public class Test { private final Object o = new Object(); private class Inner { public Inner() { this(() -> o); } private Inner(Supplier<Object> o) {} } } ``` Specifically the outer class's `o` can't be refered to in the lambda. This stops us from doing that in ML so eclipse can be happy again.
Fixed a few remaining places where tier was still used, renamed to total like in most other places.
…gs (#65989) This commit adjusts the behavior when calculating the diff between two `AbstractScopedSettings` objects, so that the default values of settings whose default values depend on the values of other settings are correctly calculated. Previously, when calculating the diff, the default value of a depended setting would be calculated based on the default value of the setting(s) it depends on, rather than the current value of those settings.
This allows `date_histogram`s with `hard_bounds` and `extended_bounds` to use the "as range" style optimizations introducedin #63643. There isn't any work to do for `exended_bounds` besides add a test. For `hard_bounds` we have to be careful when constructing the ranges that to filter.
Its been several months and we haven't bumped into any good reason to rework the variable width histogram. So let's drop experimental from it! Closes #58573
Currently, our indexing pressure metrics only track the number of "bytes" related to in-progress indexing requests. This commit introduces tracking of the number of operations. Building out this metrics will add more information for future monitoring and back-pressure.
) * [DOCS] Qualified Watcher topics to distinguish from Kib Alerting * Reverted actions anchor due to client links. * Updated redirects. * Fixed links. * Roll back anchor changes * Apply suggestions from code review Co-authored-by: James Rodewig <[email protected]> * Incorporated review comments.
Relates to #64529. Currently the cluster filter variables `clusterRequireFilters` , `clusterIncludeFilters` and `clusterExcludeFilters` are non-static, so the new instance of `FilterAllocationDecider` inited in `SetSingleNodeAllocateStep ` in ILM cannot see the changes when updating the `cluster.routing.allocation.exclude._x` settings, and finally ILM will stuck in the shrink action if one excluded node has been selected in the `SetSingleNodeAllocateStep`. `AllocationRoutedStep` has the same issue. This main changes are: 1. Create `AllocationDeciders ` in the main method of `SetSingleNodeAllocateStep ` and `AllocationRoutedStep`, and the `FilterAllocationDecider ` is constructed using the cluster settings in the cluster metadata, so the cluster level filters can be seen when executing the steps. 2. Add some tests for the change.
Add field mappings for the cgroups metrics added to beats monitoring in elastic/beats#21113, and required by elastic/kibana#79050.
The token service was part of the Gold license, but due to its use as part of Cloud SSO, we are moving it down to Standard instead.
Autoscaling policies now have default deciders depending on which roles the policy governs.
Today we document the use of `_[networkInterface]_` to specify the addresses of a network interface but do not spell out which parts of this syntax should be taken literally and which are part of the placeholder for the interface name. If you get it wrong then the exception message is confusing too since it uses the results of `NetworkInterface#toString()` which contains much more than just the name of the interface. This commit clarifies the docs and the exception message. Closes #65978.
move integration tests to multi node tests to cover node to node communication
By default, if no pipe is specified, EQL will return return matchings from the tail, end of the stream, not the front. The results are returned in ascending order.
For async searches (EQL included) the client's request headers were erroneously stored in the .tasks index. This might expose the requesting client's HTTP Authorization header. This PR fixes that by employing the usual approach to store only the security-internal headers, which carry the authentication result, instead of the original Authorization header, which is commonly utilized to redo authentication for scheduled tasks.
Fixes MlScalingReason toxcontent serialization and adds test to catch regression.
This adds a new method to the AuditTrail that intercepts the responses of transport-level actions. This new method is unlike all the other existing audit methods because it is called after the action has been run (so that it has access to the response). After careful deliberation, the new method is called for the responses of actions that are intercepted by the `SecurityActionFilter` only, and not by the transport filter. In order to facilitate the "linking" of the new audit event with the other existing events, the audit method receives the requestId as well as the authentication as arguments (in addition to the request itself and the response). This is labeled non-issue because it is only the foundation upon which later features that actually print out (some) responses can be built upon. Related #63221
This reverts commit 1c059e7.
The XPackLicenseState is a utility to handle checking the currently configured license level against the required license level of each licensed feature. Originally all licensed features were paid features and licenses always had a limited time scope. However, since the basic license was introduced, all users have an unlimited time use of basic features; there is no actual "basic license", rather it is the base case for the default distribution. Therefore, the checks on basic features in license state are no-ops because the current license always at least allows basic features, and the basic license cannot be expired. This commit removes all of the features from the license state marked as BASIC or MISSING (a level lower than BASIC that also predates the basic license and is no longer relevant). It also adds an assertion that no basic license features can be added to the license state in the future.
JDBC client can only be configured for SSL with keystores, but we can't use JKS/PKCS12 keystores in FIPS 140-2 mode. Resolves: #66095
This commit introduces a new sort field called `_shard_doc` that can be used in conjunction with a PIT to consistently tiebreak identical sort values. The sort value is a numeric long that is composed of the ordinal of the shard (assigned by the coordinating node) and the internal Lucene document ID. These two values are consistent within a PIT so this sort criteria can be used as the tiebreaker of any search requests. Since this sort criteria is stable we'd like to add it automatically to any sorted search requests that use a PIT but we also need to expose it explicitly in order to be able to: * Reverse the order of the tiebreaking, useful to search "before" `search_after`. * Force the primary sort to use it in order to benefit from the `search_after` optimization when sorting by index order (to be released in Lucene 8.8. I plan to add the documentation and the automatic configuration for PIT in a follow up since this change is already big. Relates #56828
…66533) Where CCS is being used it makes sense for ML nodes to have the remote_cluster_client role. This single extra role is not significant enough to stop an ML node being considered a "dedicated ML node".
This adds caching the fact that `RepositoryData` was too large to be cached so we don't serialize it over and over just to find out we can't cache it.
Add integration tests for creating and auto-creating system indices. This leans on the work done to test SystemIndexManager, and indeed refactors those tests as well.
* [DOCS] Clarifying TLS steps * Fix file name * Clarifying note based on reviewer feedback
This adds `hyperparameters` section to the stored model metadata document. Here is an example
```json
"hyperparameters": [
{
"name": "downsample_factor",
"value": 0.35355339059327386,
"supplied": false,
"absolute_importance": 0.123,
"relative_importance": 0.9123
},
{
"name": "alpha",
"value": 0.010766199582740555,
"supplied": false,
"absolute_importance": 0.0123,
"relative_importance": 0.0321
},
{
"name": "gamma",
"value": 0.1,
"supplied": true,
"absolute_importance": 0.0,
"relative_importance": 0.0
}
]
```
Co-authored-by: AndyHunt66 <[email protected]>
See #66586 for details.
See #63980 for details
This PR fixes two bugs that can arise when _source is disabled and we fetch nested documents: * Fix exception when highlighting `inner_hits` with disabled _source. * Fix exception in nested `top_hits` with disabled _source. * Add more tests for highlighting `inner_hits`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
gradle check?