Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update #1

Merged
merged 6,894 commits into from
Oct 16, 2024
Merged

update #1

merged 6,894 commits into from
Oct 16, 2024

Conversation

gitworkflows
Copy link

  • Have you signed the contributor license agreement?
  • Have you followed the contributor guidelines?
  • If submitting code, have you built your formula locally prior to submission with gradle check?
  • If submitting code, is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed.
  • If submitting code, have you checked that your submission is for an OS and architecture that we support?
  • If you are submitting this code for a class then read our policy for that.

Copy link

Documentation preview:

DaveCTurner and others added 29 commits October 9, 2024 06:59
Today when we reboot a node in a test case derived from
`AbstractCoordinatorTestCase` we lose the contents of
`blackholedRegisterOperations`, but it's important that these operations
_eventually_ run. With this commit we copy these operations over into
the new node.
* Minimize storing array source

* restrict to fields

* revert changes for `addIgnoredFieldFromContext`

* fix test

* spotless

* count nulls
Also changes mappings from body_* to body.*
…related (#114326)

Revert "Fix BWC for file-settings based role mappings (#113900)" and related changes. Reverted commits:

- 763764c
- bc8f9dc
- ce07060

This is due to a bug in the above fix. We will reintroduce a paired down version of the fix in a subsequent PR.
As per request #114315 (comment) doing the PR on the main branch.
* Minimize storing array source

* restrict to fields

* revert changes for `addIgnoredFieldFromContext`

* fix test

* spotless

* count nulls

* Avoid noisy errors in testSyntheticSourceKeepArrays

* update

* update

* update

* update
* Entitlements for System.exit

* Respond to Simon's comments

* Rename trampoline -> bridge

* Require exactly one bridge jar

* Use Type helpers to generate descriptor strings

* Various cleanup from PR comments

* Remove null "receiver" for static methods

* Use List<Type> instead of voidDescriptor

* Clarifying comment

* Whoops, getMethod

* SuppressForbidden System.exit

* Spotless

* Use embedded provider plugin to keep ASM off classpath

* Oops... forgot the punchline

* Move ASM license to impl

* Use ProviderLocator and simplify bridgeJar logic

* Avoid eager resolution of configurations during task configuration

* Remove compile-time dependency agent->bridge

---------

Co-authored-by: Mark Vieira <[email protected]>
This PR default-enables cluster-state role mappings as the first part of the mitigation for a regression in ECK introduced by #107410. 

Prior to this PR, cluster-state role mappings were written to cluster-state, but not read from it. 

With this PR, cluster-state role mappings will be read and used to assign roles to users, i.e. in user role resolution. 

However, they will not be included in the output of the [Get role mappings API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-get-role-mapping.html) yet. Exposing them via API is a target for a follow-up fix.

Relates: ES-9628
Supersedes: #113900
…st {p0=synonyms/60_synonym_rule_get/Synonym set not found} #114432
this workarounds an issue we currently see on windows ci boxes where we
run into timeouts in this step in our builds
These look to have been muted due to suite timeouts that we've since
fixed. Let's try running these again.

Closes #109687
Closes #112144
Closes #112624
Closes #113315
Closes #113316
Closes #113327
Closes #113340
We are getting InterruptedExceptions ~1% of the time when running these
tests, but we can remove the threading from this test and still verify
the one-by-one behavior.

Fix #112471

Co-authored-by: Elastic Machine <[email protected]>
…st {p0=synonyms/60_synonym_rule_get/Get a synonym rule} #114443
…st {p0=synonyms/60_synonym_rule_get/Synonym rule not found} #114444
…ce, and AlibabaCloudSearchService (#113981)

* Adding chunking settings to GoogleVertexAiService, AzureAiStudioService, and AlibabaCloudSearchService

* Update docs/changelog/113981.yaml

* Updating AlibabaService chunkedInfer to handle sparse embedding task types

---------

Co-authored-by: Elastic Machine <[email protected]>
This removes `ccs_telemetry` feature flag, and instead introduces an
undocumented, true by default setting: - `search.ccs.collect_telemetry`
- enables CCS search telemetry collection and
`_cluster/stats?include_remote=true`. Can be disabled if this is causing
any problems.
* (Doc+) Cross-link CAT APIs to parent object

---------

Co-authored-by: Lisa Cawley <[email protected]>
Co-authored-by: shainaraskas <[email protected]>
This type parameter is only needed so that the `.timeout(TimeValue)`
method returns a request of the right type, but this still requires an
unchecked cast. Yet there's no real need to return anything from this
method, we can just use a regular setter. This commit does that.
Here we are testing mappings of `host` and `timestamp` fields as they are
used as default fields to sort on when using LogsDB. LogsDB uses a
`host.name` field mapped as a `keyword` and a `@timestamp` field, required
by data streams. Some mappings throw errors as a result of incompatibilities
when trying to merge object fields. Such errors are expected.
…ence/30_semantic_text_inference/Calculates embeddings using the default ELSER 2 endpoint} #114412
This PR applies a temporary patch to fix an issue with ordered and unordered intervals source.
The flattening that is applied in Lucene modifies the final gap preventing valid queries to match.
The fix already exists in Lucene but will be released in Lucene 10.x later this year.
Since the bug prevents the combination of ordered and unordered intervals with gaps, this change applies
a workaround to ensure that the bug is fixed in Elasticsearch 8x.

Relates #113554
…ence/40_semantic_text_query/Query a field that uses the default ELSER 2 endpoint} #114376
elasticsearchmachine and others added 27 commits October 16, 2024 16:05
In v9 the `getRestApiVersion()` method on `RestRequest`,
`XContentBuilder` and `XContentParser` can never return `V_7`, so we can
replace all the expressions of the form `$x$.getRestApiVersion() == V_7`
with `false`. This commit does that, and then refactors away the
resulting dead code using (largely) automated transformations.
S3 register reads are subject to the regular client retry policy, but in
practice we see failures of these reads sometimes for errors that are
transient but for which the SDK does not retry. This commit adds another
layer of retries to these reads.

Relates ES-9721
…haracters (#114837)

* Add logs and headers

For pipeline creation when name is invalid

* Fix YAML tests and add YAML test for warnings

* Update docs/changelog/114837.yaml

* Changelog entry

* Changelog entry

* Update docs/changelog/114837.yaml

* Changelog entry
…r options internally. (#114812)

**Introduction**

> In order to make adoption of failure stores simpler for all users, we
are introducing a new syntactical feature to index expression
resolution: The selector. > > Selectors, denoted with a :: followed by a
recognized suffix will allow users to specify which component of an
index abstraction they would like to operate on within an API call. In
this case, an index abstraction is a concrete index, data stream, or
alias; Any abstraction that can be resolved to a set of indices/shards.
We define a component of an index abstraction to be some searchable unit
of the index abstraction. > > To start, we will support two components:
data and failures. Concrete indices are their own data components, while
the data component for index aliases are all of the indices contained
therein. For data streams, the data component corresponds to their
backing indices. Data stream aliases mirror this, treating all backing
indices of the data streams they correspond to as their data component.
>  > The failure component is only supported by data streams and data
stream aliases. The failure component of these abstractions refer to the
data streams' failure stores. Indices and index aliases do not have a
failure component.

For more details and examples see
#113144. All this work has
been cherry picked from there.

**Purpose of this PR**

This PR is replacing the `FailureStoreOptions` with the
`SelectorOptions`, there shouldn't be any perceivable change to the user
since we kept the query parameter "failure_store" for now. It will be
removed in the next PR which will introduce the parsing of the
expressions. 

_The current PR is just a refactoring and does not and should not change
any existing behaviour._
This method just delegates to `getLocalNode()`, we may as well call the
more widely-used method with the shorter name directly.
* Use DataType.isString

* Add DataType.stringTypes()

* Fix shouldHideSignature check
* Fix NPE in AdaptiveAllocationsScalerService

* Update docs/changelog/114880.yaml

* Delete docs/changelog/114880.yaml
Fixes #114588
Fixes #114587
Fixes #114586
Fixes #114585
Fixes #113008
Fixes #113007
Fixes #113006
Fixes #113005

Fixed the long precision issue by allowing a +/-1 range.

Also made a minor refactor to simplify using different matchers for different types.
…114713)

Any similar functionality in the future should use capabilities instead
Even with Kahan summation, we were occasionally getting floating point differences at the 14th decimal point, well beyond anything a GIS use case would care about.
This was returning an invalid result `POINT(NaN NaN)` and now instead returns `null`.
…ster.stats/30_ccs_stats/cross-cluster search stats search} #114902
* Reset array scope tracking for nested objects

* update

* update

* update
Two tests shared the same name in `ToDatetimeTests`, so that needed
fixing. But then also the ranges in the masked test needed adjusting
after the change that added the masking test.

Fixes #108093
…dex mode (#114573)

Here we check for the existence of a `host.name` field in index sort settings
when the index mode is `logsdb` and decide to inject the field in the mapping
depending on whether it exists or not. By default `host.name` is required for
sorting in LogsDB. This reduces the chances for errors at mapping or template
composition time as a result of injecting the `host.name` field only if strictly
required. A user who wants to override index sort settings without including
a `host.name` field would be able to do so without finding an additional
`host.name` field in the mappings (injected automatically). If users override the
sort settings and a `host.name` field is not included we don't need
to inject such field since sorting does not require it anymore.

As a result of this change we have the following:
* the user does not provide any index sorting configuration: we are responsible for injecting the default sort fields and their mapping (for `logsdb`)
* the user explicitly provides non-empty index sorting configuration: the user is also responsible for providing correct mappings and we do not modify index sorting or mappings

Note also that all sort settings `index.sort.*` are `final` which means doing this
check once, when mappings are merged at template composition time, is enough.
We will deprecate the `_source.mode` mapping level configuration
in favor of the index-level `index.mapping.source.mode` setting.
As a result, we go through the documentation and update it to reflect
the introduction of the setting.
* (Doc+) link video for resolving shards too large

👋 howdy, team (cc: @anniegale9538 )!  Playing forward #111254, [this video](https://www.youtube.com/watch?v=sHyNYnwbYro) demonstrates an example resolving shards too large via reindex under [this section](https://www.elastic.co/guide/en/elasticsearch/reference/master/size-your-shards.html#shard-size-recommendation) as it's a top support ask.

---------

Co-authored-by: Liam Thompson <[email protected]>
* (Doc+) Cross-link max shards

👋 It appears we have two docs of similar content about max open shards. This one contains the error users search (so is what we linked the error to in #110993) but the other I believe is a placeholder doc for the health api code. Should maybe consolidate some day but in the mean time at least cross-link.

---------

Co-authored-by: Liam Thompson <[email protected]>
@gitworkflows gitworkflows merged commit ee97dd3 into gsoc2:main Oct 16, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.