Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change successfulSearchShardIndices to Set<Index> #16110

Merged
merged 3 commits into from
Oct 8, 2024

Conversation

dzane17
Copy link
Contributor

@dzane17 dzane17 commented Sep 27, 2024

Description

Change successfulSearchShardIndices to Set<Index>

Passing the entire Index object allows greater flexibility over index String names. successfulSearchShardIndices is used in the Query Insights plugin to determine query field types.

Related Issues

RFC opensearch-project/query-insights#69
PR opensearch-project/query-insights#130

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dzane17 dzane17 added the backport 2.x Backport to 2.x branch label Sep 27, 2024
Copy link
Contributor

✅ Gradle check result for 9a5dac6: SUCCESS

Copy link

codecov bot commented Sep 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.03%. Comparing base (a81b868) to head (058fb78).
Report is 7 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #16110      +/-   ##
============================================
+ Coverage     71.94%   72.03%   +0.08%     
- Complexity    64612    64729     +117     
============================================
  Files          5298     5301       +3     
  Lines        301952   302355     +403     
  Branches      43627    43681      +54     
============================================
+ Hits         217247   217805     +558     
+ Misses        66884    66704     -180     
- Partials      17821    17846      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

✅ Gradle check result for 3ce3c96: SUCCESS

@reta
Copy link
Collaborator

reta commented Sep 28, 2024

@dzane17 could you please provide more context behind this change: why do you need to expose the IndicesService to plugins? thank you

@reta
Copy link
Collaborator

reta commented Oct 3, 2024

@cwperks - Please correct me if I am wrong, but onIndexModule can only be leveraged for listening shard level search events, not request level. We need the search request listener for what we are trying to achieve here! cc: @reta

Got it, thanks @jainankitk , so here is the way we could make it work, which is a bit cumbersome but solves the dependency problem, the QueryInsightsService in injected as the service class:

    @Override
    public Collection<Class<? extends LifecycleComponent>> getGuiceServiceClasses() {
        return List.of(QueryInsightsService.class);
    }

Since QueryInsightsListener is created before QueryInsightsService (and it is injectable), we could inject QueryInsightsListener right there and initialize it with the indicesService or QueryInsightsService (fe using setter) since none of this services are needed during QueryInsightsListener initialization:

@Inject
    public QueryInsightsService(
        final ClusterSettings clusterSettings,
        final ThreadPool threadPool,
        final Client client,
        final MetricsRegistry metricsRegistry,
        final NamedXContentRegistry namedXContentRegistry,
        final IndicesService indicesService,
        final QueryInsightsListener listener
    ) {
  }

@dzane17 dzane17 changed the title Adding MapperServiceProvider for plugins Change successfulSearchShardIndices to Set<Index> Oct 7, 2024
CHANGELOG.md Outdated Show resolved Hide resolved
@dzane17
Copy link
Contributor Author

dzane17 commented Oct 7, 2024

Thanks all for the input. Since indicesService does not contain mappings for ALL indices in the cluster, we have no choice but to retrieve mappings from cluster state. This involves some clunky map parsing so we will introduce a small cache in Query-Insights to limit the frequency of this field->fieldType lookup. ClusterService is already present in the plugin createComponents method so I have reduced this PR to remaining structural changes needed in core.

Signed-off-by: Ankit Jain <[email protected]>
Copy link
Contributor

github-actions bot commented Oct 8, 2024

❌ Gradle check result for a489774: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

github-actions bot commented Oct 8, 2024

❌ Gradle check result for 058fb78: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@jainankitk
Copy link
Collaborator

jainankitk commented Oct 8, 2024

Seems due to caching changes:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/49029/testReport/) (1 failure / ±0)

    [org.opensearch.indices.IndicesRequestCacheCleanupIT.testDynamicStalenessThresholdUpdate](https://build.ci.opensearch.org/job/gradle-check/49029/testReport/junit/org.opensearch.indices/IndicesRequestCacheCleanupIT/testDynamicStalenessThresholdUpdate/)

@sgup432 - Can you take a look at this?

Copy link
Contributor

github-actions bot commented Oct 8, 2024

✅ Gradle check result for 058fb78: SUCCESS

@jainankitk jainankitk merged commit 5279d21 into opensearch-project:main Oct 8, 2024
33 of 34 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-16110-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 5279d21f8c85e78a91159af6e17e52789355f2a4
# Push it to GitHub
git push --set-upstream origin backport/backport-16110-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-16110-to-2.x.

@dzane17 dzane17 deleted the indices-service branch October 8, 2024 23:35
dzane17 added a commit to dzane17/OpenSearch that referenced this pull request Oct 8, 2024
…#16110)

Signed-off-by: David Zane <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
(cherry picked from commit 5279d21)
jainankitk pushed a commit that referenced this pull request Oct 9, 2024
Signed-off-by: David Zane <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
(cherry picked from commit 5279d21)
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 16, 2024
…#16110)

* Change successfulSearchShardIndices to Set<Index>

Signed-off-by: David Zane <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Ankit Jain <[email protected]>

---------

Signed-off-by: David Zane <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
Co-authored-by: Ankit Jain <[email protected]>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 17, 2024
…#16110)

* Change successfulSearchShardIndices to Set<Index>

Signed-off-by: David Zane <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Ankit Jain <[email protected]>

---------

Signed-off-by: David Zane <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
Co-authored-by: Ankit Jain <[email protected]>
dk2k pushed a commit to dk2k/OpenSearch that referenced this pull request Oct 21, 2024
…#16110)

* Change successfulSearchShardIndices to Set<Index>

Signed-off-by: David Zane <[email protected]>

* Update CHANGELOG.md

Signed-off-by: Ankit Jain <[email protected]>

---------

Signed-off-by: David Zane <[email protected]>
Signed-off-by: Ankit Jain <[email protected]>
Co-authored-by: Ankit Jain <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport-failed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants