Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create and delete point in time for processing an index #2839

Merged
merged 1 commit into from
Jun 8, 2023

Conversation

graytaylor0
Copy link
Member

@graytaylor0 graytaylor0 commented Jun 6, 2023

Description

Create and delete point in time for an index to process it

  • Note: The deletePit request does not work when enable_sigv4 is set to true. I have not been able to debug, but it works properly when enable_sigv4 is false. I will create a separate issue to work on this bug in the future.

Issues Resolved

related to #1985

Check List

  • New functionality includes testing.
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@kkondaka
Copy link
Collaborator

kkondaka commented Jun 7, 2023

Please comment out the failing tests, if they are failing because of sigv4

@graytaylor0
Copy link
Member Author

graytaylor0 commented Jun 7, 2023

Please comment out the failing tests, if they are failing because of sigv4

A couple were failing from an unnecessary mock. I fixed those. No unit tests are failing from sigv4 issues. Manual testing was done to see the delete pit call fail with sigv4


public class OpenSearchAccessor implements SearchAccessor, ClusterClientFactory {

private static final Logger LOG = LoggerFactory.getLogger(OpenSearchAccessor.class);

static final String PIT_RESOURCE_LIMIT_ERROR_TYPE = "rejected_execution_exception";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error type seems more generic than the PIT logic. Are there other cases where we can get a RejectedExecutionException unrelated to PIT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably but what is the alternative? Check the message contains something unique to the PIT message? Even for other exception we would log it and then give up the index and wait to start processing again, which is ok

@graytaylor0 graytaylor0 merged commit 168a9a4 into opensearch-project:main Jun 8, 2023
MaGonzalMayedo pushed a commit to MaGonzalMayedo/data-prepper that referenced this pull request Jun 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants