Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: deal with event with object size zero #3806

Conversation

chenqi0805
Copy link
Collaborator

Description

This PR fixes parsing on empty folder creation S3 event in s3 source by skipping the zero size object processing.

Issues Resolved

Resolves #3727

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@@ -53,6 +53,7 @@ public class SqsWorker implements Runnable {
static final String SQS_MESSAGES_DELETED_METRIC_NAME = "sqsMessagesDeleted";
static final String SQS_MESSAGES_FAILED_METRIC_NAME = "sqsMessagesFailed";
static final String SQS_MESSAGES_DELETE_FAILED_METRIC_NAME = "sqsMessagesDeleteFailed";
static final String ZERO_SIZED_OBJECT_COUNT_METRIC_NAME = "zeroSizedObject";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me except the metrics name. All the s3 metrics starts with s3 or s3Object. It can be s3ObjectsWithSizeZero or s3Zero SizedObjects or if you have any other suggestions.

Or we already have s3ObjectSizeBytes metric, so it can be s3ObjectSizeZeroBytes.

oeyh
oeyh previously approved these changes Dec 5, 2023
@@ -53,6 +53,7 @@ public class SqsWorker implements Runnable {
static final String SQS_MESSAGES_DELETED_METRIC_NAME = "sqsMessagesDeleted";
static final String SQS_MESSAGES_FAILED_METRIC_NAME = "sqsMessagesFailed";
static final String SQS_MESSAGES_DELETE_FAILED_METRIC_NAME = "sqsMessagesDeleteFailed";
static final String S3_OBJECTS_WITH_SIZE_ZERO_METRIC_NAME = "s3ObjectsWithSizeZero";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would s3ObjectsEmpty be a clearer and more concise name?

Signed-off-by: George Chen <[email protected]>
@dlvenable
Copy link
Member

@chenqi0805 , Should we backport this to 2.6.1?

@chenqi0805 chenqi0805 merged commit ad62f74 into opensearch-project:main Dec 5, 2023
36 of 49 checks passed
@chenqi0805 chenqi0805 deleted the fix/3727-skip-zero-sized-objects-in-s3-source branch December 5, 2023 19:09
opensearch-trigger-bot bot pushed a commit that referenced this pull request Dec 5, 2023
* FIX: deal with event with object size zero

Signed-off-by: George Chen <[email protected]>
(cherry picked from commit ad62f74)
chenqi0805 added a commit that referenced this pull request Dec 6, 2023
* FIX: deal with event with object size zero

Signed-off-by: George Chen <[email protected]>
(cherry picked from commit ad62f74)

Co-authored-by: Qi Chen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] S3 source processes SQS notification when S3 folder is created
4 participants