Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] S3 Source poll delay will always call Thread.sleep() if poll delay is greater than 0 #1550

Closed
asifsmohammed opened this issue Jun 29, 2022 · 1 comment · Fixed by #1551
Labels
bug Something isn't working untriaged
Milestone

Comments

@asifsmohammed
Copy link
Collaborator

asifsmohammed commented Jun 29, 2022

Describe the bug
When we process a batch of SQS messages, we wait for some time before processing next batch. The wait time is based on number of messages processed and poll time. If poll time is greater than 0 and maximum messages is default value (which is 10 and it's the maximum value) SQS worker will quite likely have a delay because SQS might never return maximum number of messages.
SQS will return as soon as it gets at least one message even if maximum messages is set to 10.

if (messagesProcessed < sqsOptions.getMaximumMessages() && s3SourceConfig.getSqsOptions().getPollDelay().toMillis() > 0) {

Expected behavior
SQS should wait if at least one message is processed instead of checking against maximum messages.

@asifsmohammed asifsmohammed added bug Something isn't working untriaged labels Jun 29, 2022
@asifsmohammed asifsmohammed changed the title [BUG] S3 Source poll delay will always call Thread.sleep() if greater than 0 [BUG] S3 Source poll delay will always call Thread.sleep() if poll delay is greater than 0 Jun 29, 2022
@dlvenable
Copy link
Member

It might not be fair to say that it will always delay. :) But, it appears quite likely that it often will introduce the delay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working untriaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants