sqs consumers

[aspacca]

Enhancement

What does this PR do?

Why is it important?

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @aspacca? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-11-29T02:22:16.238+0000

• Duration: 131 min 7 sec

Test stats 🧪

Test	Results
Failed	0
Passed	5024
Skipped	340
Total	5364

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[aspacca]

@andrewkroh

For what I can see I think the real game change is your #33658

This PR could anyway help to scale out an sqs+s3 input on a single agent instance, giving the option to basically having a similar behaviour (specifically joint with your above PR) to set up different repeated single inputs for the same sqs queue

nit: I found the benchmark a little unclear, I've tried to make it more readable and effective (no matter how many msgs etc the constant receive was always returning 1 message)

I still have to add docs and default settings for number_of_sqs_consumers

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b sqs_consumers upstream/sqs_consumers
git merge upstream/main
git push upstream sqs_consumers

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b sqs_consumers upstream/sqs_consumers
git merge upstream/main
git push upstream sqs_consumers

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b sqs_consumers upstream/sqs_consumers
git merge upstream/main
git push upstream sqs_consumers

[jlind23]

Closing this as it is for now outdated and hasn't been worked on for quite some time. Feel free to reopen it if need be.