Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Chore]: Update aws sdk to the latest version #509

Closed
kibertoad opened this issue Jul 11, 2024 · 7 comments · Fixed by #510
Closed

[Chore]: Update aws sdk to the latest version #509

kibertoad opened this issue Jul 11, 2024 · 7 comments · Fixed by #510

Comments

@kibertoad
Copy link

kibertoad commented Jul 11, 2024

Describe the bug

AWS SDK has introduced memory leak while updating one of the dependencies, namely due to this change: smithy-lang/smithy-typescript@c2a5595

See the bug issue: smithy-lang/smithy-typescript#1332

The problematic version of the dependency was introduced in AWS SDK in 3.609.0, due to smithy dependency updates, see https://github.com/aws/aws-sdk-js-v3/releases/tag/v3.609.0

For that particular reason, latest peerDependency of sqs-consumer is rather unfortunate:

"peerDependencies": {
    "@aws-sdk/client-sqs": "^3.609.0"
  },

There is a significant risk that users, unaware of the memory leak, may use one of the affected versions, due to the current peerDependency range.

This issue was resolved in the latest version, see aws/aws-sdk-js-v3@v3.613.0...v3.614.0#diff-1fc4f6a20ec76cddaa471b6988d57674be4ae65d3d2dbe6ac5e1645bd4ba8fadR47

I would strongly recommend to bump peerDependency version to 3.614.0 within a patch release, in order to eliminate the memory leak risk.

Your minimal, reproducible example

Steps to reproduce

Any SQS consumer running on a version between 3.609 to 3.613 will be consistently leaking memory over time.

Expected behavior

No memory leaks occur as SQS consumer is polling for messages.

How often does this bug happen?

Every time

Screenshots or Videos

No response

Platform

Any OS

Package version

10.3.0

AWS SDK version

3.609

Additional context

No response

@nicholasgriffintn
Copy link
Member

nicholasgriffintn commented Jul 11, 2024

Because we use a ^, it should download this version for anyone installing since the update, that said, people should manually check which we noted in the release (v11), which we decided to release before an AWS SDK release to inform users that they need to check their package lock.

AWS also uses a ^ for this package, so previous versions would also download the latest version.

So it shouldn't be that wide of an issue.

That said, we can certainly release a patch with the new version.

@nicholasgriffintn nicholasgriffintn changed the title [Bug]: Memory leak risk [Chore]: Update aws sdk to the latest version Jul 11, 2024
@kibertoad
Copy link
Author

@nicholasgriffintn That's true, which is why it's a risk and not a guaranteed memory leak. However, if someone updated to affected version specifically and stored pinned version in a lockfile, they would have a problem.

I'm not saying that sqs-consumer is making the problem worse, just that it can help to alleviate it at a low cost.

And affected versions definitely shouldn't be used in prod. We noticed the problem because our services started to OOM at a regular basis.

Copy link

🎉 This issue has been resolved in version 11.0.1-canary.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

Copy link

🎉 This issue has been resolved in version 11.0.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@nicholasgriffintn
Copy link
Member

Agreed. Just released 11.0.1 with the sdk version update. Links above ^.

@kibertoad
Copy link
Author

thank you!

Copy link

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants