"Modularized packages" are still large

[kyeotic]

Describe the bug

One of the goals of the SDKv3 was to provide modularized packages to reduce installation size. I believe this is currently a failed goal.

Your environment

Mac OS (should be the same on all OSes)

SDK version number

"dependencies": {
    "@aws-sdk/client-dynamodb": "^3.28.0",
    "@aws-sdk/lib-dynamodb": "^3.28.0",
    "@aws-sdk/types": "^3.25.0"
  }

Is the issue in the browser/Node.js/ReactNative?

NodeJS

Steps to reproduce

Install the above packages for DynamoDB

Observed behavior

Install size is 24mb

Expected behavior

Install size is smaller

Screenshots

If applicable, add screenshots to help explain your problem.

Here is a disk-usage size and screenshot of all the sub-packages installed

For comparison, here is the entire AWS SDK v2

Just DynamoDB is 1/3 the size of the entire v2 SDK. You can get larger than the entire v2 SDK by including 4(!) packages if you pick the right ones.

Additional context

This problem seems to be three-fold

1. The "base" packages, the ones used by all modules, includes packages that are unused or unnecessary (e.g. browser support packages for node)
2. the build targets used create bloat and disable tree-shaking
3. The distributed files include everything: source code, transpiled code for each platforms, dotfiles, types, documentation, etc.

These problems are all recursive. Each package in @aws-sdk has them, so they appear in descendent dependencies. Just fixing 3️⃣ will drastically improve the bundle size as each layer will get smaller, providing an exponential decrease in total size.

Fixing this will require a global approach to development across all packages. I understand that this is a tall order, but if modularized packages was the primary purpose for v3's architecture it is the only way to actually achieve meaningful results.

Recommendations:

• - publish at least two different versions to npm. One for node, one for the browser. The dependencies included to support both represent a significant portion of the bloat.
• - support tree-shaking for nodejs build, see esm build targets es5 #2056
• - target only LTS versions of Node instead of es5, see esm build targets es5 #2056
• - use package.json['files'] to limit published code to the compiled code (remove src code from the publish)

[trivikr]

@kyeotic The AWS SDK for JavaScript team is working on reducing the install size of v3.
Please test or create issues on our experimental repo if you have additional questions/suggestions https://github.com/trivikr/temp-client-s3

You can also share your additional comments/suggestions in this issue.

[kyeotic]

@trivikr It seems you've already demonstrated the value of some of these suggestions, like not publishing source. When can we expect some of these learnings to make there way back into the AWS SDK v3?

[kyeotic]

@trivikr I think another very valuable modularization would be to remove all the credentials detection out of these packages. If I want to use DynamoDB in a given context and already know where the credentials are and have access, 100-300kb of code designed to look in a dozen places is dead weight. For users that do want auto-detection they can of course pull in those packages in, but being able to omit them will see a significant size reduction for users that can do without.

[trivikr]

When can we expect some of these learnings to make there way back into the AWS SDK v3?

We are aiming to get these improvements published within a week or two.

[trivikr]

I think another very valuable modularization would be to remove all the credentials detection out of these packages.

This is a great idea. We can experiment with dynamic import of credential providers only when credentials are not explicitly provided. We'll explore it.

[simonbuchan]

@trivikr If you're looking for install size improvements, then this would be the breaking change of making e.g. @aws-sdk/credential-provider-node an optional peer dependency?

[trivikr]

Closing this issue as the AWS SDK for JavaScript team has reduced install size by ~35%
Tweet https://twitter.com/trivikram/status/1446596564969930753

We've created a feature request for splitting CJS and ES versions in #2889
If there are other specific suggestions, please create a new feature request.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.