Reduce install and bundle size by optimizing code gen

[everett1992]

Describe the feature

The sdk code gen creates a lot of duplicate code for every command and type in the service model. Optimizing the code generator to use more helper functions should reduce the size of each client - affecting local install size, bundle size, parsing time, and potentially runtime performance. I'm not confident about runtime performance, but my understanding is that javascript runtimes are better at optimizing functions that are called repeatedly.

Examples from @aws-sdk/[email protected]

dist-cjs/models/models_0.js¹

const ArchivalSummaryFilterSensitiveLog = (obj) => ({ ...obj });
exports.ArchivalSummaryFilterSensitiveLog = ArchivalSummaryFilterSensitiveLog;
const AttributeDefinitionFilterSensitiveLog = (obj) => ({ ...obj, });
exports.AttributeDefinitionFilterSensitiveLog = AttributeDefinitionFilterSensitiveLog;

repeated 154 times for different functions.
This could be reduced with a helper saving ~18 bytes per type.

const assign = (obj) => ({ ...obj });
const ArchivalSummaryFilterSensitiveLog = assign;
exports.ArchivalSummaryFilterSensitiveLog = ArchivalSummaryFilterSensitiveLog;
const AttributeDefinitionFilterSensitiveLog = assign;
exports.AttributeDefinitionFilterSensitiveLog = AttributeDefinitionFilterSensitiveLog;

dist-cjs/protocols/Aws_json1_0.js

Every command matches this pattern

const serializeAws_json1_0BatchExecuteStatementCommand = async (input, context) => {
    const headers = {
        "content-type": "application/x-amz-json-1.0",
        "x-amz-target": "DynamoDB_20120810.BatchExecuteStatement",
    };
    let body;
    body = JSON.stringify(serializeAws_json1_0BatchExecuteStatementInput(input, context));
    return buildHttpRpcRequest(context, headers, "/", undefined, body);
};
exports.serializeAws_json1_0BatchExecuteStatementCommand = serializeAws_json1_0BatchExecuteStatementCommand;

Switching to a helper saves about 40kb

const command = (target, fn, input, context) => {
  const headers = {
    "content-type": "application/x-amz-json-1.0",
    "x-amz-target": target,
  };
  let body;
  body = JSON.stringify(fn(input, context));
  return buildHttpRpcRequest(context, headers, "/", undefined, body);
}

// repeat for each command. Event this could be optimized for size, but I think it's caused by tsc, not aws codegen.
const serializeAws_json1_0BatchExecuteStatementCommand = command.bind(null, "DynamoDB_20120810.BatchExecuteStatement", serializeAws_json1_0BatchExecuteStatementInput)
exports.serializeAws_json1_0BatchExecuteStatementCommand = serializeAws_json1_0BatchExecuteStatementCommand;

These are the simplest examples I could find but a lot of the generated code has repetitive structure that could be abstracted into helpers.

Use Case

Bundling aws sdk and smity clients for the web or lambda.

Proposed Solution

Optimize the code gen

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

3.180.0

Environment details (OS name and version, etc.)

node

Footnotes

1. My examples are from dist-cjs which was compiled targeting ES2018, rather than dist-es which targets es5, but the repetition is present in both, and the source typescript. ↩

[kuhe]

This is a favorite topic of mine. I have worked on improvements in this area such as smithy-lang/smithy-typescript#576, which I plan to also apply to JSON protocols like your example.

But they are not a high priority compared to functional features and fixes, and I don't know when I can continue this.

PRs are welcome. For this one the generator is located at https://github.com/awslabs/smithy-typescript/blob/main/smithy-typescript-codegen/src/main/java/software/amazon/smithy/typescript/codegen/integration/HttpRpcProtocolGenerator.java#L213 as generateOperationSerializer.

[kuhe]

#4544 contributed towards this

JSON serde optimization: #4625

[kuhe]

recent (2023 Q4 / 2024 Q1) PRs reducing generated code size:

smithy-lang/smithy-typescript#1107
#5566
#5567
smithy-lang/smithy-typescript#1108
smithy-lang/smithy-typescript#1114
#5590
smithy-lang/smithy-typescript#1115
smithy-lang/smithy-typescript#1118
smithy-lang/smithy-typescript#1120
#5604

[kuhe]

closing this because it is being addressed in a continuous manner

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

[kuhe]

Don't ask me why I made this list, but just as a record, here are some PRs related to reducing generated code size:

• feat(codegen): general data mapping function #3830 generalized data mapping function ( codegen )
• chore(clients): reduce object copying in iterators #4198 reduce object copy operations ( runtime perf.)
• chore(scripts): add compression routines for endpoints rulesets #4300 created compression algo for Endpoints2.0 Rulesets ( runtime perf.)
• fix(clients): remove aggregated client from paginators #4496 remove the need for aggregated client in paginators (runtime perf.)
• Skip code generation of data filter functions for shapes that do not contain sensitive fields (recursive) smithy-lang/smithy-typescript#722 remove unnecessary log filters ( codegen )
• shorten internal serde fn names smithy-lang/smithy-typescript#730 shorten serde function names ( codegen )
• feat(codegen): add serde helper function #4616 general data extraction function ( codegen )
• use serde helper function & serde shortcuts smithy-lang/smithy-typescript#735 serde elision for JSON ( codegen )
• use aggregated client runtime generator smithy-lang/smithy-typescript#736 generate aggregated clients using a factory ( codegen )
• feat: generate unified error dispatcher smithy-lang/smithy-typescript#1150 generate unified error dispatcher ( codegen )
• feat: xml serde reduction smithy-lang/smithy-typescript#1108 generate re-usable string constants ( codegen )
• feat(xml-builder): additional serde helper methods #5567 XML builder codegen
• feat(credential-providers): lazy load STS & SSO clients in credential providers #5681 lazy load optional credential resolvers (runtime perf.)
• chore(codegen): use lib serde helpers instead of codegen #5855 reduce generation of duplicate functions ( codegen )
• feat: add requestBuilder smithy-lang/smithy-typescript#1107 generate HttpRequest objects in builder pattern ( codegen )
• feat: paginator factory applied to codegen smithy-lang/smithy-typescript#1114 generate paginators with a factory ( codegen )
• feat: Command classBuilder smithy-lang/smithy-typescript#1118 generate commands with a factory ( codegen )
• feat: use inline cjs build smithy-lang/smithy-typescript#1146 partial bundling for CJS artifacts (runtime perf.)