chore(scripts): add compression routines for endpoints rulesets

[kuhe]

Issue

#4299

Description

Adds a script that compresses each service's ruleset object with one of two algorithms:

• attempts to use the pattern detection recursive algorithm
• if that fails, it uses the whitespace removal algorithm

This PR does not integrate this script into the client generation process.

main branch

du -sh clients/client-s3/src/endpoint/ruleset.ts
804K	clients/client-s3/src/endpoint/ruleset.ts

npm notice name:          @aws-sdk/client-s3
npm notice version:       3.234.0
npm notice filename:      @aws-sdk/client-s3-3.234.0.tgz
npm notice package size:  355.8 kB
npm notice unpacked size: 6.2 MB (yarn build && npm pack without downlevel types)

PR branch

du -sh clients/client-s3/src/endpoint/ruleset.ts
28K	clients/client-s3/src/endpoint/ruleset.ts

npm notice name:          @aws-sdk/client-s3
npm notice version:       3.234.0
npm notice filename:      @aws-sdk/client-s3-3.234.0.tgz
npm notice package size:  320.5 kB
npm notice unpacked size: 3.8 MB (yarn build && npm pack without downlevel types)

Testing

each algorithm runs self-verification using the input it is given before it is accepted. Because the whitespace removal algorithm is more or less immune to failure, it is the fallback.

Secondly, to show the two formats have no performance impact, the following performance test loads

• A) the compressed JS format with patterns represented by variable references
• B) the standard uncompressed JSON format

and then walks the entire object and inspects each value.

const marks = {
  js: [],
  json: [],
};

performanceTest();

function performanceTest() {
  {
    for (let i = 0; i < 10000; ++i) {
      const start = performance.now();
      loadJs();
      const end = performance.now();
      marks.js.push(end - start);
    }
  }

  {
    for (let i = 0; i < 10000; ++i) {
      const start = performance.now();
      loadJson();
      const end = performance.now();
      marks.json.push(end - start);
    }
  }

  console.log("js", marks.js.reduce((a, b) => a + b, 0) / marks.js.length);
  console.log("json", marks.json.reduce((a, b) => a + b, 0) / marks.json.length);
}

function loadJs() {
  const data = require("./s3");
  walkObject(data);
}

function loadJson() {
  const data = require("./s3.json");
  walkObject(data);
}

function walkObject(data) {
  switch (typeof data) {
    case "object":
      if (Array.isArray(data)) {
        data.forEach(walkObject);
        break;
      }
      Object.values(data).forEach(walkObject);
      break;
    default:
      if (data === -13534135) {
        throw new Error("comparison failed");
      }
  }
}

output:

js 0.5221970861673355
json 0.5317003727197647

[kuhe]

For codegen changes, see #4301

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.