Skip to content

Commit

Permalink
chore: timeouts on PR builds by querying against npm (aws#20409)
Browse files Browse the repository at this point in the history
As part of AWS CDK builds, we run a script that checks that the API does not introduce any breaking changes with respect to the latest published version of AWS CDK. To do so, this currently queries against the GitHub API to check what the latest releases were. However, this can fail because without using specific GitHub credentials, we are limited to 60 API calls per minute. Even [with retries](aws#19336), this can still timeout, in which case the build fails suddenly without an error message:

```
<thousands of lines of logs omitted>
lerna success - @aws-cdk/prlint
lerna success - @aws-cdk/ubergen
lerna success - @aws-cdk/yarn-cling

real	21m40.119s
user	353m7.607s
sys	27m49.086s
Listing jsii packages...
lerna notice cli v4.0.0
lerna info ci enabled
lerna success found 254 packages
Filtering on existing packages on NPM...
Determining baseline version... 
Build version:    1.156.1.
```

To avoid this failure mode entire, this PR updates the script to check for the latest AWS CDK version by querying against npm. Both npm and GitHub should be accurate sources of truth AFAIK, and npm does not impose as stringent rate limits on its callers.

----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [ ] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/master/INTEGRATION_TESTS.md)?
	* [ ] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
Chriscbr authored and wphilipw committed May 23, 2022
1 parent 8168cae commit b22bd2c
Showing 1 changed file with 3 additions and 31 deletions.
34 changes: 3 additions & 31 deletions scripts/find-latest-release.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,33 +16,9 @@ async function main(matchRange) {
throw new Error(`Not a valid range: ${matchRange}`);
}

let releases;
for (let attempt = 0; attempt < 10; attempt++) {
const { stdout, error } = cp.spawnSync('curl', ['https://api.github.com/repos/aws/aws-cdk/releases?per_page=100'], { maxBuffer: 10_000_000 });
if (error) { throw error; }

const response = JSON.parse(stdout);
if (response.message) {
// This is actually an error response. Only recover from throttling errors.
if (!response.message.includes('API rate limit')) {
throw new Error(response.message);
}

// 60 requests/hour, so we need to sleep for a full minute to get any kind of response
const sleepTime = Math.floor(Math.random() * 60 * Math.pow(2, attempt));
await sleep(sleepTime * 1000);
continue;
}

releases = response;
break;
}
if (!releases) {
throw new Error('Retries exhaused');
}


const versions = releases.map(r => r.name.replace(/^v/, '')); // v1.2.3 -> 1.2.3
const { stdout, error } = cp.spawnSync('npm', ['view', 'aws-cdk', 'versions', '--json'], { maxBuffer: 10_000_000 });
if (error) { throw error; }
const versions = JSON.parse(stdout.toString('utf-8'));

const sat = semver.maxSatisfying(versions, range);
if (!sat) {
Expand All @@ -51,10 +27,6 @@ async function main(matchRange) {
console.log(sat);
}

function sleep(ms) {
return new Promise(ok => setTimeout(ok, ms));
}

main(process.argv[2]).catch(e => {
console.error(e);
process.exitCode = 1;
Expand Down

0 comments on commit b22bd2c

Please sign in to comment.