When crates.io gives 429, cargo should back off and retry later #13530
Labels
A-interacts-with-crates.io
Area: interaction with registries
A-networking
Area: networking issues, curl, etc.
A-registries
Area: registries
C-bug
Category: bug
Command-publish
S-needs-team-input
Status: Needs input from team on whether/how to proceed.
Problem
Our workspace contains 46 cargo packages. (Because cargo insists that each crate must be a separate package, and we want to split up crates for code sanity and compilation time reasons.)
This means that in our recent release, our on-duty release technician hit the rate limit. This aborted publication of the workspace, requiring manual retries and wrangling.
Steps
Have a workspace with more than 30 (the current burst rate limit) crates. Try to publish it by publising each crate, in topo order, with
cargo publish
(using some automated tool).Possible Solution(s)
cargo should handle a 429 response by backing off and retrying, using an exponential backoff algorithm.
In rust-lang/crates.io#1643 the crates.io team report already having raised the rate limit. In the error message from crates.io they suggest emailing
help@
to ask for a rate limit increase. Such a workflow is IMO undesirable, especially as Rust gets more adoption.Notes
I don't think increasing the rate limit (globally, or on request) is the right fix. If 429 is a hard error there is a tension between preventing misuse, and not breaking large projects' releases. But this tension can be abolished by handling 429 gracefully.
#13397 would probably have assisted the recovery from this situation (and also the local disk space problem our releasae technician also ran into).
See also: rust-lang/crates.io#3229 (requesting docs) #6714 (requesting better error message display).
Version
(edited to fix ticket links)
The text was updated successfully, but these errors were encountered: