Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve asset export retrying #5563

Merged
merged 4 commits into from
Jan 26, 2024
Merged

Improve asset export retrying #5563

merged 4 commits into from
Jan 26, 2024

Conversation

sgulseth
Copy link
Member

@sgulseth sgulseth commented Jan 25, 2024

Description

We are experiencing that our export assets endpoints hangs up the connections, and for some reason the CLI is trying to reuse the dead connection. This PR adds more retry mechanisms to download assets.

What to review

Testing

$ npm run build
$ cd dev/test-studio
$ DEBUG=sanity* ../../packages/@sanity/cli/bin/sanity dataset export test --overwrite

Notes for release

Copy link

vercel bot commented Jan 25, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
performance-studio ✅ Ready (Inspect) Visit Preview Jan 26, 2024 0:11am
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 26, 2024 0:11am
1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Jan 26, 2024 0:11am

@sgulseth sgulseth changed the title fix: hoist retry mechanism when downloading assets Improve asset export retrying Jan 25, 2024
Copy link
Contributor

github-actions bot commented Jan 25, 2024

Package Documentation Change
sanity -3%
Full Report
@sanity/diff
This branch Next branch
13 documented 13 documented
16 not documented 16 not documented
@sanity/block-tools
This branch Next branch
4 documented 4 documented
9 not documented 9 not documented
@sanity/types
This branch Next branch
55 documented 55 documented
239 not documented 239 not documented
sanity/desk
This branch Next branch
84 documented 84 documented
64 not documented 64 not documented
@sanity/portable-text-editor
This branch Next branch
21 documented 21 documented
44 not documented 44 not documented
@sanity/mutator
This branch Next branch
7 documented 7 documented
4 not documented 4 not documented
@sanity/cli
This branch Next branch
1 documented 1 documented
31 not documented 31 not documented
@sanity/util/legacyDateFormat
This branch Next branch
0 documented 0 documented
5 not documented 5 not documented
@sanity/schema/_internal
This branch Next branch
0 documented 0 documented
12 not documented 12 not documented
@sanity/util/paths
This branch Next branch
1 documented 1 documented
15 not documented 15 not documented
sanity/router
This branch Next branch
17 documented 17 documented
26 not documented 26 not documented
@sanity/schema
This branch Next branch
0 documented 0 documented
2 not documented 2 not documented
sanity/structure
This branch Next branch
2 documented 2 documented
6 not documented 6 not documented
sanity/cli
This branch Next branch
2 documented 2 documented
0 not documented 0 not documented
@sanity/vision
This branch Next branch
0 documented 0 documented
3 not documented 3 not documented
@sanity/util/fs
This branch Next branch
0 documented 0 documented
3 not documented 3 not documented
sanity/_internal
This branch Next branch
0 documented 0 documented
1 not documented 1 not documented
sanity/_internalBrowser
This branch Next branch
0 documented 0 documented
3 not documented 3 not documented
@sanity/util/content
This branch Next branch
1 documented 1 documented
5 not documented 5 not documented
sanity
This branch Next branch
177 documented 183 documented
843 not documented 846 not documented

Copy link
Contributor

github-actions bot commented Jan 25, 2024

Component Testing Report Updated Jan 26, 2024 12:14 PM (UTC)

File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 34s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 6s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 12s 3 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 13s 4 2 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 12s 6 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 32s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 18s 9 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 1s 18 0 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 13s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 6s 3 0 0

Copy link
Contributor

@j33ty j33ty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I have tried this change locally, this seem to be working as expected, as seen in debug mode. This approach will make CLI export resilient against flaky server-side and network errors.

@sgulseth sgulseth marked this pull request as ready for review January 26, 2024 14:10
@sgulseth sgulseth requested a review from a team as a code owner January 26, 2024 14:10
@sgulseth sgulseth requested review from binoy14 and removed request for a team January 26, 2024 14:10
Copy link
Contributor

@binoy14 binoy14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@sgulseth sgulseth added this pull request to the merge queue Jan 26, 2024
Merged via the queue into next with commit 57fd3e0 Jan 26, 2024
40 checks passed
@sgulseth sgulseth deleted the fix/retry-download-asset branch January 26, 2024 14:25
ricokahler pushed a commit that referenced this pull request Jan 26, 2024
* fix: hoist retry mechanism when downloading assets

* feat: improve error handling in downloadAsset

* feat: retry more failure cases in downloadAsset

* feat: rety asset downloading 10 times
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants