Skip to content
This repository has been archived by the owner on Mar 14, 2023. It is now read-only.

AWS S3 Slow Upload Speeds #1

Closed
lriverawong opened this issue Dec 5, 2022 · 5 comments
Closed

AWS S3 Slow Upload Speeds #1

lriverawong opened this issue Dec 5, 2022 · 5 comments

Comments

@lriverawong
Copy link

Currently on M1 instances we are facing an issue in which store_artifacts and persist_to_workspace steps are slow to complete. The reason for this is because upload speeds to AWS S3 is slower for this specific resource class. The team is currently investigating avenues to fix this.
Thank you for your patience.

@tokenfemale
Copy link

We have continued to look into this and determined that this is problem exists across regions and that upload speeds to locations other than AWS S3 are fast. We confirmed that download speeds from AWS S3 are fast so workspace restore steps should be working as expected.
We are continuing to investigate this and work with AWS to find a solution.

@alexazl
Copy link
Contributor

alexazl commented Dec 8, 2022

We are working with AWS to resolve this issue, but you can track its progress in the aws-cli public repo. We will continue to provide updates in this thread, as we get them.

@xbrett
Copy link

xbrett commented Jan 31, 2023

Are there any updates here? I am opening an escalation case with AWS as well. Additionally, could you provide additional details as to the specific AWS calls that the persist_to_workspace is using as well as the configuration details for the clients installed on the machines? @alexazl

@alexazl
Copy link
Contributor

alexazl commented Feb 2, 2023

We have implemented a temporary workaround and customers should no longer experience network performance issues. Until AWS deploys a fix for the bug, we will be using S3 Transfer Acceleration. Once the bug is fixed, we will revert to using normal S3 data transfer. If you notice any further poor performance, please let us know. Thanks for your patience! cc @xbrett

@tokenfemale
Copy link

We have implemented a temporary workaround and customers should no longer experience network performance issues. Until AWS deploys a fix for the bug, we will be using S3 Transfer Acceleration. Once the bug is fixed, we will revert to using normal S3 data transfer. If you notice any further poor performance, please let us know. Thanks for your patience!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants