Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Requester Pays #520

Merged
merged 11 commits into from
Sep 14, 2022
Merged

Support for Requester Pays #520

merged 11 commits into from
Sep 14, 2022

Conversation

HaileyJang
Copy link
Contributor

Skyplane CLI will now take in additional flag to support requester pays bucket. Requester-pays bucket are buckets that charges the person who downloads the objects. More information in the following link: https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html

Copy link
Contributor

@parasj parasj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, this will really help our users! I added some minor comments to improve maintainability with a single code path for both workflows.

skyplane/chunk.py Outdated Show resolved Hide resolved
skyplane/cli/cli.py Outdated Show resolved Hide resolved
skyplane/cli/cli_impl/cp_replicate.py Outdated Show resolved Hide resolved
skyplane/cli/cli_impl/cp_replicate.py Outdated Show resolved Hide resolved
skyplane/cli/cli_impl/cp_replicate.py Outdated Show resolved Hide resolved
skyplane/obj_store/s3_interface.py Outdated Show resolved Hide resolved
skyplane/obj_store/s3_interface.py Outdated Show resolved Hide resolved
Copy link
Contributor

@parasj parasj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Also update the error message here to note that the user may need to enable requester pays:

class MissingBucketException(SkyplaneException):
def pretty_print_str(self):
err = f"[red][bold]:x: MissingBucketException:[/bold] {str(self)}[/red]"
err += "\n[bold][red]Please ensure that the bucket exists and is accessible.[/red][/bold]"
return err

`Please ensure that the bucket exists and is accessible. See https://skyplane.org/en/latest/faq.html#i-am-unable-to-copy-files-to-a-folder-in-my-destination-bucket-due-to-a-missing-object-error.```
For that documentation page, you should add more text to elaborate on requester pays setup.

skyplane/obj_store/object_store_interface.py Outdated Show resolved Hide resolved
skyplane/obj_store/s3_interface.py Outdated Show resolved Hide resolved
skyplane/obj_store/s3_interface.py Outdated Show resolved Hide resolved
@HaileyJang
Copy link
Contributor Author

FAQ changes made in #538

@HaileyJang HaileyJang merged commit f149ef9 into main Sep 14, 2022
@HaileyJang HaileyJang deleted the requester-pays-support branch September 14, 2022 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants