Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: Add example usage snippets for the bucket client flavors #426

Merged
merged 1 commit into from
Feb 4, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 41 additions & 2 deletions dagshub/repo_bucket.py
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,56 @@ def get_repo_bucket_client(repo: str, flavor: FlavorTypes = "boto"):
Creates an S3 client for the specified repository's DagsHub storage bucket

Available flavors are:
``"boto"``: Returns a `boto3.client \
``"boto"`` (Default): Returns a `boto3.client \
<https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-examples.html>`_.\
with predefined EndpointURL and credentials.
The name of the bucket is the name of the repository,
and you will need to specify it for any request you make
and you will need to specify it for any request you make.

Example usage::

boto_client = get_repo_bucket_client("user/my-repo")

# Upload file
boto_client.upload_file(
Bucket="my-repo", # name of the repo
Filename="local.csv", # local path of file to upload
Key="remote.csv", # remote path where to upload the file
)
# Download file
boto_client.download_file(
Bucket="my-repo", # name of the repo
Key="remote.csv", # remote path from where to download the file
Filename="local.csv", # local path where to download the file
)


``"s3fs"``: Returns a \
`s3fs.S3FileSystem <https://s3fs.readthedocs.io/en/latest/index.html#examples>`_\
with predefined EndpointURL and credentials.
The name of the bucket is the name of the repository,
and you will need to specify it for any request you make

Example usage::

s3fs_client = get_repo_bucket_client("user/my-repo", flavor="s3fs")

# Read from file
with s3fs_client.open("my-repo/remote.csv", "rb") as f:
print(f.read())

# Write to file
with s3fs_client.open("my-repo/remote.csv", "wb") as f:
f.write(b"Content")

# Upload file (can also upload directories)
s3fs_client.put(
"local.csv", # local path of file/dir to upload
"my-repo/remote.csv" # remote path where to upload the file
)



Args:
repo: Name of the repo in the format of ``username/reponame``
flavor: one of the possible s3 client flavor variants
Expand Down
Loading