Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update datasets download URL #3299

Merged
merged 1 commit into from
Feb 27, 2023
Merged

Update datasets download URL #3299

merged 1 commit into from
Feb 27, 2023

Conversation

jjacobelli
Copy link
Contributor

Update datasets download URL to reduce latency and costs

@jjacobelli
Update datasets download URL
f1b8627 
Signed-off-by: Jordan Jacobelli <[email protected]>
@jjacobelli jjacobelli added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Feb 24, 2023
@jjacobelli jjacobelli requested a review from a team as a code owner February 24, 2023 09:55
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@ajschmidt8
Copy link
Member

To provide additional context...

data.rapids.ai serves the contents of the rapidsai-data S3 bucket via an AWS CloudFront distribution.

The benefits of using a CloudFront distribution are:

  • dataset files are served from an endpoint closest to individual end-users. Therefore latency is decreased, which makes datasets download faster
  • data transfer costs out of AWS are cheaper

Therefore, it's in everyone's best interest to start using the new data.rapids.ai URLs for downloading datasets.

At some point in the future, the S3 URLs will be disabled and datasets will only be retrievable from data.rapids.ai.

@ajschmidt8
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit e1de2f5 into rapidsai:branch-23.04 Feb 27, 2023
@jjacobelli jjacobelli mentioned this pull request Feb 28, 2023
Merged
rapids-bot bot pushed a commit that referenced this pull request Mar 16, 2023
@jjacobelli
Update datasets download URL (#3305)
c4dc5d7 
Some URLs were not updated by #3299. This PR is fixing this and removing the `datasets/get_test_data.sh` from the ignored files

Authors:
  - Jordan Jacobelli (https://github.com/jjacobelli)
  - AJ Schmidt (https://github.com/ajschmidt8)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #3305
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ChuckHastings ChuckHastings ChuckHastings approved these changes

@ajschmidt8 ajschmidt8 ajschmidt8 approved these changes

Assignees
No one assigned
Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jjacobelli @ajschmidt8 @ChuckHastings