Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added optional flag for building Arrow with S3 filesystem support #8531

Merged
merged 5 commits into from
Jun 16, 2021

Conversation

jdye64
Copy link
Contributor

@jdye64 jdye64 commented Jun 16, 2021

Added the cmake build flag CUDF_ENABLE_ARROW_S3 to conditionally build Arrow with S3 filesystem support. By default support for S3 filesystem will be enabled. If S3 support needs to be disabled you can pass -DCUDF_ENABLE_ARROW_S3=OFF

@jdye64 jdye64 requested review from a team as code owners June 16, 2021 12:52
@github-actions github-actions bot added CMake CMake build issue Java Affects Java cuDF API. libcudf Affects libcudf (C++/CUDA) code. labels Jun 16, 2021
Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Java approval

@jlowe jlowe added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Jun 16, 2021
@robertmaynard robertmaynard self-requested a review June 16, 2021 13:28
Copy link
Contributor

@robertmaynard robertmaynard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when S3 is disabled and arrow_io_source is called with an S3 backed URI? Should we have a test that we get the expected exception and don't crash?

@jdye64
Copy link
Contributor Author

jdye64 commented Jun 16, 2021

What happens when S3 is disabled and arrow_io_source is called with an S3 backed URI? Should we have a test that we get the expected exception and don't crash?

In that case https://github.com/rapidsai/cudf/blob/branch-21.08/cpp/include/cudf/io/datasource.hpp#L321 would trigger. Always good to add a test however.

@codecov
Copy link

codecov bot commented Jun 16, 2021

Codecov Report

Merging #8531 (262d463) into branch-21.08 (93ce6c7) will not change coverage.
The diff coverage is n/a.

❗ Current head 262d463 differs from pull request most recent head 20b2a4f. Consider uploading reports for the commit 20b2a4f to get more accurate results
Impacted file tree graph

@@              Coverage Diff              @@
##           branch-21.08    #8531   +/-   ##
=============================================
  Coverage         82.57%   82.57%           
=============================================
  Files               109      109           
  Lines             17870    17870           
=============================================
  Hits              14757    14757           
  Misses             3113     3113           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93ce6c7...20b2a4f. Read the comment docs.

@trxcllnt
Copy link
Contributor

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 0196493 into rapidsai:branch-21.08 Jun 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CMake CMake build issue improvement Improvement / enhancement to an existing function Java Affects Java cuDF API. libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants