Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use tempfile for parquet tests #1165

Merged
merged 3 commits into from
Jan 16, 2022
Merged

Conversation

tustvold
Copy link
Contributor

@tustvold tustvold commented Jan 12, 2022

Which issue does this PR close?

Closes #1164

Rationale for this change

This avoids running the parquet tests from "leaking" random parquet files in your local filesystem. It also eliminates a reasonable amount of plumbing, and potential source of test concurrency bugs, with filenames being plumbed through the tests.

What changes are included in this PR?

This changes to using tempfile for temporary files, which is also used by arrow

Are there any user-facing changes?

No, the util module is marked experimental

@github-actions github-actions bot added the parquet Changes to the parquet crate label Jan 12, 2022
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tustvold -- I took the liberty of merging from master and resolving a conflict as well, FYI

@codecov-commenter
Copy link

Codecov Report

Merging #1165 (cbe7815) into master (66b84f3) will decrease coverage by 0.01%.
The diff coverage is 98.96%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1165      +/-   ##
==========================================
- Coverage   82.66%   82.64%   -0.02%     
==========================================
  Files         173      173              
  Lines       50902    50865      -37     
==========================================
- Hits        42077    42039      -38     
- Misses       8825     8826       +1     
Impacted Files Coverage Δ
parquet/src/util/test_common/file_util.rs 75.00% <ø> (-18.11%) ⬇️
parquet/src/arrow/arrow_writer.rs 97.87% <98.52%> (-0.21%) ⬇️
parquet/src/arrow/arrow_reader.rs 91.61% <100.00%> (ø)
parquet/src/arrow/schema.rs 85.56% <100.00%> (ø)
parquet/src/column/writer.rs 92.50% <100.00%> (ø)
parquet/src/file/footer.rs 95.32% <100.00%> (ø)
parquet/src/file/writer.rs 93.30% <100.00%> (ø)
parquet/src/util/io.rs 90.83% <100.00%> (+0.07%) ⬆️
arrow/src/datatypes/datatype.rs 66.38% <0.00%> (-0.43%) ⬇️
arrow/src/array/transform/mod.rs 85.43% <0.00%> (-0.14%) ⬇️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66b84f3...cbe7815. Read the comment docs.

@alamb alamb merged commit e45d118 into apache:master Jan 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Parquet Tests Cleanup Temporary Files
4 participants