Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Port tests in partitioned_csv.rs to sqllogictest #8919

Merged
merged 1 commit into from
Jan 22, 2024

Conversation

simicd
Copy link
Contributor

@simicd simicd commented Jan 19, 2024

Which issue does this PR close?

Closes #8208

Rationale for this change

Migrate tests from Rust to sqllogictest

What changes are included in this PR?

  • Remove duplicated code from partitioned_csv.rs
  • Create new unit test with partitioned csv files in csv_files.slt

Are these changes tested?

Yes

Are there any user-facing changes?

No

@github-actions github-actions bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Jan 19, 2024
@@ -482,7 +482,7 @@ async fn sort_on_window_null_string() -> Result<()> {
async fn test_prepare_statement() -> Result<()> {
let tmp_dir = TempDir::new()?;
let partition_count = 4;
let ctx = partitioned_csv::create_ctx(&tmp_dir, partition_count).await?;
let ctx = create_ctx_with_partition(&tmp_dir, partition_count).await?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same code existed in partitioned_csv.rs and mod.rs - dropped the former and replaced with function calls to mod.rs

@@ -63,3 +63,76 @@ id6 value"6
id7 value"7
id8 value"8
id9 value"9


# Read partitioned csv
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partitioned_csv.rs did not have any unit tests so I created a new one to test reading from partitioned csv files.

use tempfile::TempDir;

/// Generate CSV partitions within the supplied directory
fn populate_csv_partitions(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @simicd -- this looks great

@alamb alamb merged commit e986e15 into apache:main Jan 22, 2024
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Port tests in partitioned_csv.rs to sqllogictest
2 participants