Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: rename FileStream.file_reader to file_opener & update doc #8883

Merged
merged 1 commit into from
Jan 17, 2024

Conversation

SteveLauC
Copy link
Contributor

Which issue does this PR close?

Can be seen as a fix to #2990.

Rationale for this change

The FileStream.file_reader field is a generic type that implements the FileOpener trait, so it is kinda confusing to call it file_reader rather than file_opener.

#2291 tried to do some naming refactoring to this file, but it seems that @tustvold forgot to update this field name I guess?

And that doc comment was added in #1138, which is quite old and outdated.

What changes are included in this PR?

  1. rename FileStream.file_reader to FileStream.file_opener.
  2. Update the doc for this field.

Are these changes tested?

No

Are there any user-facing changes?

This field is private so I guess no.

@github-actions github-actions bot added the core Core DataFusion crate label Jan 16, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution @SteveLauC - looks like a nice improvement to me.

@@ -269,7 +267,7 @@ impl<F: FileOpener> FileStream<F> {
file_iter: files.into(),
projected_schema,
remain: config.limit,
file_reader,
file_opener: file_reader,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps you can change the name of the parameter to this function to file_opener to match the field name too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed this one:D

/// is not capable of limiting the number of records in the last batch, the file
/// stream will take care of truncating it.
file_reader: F,
/// A generic file opener, calling `open()` on it will return a `FileOpenFuture`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// A generic file opener, calling `open()` on it will return a `FileOpenFuture`,
/// A generic [`FileOpener`]/ Calling `open()` returns a [`FileOpenFuture`],

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Though I changed that / to a .

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @SteveLauC

@alamb alamb merged commit 81d9d88 into apache:main Jan 17, 2024
22 checks passed
@SteveLauC SteveLauC deleted the file_opener branch January 17, 2024 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core DataFusion crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants