Feature/infer schema upstream #43
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Merged to upstream |
gokselk
pushed a commit
to gokselk/datafusion-upstream
that referenced
this pull request
Nov 7, 2024
…regation (apache#12946) * Improve unparsing for ORDER BY with Aggregation functions (synnada-ai#38) * Improve UNION unparsing (synnada-ai#39) * Scalar functions in ORDER BY unparsing support (synnada-ai#41) * Improve unparsing for complex Window functions with Aggregation (synnada-ai#42) * WindowFunction order_by should respect `supports_nulls_first_in_sort` dialect setting (synnada-ai#43) * Fix plan_to_sql * Improve
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
No related issue
Rationale for this change
I came across the following obstacles trying to write some custom schema inference code:
schema_infer_max_recoption.read_to_delimited_chunks_from_streamdoes not necessarily have to beBoxStream<'static>What changes are included in this PR?
schema_infer_max_recanOptionAre these changes tested?
Tested with existing code and tests.
Are there any user-facing changes?
CsvOptions::schema_infer_max_recandJsonOptions::schema_infer_max_recare nowOption<usize>s as opposed tousize(with_schema_infer_max_recsignature is unchanged)read_to_delimited_chunks,read_to_delimited_chunks_from_stream,convert_stream,convert_to_compress_streamnow take lifetime parametersI believe (1) is a breaking change but I'm not sure about (2)