test: Port (last) repartition.rs query to sqllogictest
#8936
Conversation
github-actions
bot
added
core
There was a problem hiding this comment.
Thank you @simicd -- this looks reasonable to me but I am not quite sure about the equivalence of these tests
@metesynnada I reviewed #5278 and given the changes in DataFusion since then I am not quite sure how to map its findings to this test.
Can you double check and provide your feedback on if the test proposed in this PR is equivalent to the test in repartition.rs?
| ----CoalesceBatchesExec: target_batch_size=8192 | ||
| ------FilterExec: c3@2 > 0 | ||
| --------RepartitionExec: partitioning=RoundRobinBatch(3), input_partitions=1 | ||
| ----------StreamingTableExec: partition_sizes=1, projection=[c1, c2, c3], infinite_source=true |
| --CoalescePartitionsExec | ||
| ----CoalesceBatchesExec: target_batch_size=8192 | ||
| ------FilterExec: c3@2 > 0 | ||
| --------RepartitionExec: partitioning=RoundRobinBatch(3), input_partitions=1 |
There was a problem hiding this comment.
This partitioning is round robin partitioning where the original test uses hash partitioning. I am not sure if that is equivalent.
There was a problem hiding this comment.
I see! Thanks for looking into it
There was a problem hiding this comment.
I don't think there will be a difference. It's hard to reproduce the hash partitioning with unbounded tables. However, this would work well enough.
Co-authored-by: Andrew Lamb <[email protected]>
| --CoalescePartitionsExec | ||
| ----CoalesceBatchesExec: target_batch_size=8192 | ||
| ------FilterExec: c3@2 > 0 | ||
| --------RepartitionExec: partitioning=RoundRobinBatch(3), input_partitions=1 |
There was a problem hiding this comment.
I don't think there will be a difference. It's hard to reproduce the hash partitioning with unbounded tables. However, this would work well enough.
|
I took the liberty of merging this branch into |
|
Thanks again |
Which issue does this PR close?
Closes #8212
Rationale for this change
Move tests to sqllogictest
What changes are included in this PR?
Are these changes tested?
Yes
Are there any user-facing changes?
No