Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-38516: [Go][Parquet] Increment the number of rows written when appending a new row group #38517

Merged
merged 1 commit into from
Nov 13, 2023

Conversation

tschaub
Copy link
Contributor

@tschaub tschaub commented Oct 30, 2023

Rationale for this change

This makes it so the NumRows method on the file.Writer reports the total number of rows written across multiple row groups.

Are these changes tested?

A regression test is added that asserts that the total number of rows written matches expectations.

@github-actions
Copy link

⚠️ GitHub issue #38516 has been automatically assigned in GitHub to PR creator.

@mapleFU
Copy link
Member

mapleFU commented Nov 4, 2023

cc @zeroshade

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Nov 4, 2023
@zeroshade zeroshade merged commit 1ff43ab into apache:main Nov 13, 2023
24 of 25 checks passed
@zeroshade zeroshade removed the awaiting committer review Awaiting committer review label Nov 13, 2023
@github-actions github-actions bot added the awaiting merge Awaiting merge label Nov 13, 2023
Copy link

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit 1ff43ab.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

loicalleyne pushed a commit to loicalleyne/arrow that referenced this pull request Nov 13, 2023
…en appending a new row group (apache#38517)

### Rationale for this change

This makes it so the `NumRows` method on the `file.Writer` reports the total number of rows written across multiple row groups.

### Are these changes tested?

A regression test is added that asserts that the total number of rows written matches expectations.

* Closes: apache#38516

Authored-by: Tim Schaub <[email protected]>
Signed-off-by: Matt Topol <[email protected]>
@tschaub tschaub deleted the num-rows-issue branch November 13, 2023 22:55
dgreiss pushed a commit to dgreiss/arrow that referenced this pull request Feb 19, 2024
…en appending a new row group (apache#38517)

### Rationale for this change

This makes it so the `NumRows` method on the `file.Writer` reports the total number of rows written across multiple row groups.

### Are these changes tested?

A regression test is added that asserts that the total number of rows written matches expectations.

* Closes: apache#38516

Authored-by: Tim Schaub <[email protected]>
Signed-off-by: Matt Topol <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Go][Parquet] File writer only tracks the number of rows written in the last row group
3 participants