Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-17642: [C++] Add ordered aggregation #14352

Closed
wants to merge 14 commits into from
Closed

ARROW-17642: [C++] Add ordered aggregation #14352

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
8abc366
ARROW-17642: [C++] Add ordered aggregation
rtpsw Oct 8, 2022
9af5d1a
aggregate-node support
rtpsw Oct 10, 2022
fd3b3e8
CI fixes
rtpsw Oct 15, 2022
ec4e1ab
remove unused function
rtpsw Oct 15, 2022
0aaed62
add tests
rtpsw Oct 16, 2022
6901b1a
expose streaming GroupBy
rtpsw Oct 17, 2022
ace4ef9
requested fixes
rtpsw Nov 14, 2022
43949ad
requested fixes
rtpsw Nov 20, 2022
c90d2bf
Merge branch 'master' into ARROW-17642
rtpsw Nov 27, 2022
8f280ee
grouping state clean up
rtpsw Nov 28, 2022
60c70ca
Merge branch 'master' into ARROW-17642
rtpsw Dec 24, 2022
c9abbd6
fix merge
rtpsw Dec 24, 2022
26f991c
requested fixes
rtpsw Dec 24, 2022
9940fa2
segmented group-by using plan
rtpsw Jan 1, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions cpp/src/arrow/compute/exec.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,12 +140,11 @@ ExecBatch ExecBatch::Slice(int64_t offset, int64_t length) const {
return out;
}

Result<ExecBatch> ExecBatch::Make(std::vector<Datum> values) {
if (values.empty()) {
Result<ExecBatch> ExecBatch::Make(std::vector<Datum> values, int64_t length) {
if (values.empty() && length < 0) {
return Status::Invalid("Cannot infer ExecBatch length without at least one value");
}

int64_t length = -1;
for (const auto& value : values) {
if (value.is_scalar()) {
continue;
Expand Down
13 changes: 12 additions & 1 deletion cpp/src/arrow/compute/exec.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ struct ARROW_EXPORT ExecBatch {

explicit ExecBatch(const RecordBatch& batch);

static Result<ExecBatch> Make(std::vector<Datum> values);
static Result<ExecBatch> Make(std::vector<Datum> values, int64_t length = -1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naively one looking at this might be confused why you now need both ExecBatch::Make and ExecBatch::ExecBatch since both take a vector of values and a length.

Looking closer it seems the Make function does the extra work of verifying that the length of the datums match the given length.

Could you add some comments explaining this for future readers?


Result<std::shared_ptr<RecordBatch>> ToRecordBatch(
std::shared_ptr<Schema> schema, MemoryPool* pool = default_memory_pool()) const;
Expand Down Expand Up @@ -233,6 +233,17 @@ struct ARROW_EXPORT ExecBatch {

ExecBatch Slice(int64_t offset, int64_t length) const;

Result<ExecBatch> SelectValues(const std::vector<int>& ids) const {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know there aren't many comment blocks in this file but ExecBatch is less experimental than it used to be. Can you comment this method?

Also, is there any particular reason to have the implementation in the header file?

std::vector<Datum> selected_values(ids.size());
for (size_t i = 0; i < ids.size(); i++) {
if (ids[i] < 0 || static_cast<size_t>(ids[i]) >= values.size()) {
return Status::Invalid("ExecBatch invalid value selection: ", ids[i]);
}
selected_values[i] = values[ids[i]];
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
std::vector<Datum> selected_values(ids.size());
for (size_t i = 0; i < ids.size(); i++) {
if (ids[i] < 0 || static_cast<size_t>(ids[i]) >= values.size()) {
return Status::Invalid("ExecBatch invalid value selection: ", ids[i]);
}
selected_values[i] = values[ids[i]];
}
std::vector<Datum> selected_values;
selected_values.reserve(ids.size());
for (int idx : ids) {
if (idx < 0 || idx >= static_cast<int>(values.size())) {
return Status::Invalid("ExecBatch invalid value selection: ", idx);
}
selected_values.push_back(values[idx]);
}

Minor nit: this seems slightly more readable with a foreach loop.

return ExecBatch(std::move(selected_values), length);
}

/// \brief A convenience for returning the types from the batch.
std::vector<TypeHolder> GetTypes() const {
std::vector<TypeHolder> result;
Expand Down
Loading