Skip to content

Commit

Permalink
Enable reading explicit row number column from Prism connector (faceb…
Browse files Browse the repository at this point in the history
…ookincubator#11072)

Summary:
Pull Request resolved: facebookincubator#11072

Also fix the selective Nimble reader:
1. Add the missing implementation for
`FlatMapAsStructColumnReader::estimateMaterializedSize`
2. Fix the performance bug that we refetch the `StripeGroup` for each flatmap node in each stripe

Reviewed By: pedroerp

Differential Revision: D63151114
  • Loading branch information
Yuhta authored and facebook-github-bot committed Sep 23, 2024
1 parent 4e45bc5 commit f8890c5
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
2 changes: 1 addition & 1 deletion velox/dwio/common/ColumnSelector.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -344,7 +344,7 @@ std::shared_ptr<ColumnSelector> ColumnSelector::fromScanSpec(
const RowTypePtr& rowType) {
std::vector<std::string> columnNames;
for (auto& child : spec.children()) {
if (child->isConstant()) {
if (child->isConstant() || child->isExplicitRowNumber()) {
continue;
}
std::string name = child->fieldName();
Expand Down
4 changes: 4 additions & 0 deletions velox/exec/tests/TableScanTest.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -5090,6 +5090,10 @@ TEST_F(TableScanTest, rowNumberInRemainingFilter) {
auto file = TempFilePath::create();
writeToFile(file->getPath(), {vector});
auto outputType = ROW({"c1"}, {BIGINT()});
// FIXME: We should not need r1 in table schema for production code; this is
// just to infer type of r1 in remaining filter expression parser. We
// probably need to find a way to specify the type of r1 without adding it to
// table schema to test this case. For now it's essentially untested.
auto schema = ROW({"c1", "r1"}, {BIGINT(), BIGINT()});
auto plan = PlanBuilder()
.startTableScan()
Expand Down

0 comments on commit f8890c5

Please sign in to comment.