[C++][Parquet] Dataset: ParquetFileFragment::EvaluateStatisticsAsExpression
should better checks Statistics::HasNullCount
#43712
Milestone
Describe the enhancement requested
ParquetFileFragment::EvaluateStatisticsAsExpression
filters parquet file with parquet statistics, the function is listed below:arrow/cpp/src/arrow/dataset/file_parquet.cc
Line 370 in 2e434da
statistics.null_count()
is used here, however, there're merely case when!statistics.HasNullCount()
. So this function should checkstatistics.HasNullCount()
before using that!statistics.HasNullCount()
is merely happens, since parquet-java and parquet-c++ always writes this even when null-count == 0. However, parquet-rs previously don't write it when count == 0 . And maybe some legacy file without this.So as a result, we need check
!statistics.HasNullCount()
hereComponent(s)
C++, Parquet
The text was updated successfully, but these errors were encountered: