Prototype ArrayView Types

[tustvold]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

There is ongoing discussion of introducing an ArrayView type to the format - https://lists.apache.org/thread/r28rw5n39jwtvn08oljl09d4q2c1ysvb

We should explore the design space around this, in particular to gather some empirical data as to the impact of introducing such a type.

Describe the solution you'd like

I would like to prototype an implementation of StringView and explore integrating it into the parquet reader, where it ostensibly could yield to some non-trivial performance improvements

Describe alternatives you've considered

Additional context

[tustvold]

The benchmarks in #4378 show half the execution time being spent rewriting data to remove empty array slices. This likely could be optimised, and it is unclear how realistic the benchmark is, but I thought it was an interesting data point.

Theoretically ArrayView types would remove the need for this, whilst also removing the memcpy when decoding byte arrays. I'd anticipate roughly a 2x return, with bigger returns for more heavily nested data

[tustvold]

My current feelings on this matter are summarized in https://lists.apache.org/thread/1j0hdbfd0q2636zs9z0x19fkcn87gjhf

TLDR I think improving the support for sparse dictionaries may be sufficient to support this use case

[alamb]

Filed #5374 to track implementing what was added to the spec

[alamb]

Here was a draft PR: #4585

[alamb]

From my perspective, the prototype was completed in #4585 and follow on work is tracked in #5374 so closing this ticket down