RLE and dictionary filter only enabled for UTF8 since format version 17.

[KiterLuc]

This fixes an issue in the filter pipeline where we should only skip offsets unfiltering for RLE/dictionary filters for UTF8 strings starting at version 17.

---

TYPE: IMPROVEMENT
DESC: RLE and dictionary filter only enabled for UTF8 since format version 17.

[ypatia]

Thanks for the fix @KiterLuc !
Added a backport label for 2.14 but I wanted to check with @ihnorton : are we planning some maintenance release for 2.13 too? The reader_base.cc part of this bugfix is a zero day bug for RLE/dict when we have nullable string attributes, so it would need to be backported to every maintenance release we are planning to issue.

[Shelnutt2]

Thanks for the fix @KiterLuc ! Added a backport label for 2.14 but I wanted to check with @ihnorton : are we planning some maintenance release for 2.13 too? The reader_base.cc part of this bugfix is a zero day bug for RLE/dict when we have nullable string attributes, so it would need to be backported to every maintenance release we are planning to issue.

We only enabled this for 2.14 right? Is 2.13 effected?

A further question I have is, we didn't allow creation of utf-8 attributes with RLE or Dictionary before format 17, so while the safety check is needed, its not possible for someone to have an array where this would cause a problem?

[ypatia]

Thanks for the fix @KiterLuc ! Added a backport label for 2.14 but I wanted to check with @ihnorton : are we planning some maintenance release for 2.13 too? The reader_base.cc part of this bugfix is a zero day bug for RLE/dict when we have nullable string attributes, so it would need to be backported to every maintenance release we are planning to issue.

We only enabled this for 2.14 right? Is 2.13 effected?

A further question I have is, we didn't allow creation of utf-8 attributes with RLE or Dictionary before format 17, so while the safety check is needed, its not possible for someone to have an array where this would cause a problem?

first question:
The UTF8 part is for 2.14 only, yes. However, those are 2 separate bugfixes Luc is packing in this PR. The "nullable" part in reader_base.cc applies to STRING_ASCII as well and AFAIU it was there since day 0. (@KiterLuc a regression UT would help confirm/show that btw.). Also note that this is a reader bug, so what is written is with previous versions should be fine - no on-disk format affected.

second question:
I believe there is no problem for someone using UTF8 just from the wrong versions (from the first issue yes though, if he is using nullables). AFAIK Luc found the problem because of some backward_compatibility test failing. @KiterLuc can you confirm though ?

[KiterLuc]

Thanks for the fix @KiterLuc ! Added a backport label for 2.14 but I wanted to check with @ihnorton : are we planning some maintenance release for 2.13 too? The reader_base.cc part of this bugfix is a zero day bug for RLE/dict when we have nullable string attributes, so it would need to be backported to every maintenance release we are planning to issue.

We only enabled this for 2.14 right? Is 2.13 effected?
A further question I have is, we didn't allow creation of utf-8 attributes with RLE or Dictionary before format 17, so while the safety check is needed, its not possible for someone to have an array where this would cause a problem?

first question: The UTF8 part is for 2.14 only, yes. However, those are 2 separate bugfixes Luc is packing in this PR. The "nullable" part in reader_base.cc applies to STRING_ASCII as well and AFAIU it was there since day 0. (@KiterLuc a regression UT would help confirm/show that btw.). Also note that this is a reader bug, so what is written is with previous versions should be fine - no on-disk format affected.

second question: I believe there is no problem for someone using UTF8 just from the wrong versions (from the first issue yes though, if he is using nullables). AFAIK Luc found the problem because of some backward_compatibility test failing. @KiterLuc can you confirm though ?

We have no choice to make those two changes at once. Otherwise the back compat tests will not pass. This was actually found by the back compat tests, which include a RLE UTF8 nullable attribute. The back compat tests would actually not pass today if we were to generate back compat arrays for format 17/18, that's why I didn't add a UT for it as we technically have one already.

I have also seen that the back compat arrays previous to format 17 don't RLE encode UTF8 so we are good there.