GH-44555: [C++][Compute] Allow casting struct to bigger nullable struct

[Tom-Newton]

Rationale for this change

Sometimes its useful to add a column full of nulls in a cast. Currently this is supported in top level columns but not inside structs. Example where this is important: delta-io/delta-rs#1610

What changes are included in this PR?

Add support for filling in columns full of null for nullable struct fields.
I've gone for a fairly minimal change that achieves what I needed but I wonder if there should be a more significant change so that this casting is done by field name and ignore the field order.

Are these changes tested?

Yes. The expected behaviour in several existing tests has been altered and a couple of new tests have been added.

I also rolled out a custom build with this change internally because it suddenly became a critical problem.

Are there any user-facing changes?

Yes. There are scenarios that previously failed with struct fields don't match or are in the wrong order but now succeed after filling in nulls.

• GitHub Issue: [C++][Compute] Support casting a smaller struct to a bigger one if extra fields are nullable #44555

[github-actions]

⚠️ GitHub issue #44555 has been automatically assigned in GitHub to PR creator.

[Tom-Newton]

I'm not confident that I've taken the right approach here but I think its ready for review.

[Tom-Newton]

Ok, it looks like there is at least one more tests I need to update. I can probably sort that tomorrow.

[Tom-Newton]

I suspect this was not supposed to be different from in the physical schema. This data is used to test if a virtual column will be materialised so I guess it shouldn't be testing int64 -> float64 casting at the same time.

[Tom-Newton]

Sorry for the direct ping but @lidavidm please could you review when you get a chance. I assume you are the right person to review this?

[lidavidm]

Looks reasonable to me, thanks. CC @bkietz for some more eyes?

[lidavidm]

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c14d55d.

There were 132 benchmark results with an error:

• Commit Run on arm64-t4g-2xlarge-linux at 2024-11-25 02:34:37Z
  • tpch (R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=1
  • tpch (R) with engine=arrow, format=native, language=R, memory_map=False, query_id=TPCH-09, scale_factor=1
• and 130 more (see the report linked below)

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 2 possible false positives for unstable benchmarks that are known to sometimes produce them.

[Tom-Newton]

Ok, I don't think that one CI failure is related here. Merging now. Thanks @Tom-Newton, sorry for the delays!

Thanks for reviewing 🙂