Validation error when manually copying array data for a slice array

[bjchambers]

Describe the bug

The only commit in this branch https://github.com/bjchambers/arrow-rs/tree/reproduce-validation-error demonstrates the issue. See the added test https://github.com/bjchambers/arrow-rs/blob/reproduce-validation-error/arrow/src/array/data.rs#L1648 here.

Specifically, it creates a struct array, it slices it, then it gets all of the data from that array and attempts to create an identical array data. This fails in the validation due as follows:

thread 'array::data::tests::test_try_new_sliced_struct' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidArgumentError("Struct([Field { name: \"a\", data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: None }, Field { name: \"b\", data_type: Boolean, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: None }]) child array #0 for field a has length smaller than expected for struct array (3 < 4)")', arrow/src/array/data.rs:1738:10
stack backtrace:

To Reproduce
Steps to reproduce the behavior:

Expected behavior
The array data can be constructed since it is a copy of existing (valid) data.

Additional context
Add any other context about the problem here.

[bjchambers]

@alamb in case you have thoughts, since this looks to be in the new validation code.

[alamb]

@bjchambers sounds like a bug introduced in #810 -- are you planning to look at it, or shall I?

[bjchambers]

I haven't looked into it much. If you're able that's great otherwise

I did see a comment near the failure that is suspicious (something about comparing length to length plus offset). But I'm not sure about the intended behavior.

[alamb]

I'll look into it tomorrow first thing in the morning

…

On Wed, Nov 10, 2021 at 5:33 PM Ben Chambers ***@***.***> wrote: I haven't looked into it much. If you're able that's great otherwise I did see a comment near the failure that is suspicious (something about comparing length to length plus offset). But I'm not sure about the intended behavior. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#940 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADXZMLIM65IKVQX74J3C7DULLXKTANCNFSM5HYUX4IQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[alamb]

@bjchambers -- PR with proposed fix is #942