[Data] Fix OutputBlockBuffer to avoid repeatedly copying remainder block

[alexeykudinkin]

Why are these changes needed?

Currently, inside OutputBlockBuffer we're

1. Repeatedly copying remainder of the original block, bringing total # of bytes copied to O(N^2) (where N is the size of the original block)
2. Creating potentially very large blocks (like in [Data] Cannot load large JSONL file #48236) that could overflow underlying Arrow data types.

This change addresses both of these issues, by establishing following protocol where

1. Finalized target blocks are copied, while
2. Remainder block is NOT (therefore continuing referencing original block)

Addresses #48236

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[pcmoritz]

Great find! So my understanding is that .slice here

ray/python/ray/data/_internal/arrow_block.py

Line 284 in 22e266a

view = _copy_table(view)

concatenated all the chunks in the ChunkedArray and that led to the overflow? :)

[pcmoritz]

There are some test failure but this seems like the right fix :)

[alexeykudinkin]

concatenated all the chunks in the ChunkedArray and that led to the overflow? :)

Correct. And we'd avoid copying here as well as it's absolutely unnecessary.