fix: re-enable optional old casting behavior in merge

[ion-elgreco]

Description

The description of the main changes of your pull request

Related Issue(s)

#2851

@ericvandever, this doesn't fix the coercion but allows you to still use large_dtypes=False to downcast so that you can still run most of your queries

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.62%. Comparing base (ad35eda) to head (cbe2265).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2853      +/-   ##
==========================================
- Coverage   72.64%   72.62%   -0.02%     
==========================================
  Files         131      131              
  Lines       40015    40015              
  Branches    40015    40015              
==========================================
- Hits        29068    29062       -6     
  Misses       9087     9087              
- Partials     1860     1866       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rtyler]

@ion-elgreco I'm sure you're getting busier as the autumn is starting, so i'll try to be quick:

• Weren't we deprecating large_dtypes?
• Since there's new logic here, would you have the time to write a test?
• Does this work come back out at some point? Or does this branch need to exist as long as large_dtypes doe?

[ion-elgreco]

@ion-elgreco I'm sure you're getting busier as the autumn is starting, so i'll try to be quick:

• Weren't we deprecating large_dtypes?
• Since there's new logic here, would you have the time to write a test?
• Does this work come back out at some point? Or does this branch need to exist as long as large_dtypes doe?

We are, but some type coercion in Datafusion isn't working properly yet, particularly list <=> large list. So I only re add it temporarily.

The test will be short lived I guess, and the transformation code is well tested so should be fine.

I made an upstream issue for this type coercion problem

Last question I don't get? :p