GH-35329: [Python] Address pandas.types.is_sparse deprecation

[mroeschke]

Rationale for this change

pandas is deprecating pandas.types.is_sparse in 2.1

What changes are included in this PR?

Changing usages of is_sparse to the recommended migration isinstance(dtype, pd.SparseDtype)

Are these changes tested?

Are there any user-facing changes?

No

closes #35329

• Closes: [Python] pandas is_sparse attribute is deprecated #35329

[AlenkaF]

Thank you for the update! +1
The MacOS failure is not related.

[github-actions]

• Closes: [Python] pandas is_sparse attribute is deprecated #35329

[github-actions]

⚠️ GitHub issue #35329 has been automatically assigned in GitHub to PR creator.

[ursabot]

Benchmark runs are scheduled for baseline = b73ddc3 and contender = de6c3cd. de6c3cd is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️1.54% ⬆️0.09%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️3.7% ⬆️2.5%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] de6c3cd2 ec2-t3-xlarge-us-east-2
[Finished] de6c3cd2 test-mac-arm
[Finished] de6c3cd2 ursa-i9-9960x
[Finished] de6c3cd2 ursa-thinkcentre-m75q
[Finished] b73ddc33 ec2-t3-xlarge-us-east-2
[Finished] b73ddc33 test-mac-arm
[Finished] b73ddc33 ursa-i9-9960x
[Finished] b73ddc33 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

[ursabot]

['Python', 'R'] benchmarks have high level of regressions.
test-mac-arm

[EBenkler]

I think this only fixes the issue in python/pyarrow/pandas-shim.pxi, but is_sparse() is also used here:
\envs\develop\lib\site-packages\pyarrow\pandas_compat.py:373: FutureWarning: is_sparse is deprecated and will be removed in a future version. Check isinstance(dtype, pd.SparseDtype) instead.
if _pandas_api.is_sparse(col):