REF: de-duplicate Block.__init__

[jbrockmendel]

• closes #xxxx
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[jreback]

ok for 1.2

[jreback]

@jbrockmendel if you can address these comments can get this in

[jbrockmendel]

xref https://issues.apache.org/jira/browse/ARROW-10768

[jorisvandenbossche]

What's the rationale for this change? Why is it needed to deprecate it? (pyarrow code currently works fine? Or is there a problem with it)

[jbrockmendel]

What's the rationale for this change? Why is it needed to deprecate it? (pyarrow code currently works fine? Or is there a problem with it)

We recently made it so that all our internal calls explicitly pass ndim. Now we're telling other packages (that ideally wouldnt be accessing internals in the first place) they need to do the same.

[jorisvandenbossche]

they need to do the same.

But so is there an actual reason to require this, apart from "we are not using the keyword anymore internally" ?
(I didn't closely follow your prior work related to ndim, so I am honestly asking. Does it give risks on bugs for the external users, or is it slower, or is it hard to maintain that functionality for us, or ... when they don't explicitly pass the ndim?)

If we do the change, we could also keep this for after the 1.2 RC, so the deprecation is only for 1.3, which gives pyarrow time to fix the code and have a release before this is released by pandas.

[jbrockmendel]

But so is there an actual reason to require this, apart from "we are not using the keyword anymore internally" ?
(I didn't closely follow your prior work related to ndim, so I am honestly asking. Does it give risks on bugs for the external users, or is it slower, or is it hard to maintain that functionality for us, or ... when they don't explicitly pass the ndim?)

The motivation that led me to update all of our internal usages to always pass ndim explicitly was that on other branches I was getting errors related to incorrect ndim inference. (Keep in mind I have active branches for both ArrayManager and EA2D among others and I don't recall off the top of my head which one(s) were producing these errors)

or is it slower

ndim inference isn't going to be particularly slow, but we do call the Block constructors a lot, so it could add up.

or is it hard to maintain that functionality for us

In general I think being more explicit and more strict in internals code makes maintenance easier, yes.

[simonjayhawkins]

If we do the change, we could also keep this for after the 1.2 RC, so the deprecation is only for 1.3, which gives pyarrow time to fix the code and have a release before this is released by pandas.

@jbrockmendel can we remove the blocker tag? and do this for 1.3?

[jbrockmendel]

and do this for 1.3?

sounds good

[jbrockmendel]

Mothballing pending apache/arrow#8957

[jbrockmendel]

re-opening; now that make_block is exposed specifically for downstream, we can do the validation there and be stricter in private-internals

[jreback]

lgtm @jorisvandenbossche if any issues.

[jorisvandenbossche]

To make it explicit: the PR now no longer includes a deprecation warning. But you still changed the behaviour of Block.__init__, or not?

[jbrockmendel]

To make it explicit: the PR now no longer includes a deprecation warning. But you still changed the behaviour of Block.init, or not?

Block.__init__ is changed, but the relevant validation is now done in api.make_block, so downstream libraries using that will be unaffected.

[jorisvandenbossche]

Yep, but thus downstream libraries using Block can be affected. But since we are not aware of such libraries, and the known ones use make_block / make_block_same_class, that's probably OK

[jreback]

Yep, but thus downstream libraries using Block can be affected. But since we are not aware of such libraries, and the known ones use make_block / make_block_same_class, that's probably OK

great

[jreback]

thanks @jbrockmendel

[jbrockmendel]

alright! between this, #40262, and a couple follow-ups to this, im hopeful to take another big chunk out of the difference between libreduction-vs-purepython paths (xref #40263)