Install pandas-stubs as a dependency if pandas is used

[twoertwein]

I think a few libraries that are checked by mypy-primer use pandas. It might be nice to install pandas-stubs as a dependency for those projects.

Background: It might be nice if pandas-stubs would use mypy-primer in the (far?) future. Especially if pandas-stubs is promoted by mypy python/mypy#14328 I have absolutely no idea how difficult it would be to integrate a mypy-primer run for pandas-stubs PRs (ideally it would only need to be run on projects that actually use pandas) but I think the first step is to add pandas-stubs as a dependency :)

[hauntsaninja]

Thanks for the suggestion! I think I'd prefer to try to stick to type checking projects as they type check themselves for the per-project dependencies.

The good news though is that this doesn't affect your ability to use mypy_primer in pandas-stubs CI. You can simply put pandas-stubs in MYPYPATH and mypy_primer will forward it along to mypy which will pick it up, and will have precedence over anything installed. This is how we do things when testing third party stubs in typeshed, for instance.

It should be pretty easy to setup. Feel free to tag me on a PR / when I get time I will eventually make one myself.

[twoertwein]

It should be pretty easy to setup. Feel free to tag me on a PR / when I get time I will eventually make one myself.

Thank you, I will come back to you (realistically: it will take some time)!

@Dr-Irv What is your opinion of having a typeshed-like mypy-primer bot for pandas-stubs PRs: bot comments on a PR to indicate how the PR changes the mypy output on various projects?

[Dr-Irv]

@Dr-Irv What is your opinion of having a typeshed-like mypy-primer bot for pandas-stubs PRs: bot comments on a PR to indicate how the PR changes the mypy output on various projects?

Not quite sure I understand what you're proposing, but I think what you're suggesting is that if someone does a PR for pandas-stubs, the bot runs mypy (mypy-primer?) on projects that use pandas-stubs and sees if the PR would change the output for those projects.

If that's the concept, I'm fine with that, although concerned it would increase our CI time, and also the time it takes to review PRs because if the bot picked something up, we'd have to figure out what the change did that caused the issue, whether the issue was the stubs, or if it was invalid use of pandas from the outside project.

[twoertwein]

if someone does a PR for pandas-stubs, the bot runs mypy (mypy-primer?) on projects that use pandas-stubs and sees if the PR would change the output for those projects.

Yes, that is the idea.

although concerned it would increase our CI time

I'm not sure how long it would take. Hopefully, it could be run on a small(er) subset of projects.

and also the time it takes to review PRs because if the bot picked something up, we'd have to figure out what the change did that caused the issue

I think it would be beneficial to catch accidental changes. I can definitely see that when we tighten some arguments that it would trigger new errors (which could take some time to understand which are (in)valid).

[hauntsaninja]

mypy_primer runs in like 8m on typeshed and that's with a much larger set of projects than you'd need.

increase the time it takes to review PRs because if the bot picked something up

You can always just ignore it, which is equivalent to what you're doing today ;-)

[Dr-Irv]

You can always just ignore it, which is equivalent to what you're doing today ;-)

Ignoring what's invisible is easy. Ignoring what's visible is much harder!

[twoertwein]

You can simply put pandas-stubs in MYPYPATH and mypy_primer will forward it along to mypy which will pick it up, and will have precedence over anything installed.

I'm locally playing around with mypy-primer and pandas-stubs but surprisingly it doesn't show me any changes:

git clone -b main --depth 1 [email protected]:pandas-dev/pandas-stubs.git
cp -r pandas-stubs pandas-stubs-new
rm -rf pandas-stubs-new/pandas-stubs/core/  # should cause some changes
python mypy_primer.py --new 1.1.1 --old 1.1.1 -k spark -o diff --old-mypypath pandas-stubs --new-mypypath pandas-stubs-new