Upgrading or installing Python latest version (3.8) does not override pip3 link

[mx-psi]

• ran brew update and can still reproduce the problem? Yes
• ran brew doctor, fixed all issues and can still reproduce the problem? Yes
• ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link? no formula failed, see below
• if brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue? n/a

What you were trying to do (and why)

I include links to the CircleCI output when possible, these should be accessible if logged to a CircleCI account.)

We run a number of CI tests on macOS for the datadog-agent repository. For this tests, we use brew to install several packages including python.

What happened (include command output)

Since around 11 am (CET) the tests started failing because the pip command could not be found (example logs available here).

Before the fix, we used this script for the setup.
The relevant lines are:

brew uninstall python@2 || true
brew upgrade python -f || brew install python -f
ln -s /usr/local/bin/pip3 /usr/local/bin/pip

We then use pip (which links to pip3) to install some packages (see here), which is where the CI fails.
After some digging, we believe the likely culprit to be PR #54912. The logs show that some links get overwritten:

[email protected] install overwrites

==> Installing [email protected]
==> Pouring [email protected]_1.high_sierra.bottle.tar.gz
Warning: These files were overwritten during `brew link` step:
/usr/local/bin/2to3
/usr/local/bin/idle3
/usr/local/bin/pydoc3
/usr/local/bin/python3
/usr/local/bin/python3-config
/usr/local/share/man/man1/python3.1
/usr/local/lib/pkgconfig/python3.pc 

but the pip3 link does not get overwritten, which seems to be what caused our CI to fail. The fix was to call pip as a module, see DataDog/datadog-agent-buildimages#61

Alternatively, we could have fixed it by rewriting the pip3 link ourselves, since the output from brew says

You can install Python packages with
 
  /usr/local/opt/[email protected]/bin/pip3 install <package>

What you expected to happen

We expected it to work correctly as it did before 11 am CET (example of successful log from yesterday)

Step-by-step reproduction instructions (by running brew install commands)

brew upgrade python -f || brew install python -f
pip3

[Bo98]

Thanks for the detailed report! I have a fix in #57654.

[Bo98]

Should be fixed now!

Also released brew 2.4.5 with some general fixes to the upgrade process.