Suppressing warnings triggers "Different tests were collected between" error for Python 3.8

[fbunt]

I very recently started getting the "Different tests were collected between gwX and gwY" error when running tests with multiple workers. It only happens under specific circumstances though. The circumstances are:

• python=3.8
• Number of workers > 2
• Warnings suppressed with "-p no:warnings" option

Versions:

$ conda list | grep pytest
pytest                    7.1.2            py38h578d9bd_0    conda-forge
pytest-forked             1.4.0              pyhd8ed1ab_0    conda-forge
pytest-xdist              2.5.0              pyhd8ed1ab_0    conda-forge

The other Python versions that I tested with were 3.7 and 3.9. I had no issues with them.

I found #432 but because of the specific circumstances, I created a new issue. I can close this and post the relevant info on that issue if needed.