version 1.6.9 fails with null module __file__ attribute

[gshuflin]

On my arch linux system, after the pants upgrade to version 1.6.9, a number of pieces of pants functionality that make use of pex started failing with the following error message:

                     Traceback (most recent call last):
                       File "/home/gregs/code/pants/.pants.d/test/pytest-prep/CPython-3.7.3/7c81bb6ec656c4064b548c392d519f0695716e81/.bootstrap/pex/pex.py", line 356, in execute
                         self.patch_sys(pex_inherit_path)
                       File "/home/gregs/code/pants/.pants.d/test/pytest-prep/CPython-3.7.3/7c81bb6ec656c4064b548c392d519f0695716e81/.bootstrap/pex/pex.py", line 285, in patch_sys
                         new_sys_path, new_sys_path_importer_cache, new_sys_modules = self.minimum_sys(inherit_path)
                       File "/home/gregs/code/pants/.pants.d/test/pytest-prep/CPython-3.7.3/7c81bb6ec656c4064b548c392d519f0695716e81/.bootstrap/pex/pex.py", line 259, in minimum_sys
                         sys_modules = cls.minimum_sys_modules(site_libs)
                       File "/home/gregs/code/pants/.pants.d/test/pytest-prep/CPython-3.7.3/7c81bb6ec656c4064b548c392d519f0695716e81/.bootstrap/pex/pex.py", line 179, in minimum_sys_modules
                         if cls._tainted_path(module_file, site_libs):
                       File "/home/gregs/code/pants/.pants.d/test/pytest-prep/CPython-3.7.3/7c81bb6ec656c4064b548c392d519f0695716e81/.bootstrap/pex/pex.py", line 157, in _tainted_path
                         paths = frozenset([path, os.path.realpath(path)])
                       File "/home/gregs/code/pants/build-support/pants_dev_deps.py37.venv/lib/python3.7/posixpath.py", line 394, in realpath
                         filename = os.fspath(filename)
                     TypeError: expected str, bytes or os.PathLike object, not NoneType

It looks like what's going on here is that https://github.com/pantsbuild/pex/blob/master/pex/pex.py#L178 is iterating over every module and extracting the __file__ attribute of that mdoule. However, some module on my system seems to have a None value for that attribute, which is causing code downstream of that line to fail. It looks like the commit b5f7e41 might be relevant.

[jsirois]

For posterity, this was a specific issue with namespace packages in Python 3.7+:

$ python3.6 -mvenv /tmp/twitter-3.6
$ python3.7 -mvenv /tmp/twitter-3.7
$ /tmp/twitter-3.6/bin/pip install -U pip twitter.common.collections twitter.common.dirutil
Collecting pip
  Using cached https://files.pythonhosted.org/packages/8d/07/f7d7ced2f97ca3098c16565efbe6b15fafcba53e8d9bdb431e09140514b0/pip-19.2.2-py2.py3-none-any.whl
Collecting twitter.common.collections
Collecting twitter.common.dirutil
Collecting twitter.common.lang==0.3.11 (from twitter.common.collections)
Installing collected packages: pip, twitter.common.lang, twitter.common.collections, twitter.common.dirutil
  Found existing installation: pip 18.1
    Uninstalling pip-18.1:
      Successfully uninstalled pip-18.1
Successfully installed pip-19.2.2 twitter.common.collections-0.3.11 twitter.common.dirutil-0.3.11 twitter.common.lang-0.3.11
$ /tmp/twitter-3.7/bin/pip install -U pip twitter.common.collections twitter.common.dirutil
Collecting pip
  Using cached https://files.pythonhosted.org/packages/8d/07/f7d7ced2f97ca3098c16565efbe6b15fafcba53e8d9bdb431e09140514b0/pip-19.2.2-py2.py3-none-any.whl
Collecting twitter.common.collections
Collecting twitter.common.dirutil
Collecting twitter.common.lang==0.3.11 (from twitter.common.collections)
Installing collected packages: pip, twitter.common.lang, twitter.common.collections, twitter.common.dirutil
  Found existing installation: pip 19.0.3
    Uninstalling pip-19.0.3:
      Successfully uninstalled pip-19.0.3
Successfully installed pip-19.2.2 twitter.common.collections-0.3.11 twitter.common.dirutil-0.3.11 twitter.common.lang-0.3.11
$ /tmp/twitter-3.6/bin/python
Python 3.6.9 (default, Jul  8 2019, 20:56:32) 
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import twitter
>>> print(twitter.__file__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'twitter' has no attribute '__file__'
>>> 
$ /tmp/twitter-3.7/bin/python
Python 3.7.4 (default, Jul 16 2019, 07:12:58) 
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import twitter
>>> print(twitter.__file__)
None
>>> 

In particular, this was a CPython cleanup noted here: https://docs.python.org/3/whatsnew/3.7.html#other-cpython-implementation-changes as a bug fix for https://bugs.python.org/issue32305.