fix 20ms startup penalty added by virtualenv

[asottile]

before:

$ rm -rf vvv; virtualenv vvv -ppython3.10; time vvv/bin/python3 -c ''; time vvv/bin/python3 -c ''
created virtual environment CPython3.10.2.final.0-64 in 712ms
  creator CPython3Posix(dest=/tmp/virtualenv/vvv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/asottile/.local/share/virtualenv)
    added seed packages: pip==22.0.4, setuptools==60.9.3, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

real	0m0.037s
user	0m0.037s
sys	0m0.000s

real	0m0.032s
user	0m0.031s
sys	0m0.000s

after:

$ rm -rf vvv; virtualenv vvv -ppython3.10; time vvv/bin/python3 -c ''; time vvv/bin/python3 -c ''
created virtual environment CPython3.10.2.final.0-64 in 677ms
  creator CPython3Posix(dest=/tmp/virtualenv/vvv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/asottile/.local/share/virtualenv)
    added seed packages: pip==22.0.4, setuptools==60.9.3, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

real	0m0.022s
user	0m0.022s
sys	0m0.000s

real	0m0.015s
user	0m0.015s
sys	0m0.000s

just to confirm the hack still works as well:

$ vvv/bin/python3 -c 'import distutils.dist; print(distutils.dist.Distribution.parse_config_files.__module__)'
_virtualenv

Thanks for contributing, make sure you address all the checklists (for details on how see

development documentation)!

• ran the linter to address style issues (tox -e fix_lint)
• wrote descriptive pull request text
• ensured there are test(s) validating the fix (N/A)
• added news fragment in docs/changelog folder
• updated/extended the documentation (N/A)

[gaborbernat]

Can you explain from where does the big performance gain comes?

[asottile]

Can you explain from where does the big performance gain comes?

deferring the imports so they aren't in the critical path

[gaborbernat]

deferring the imports so they aren't in the critical path

But the import needs to happen no matter what, so does it matter if you run it at import time or when the pth file is loaded? Also how come this import is so expensive?

[asottile]

deferring the imports so they aren't in the critical path

But the import needs to happen no matter what, so does it matter if you run it at import time or when the pth file is loaded? Also how come this import is so expensive?

nope! the import only needs to happen when distutils is imported

the import is expensive because it pulls in 15+ other modules (including typing):

$ importtime-waterfall --include-interpreter-startup t
_frozen_importlib_external (320)
  _io (170)
  marshal (43)
  posix (386)
zipimport (157)
  time (97)
encodings (578)
  codecs (351)
    _codecs (56)
  encodings.aliases (447)
encodings.utf_8 (216)
_signal (124)
io (176)
  abc (126)
    _abc (35)
site (950)
  os (421)
    stat (72)
      _stat (55)
    _collections_abc (950)
    posixpath (74)
      genericpath (42)
  _sitebuiltins (72)
  _virtualenv (386)
    functools (724)
      collections (985)
        itertools (220)
        keyword (140)
        operator (325)
          _operator (73)
        reprlib (184)
        _collections (70)
      types (263)
      _functools (57)
    importlib.abc (549)
      importlib (182)
        warnings (261)
      importlib.machinery (69)
      importlib._abc (151)
      importlib.resources.abc (27)
        importlib.resources (162)
          importlib.resources._common (311)
            pathlib (945)
              fnmatch (177)
                re (538)
                  enum (2545)
                  sre_compile (322)
                    _sre (75)
                    sre_parse (371)
                      sre_constants (329)
                  _locale (212)
                  copyreg (162)
              ntpath (131)
                nt (76)
                nt (70)
                nt (104)
                nt (101)
                nt (68)
              errno (70)
              urllib.parse (1284)
                urllib (123)
            tempfile (421)
              shutil (634)
                zlib (81)
                bz2 (403)
                  _compression (223)
                  _bz2 (213)
                lzma (259)
                  _lzma (299)
              random (517)
                math (71)
                bisect (142)
                  _bisect (36)
                _random (43)
                _sha512 (46)
              weakref (434)
                _weakrefset (230)
            contextlib (739)
            typing (2645)
              collections.abc (185)
              _typing (118)
            importlib.resources.abc (266)
            importlib.resources._adapters (322)
          importlib.resources._legacy (237)
    importlib.util (122)
  _distutils_hack (328)
  sitecustomize (254)
t (218)

after:

$ importtime-waterfall --include-interpreter-startup t
_frozen_importlib_external (305)
  _io (165)
  marshal (42)
  posix (385)
zipimport (123)
  time (93)
encodings (603)
  codecs (348)
    _codecs (56)
  encodings.aliases (443)
encodings.utf_8 (215)
_signal (122)
io (170)
  abc (123)
    _abc (35)
site (806)
  os (414)
    stat (68)
      _stat (53)
    _collections_abc (889)
    posixpath (72)
      genericpath (40)
  _sitebuiltins (71)
  _virtualenv (371)
  _distutils_hack (408)
  sitecustomize (123)
t (209)

[ofek]

Nice!