Implement periodic update feature

[gaborbernat]

• print out seeded versions in report
• periodically check in a background out of band process if the embed
  wheels are out of date
• promote app data at root level, with three implementations - via disk
  folder, via temp folder and via not available
• introduce a common wheelhouse to not replicate wheels accross python
  versions, decreasing size of the app data folder
• allow manually upgrading the embeded wheels by using the
  --upgrade-embed-wheels flag, in this case no virtual environment is
  created
• upgrade setuptools to 47.1.0

Resolves #1821.
Resolves #1840.

[codecov]

Codecov Report

Merging #1841 into master will increase coverage by 4.99%.
The diff coverage is 93.12%.

@@            Coverage Diff             @@
##           master    #1841      +/-   ##
==========================================
+ Coverage   88.84%   93.84%   +4.99%     
==========================================
  Files          82       86       +4     
  Lines        3946     4159     +213     
==========================================
+ Hits         3506     3903     +397     
+ Misses        440      256     -184     

Flag	Coverage Δ
#tests	93.84% <93.12%> (+4.99%)	⬆️

Impacted Files	Coverage Δ
src/virtualenv/config/cli/parser.py	95.60% <ø> (-0.05%)	⬇️
src/virtualenv/create/pyenv_cfg.py	97.87% <ø> (ø)
src/virtualenv/run/plugin/discovery.py	100.00% <ø> (ø)
src/virtualenv/run/session.py	98.27% <ø> (ø)
...ualenv/seed/embed/via_app_data/pip_install/copy.py	100.00% <ø> (ø)
...env/seed/embed/via_app_data/pip_install/symlink.py	93.18% <ø> (ø)
src/virtualenv/util/error.py	100.00% <ø> (ø)
...virtualenv/seed/embed/via_app_data/via_app_data.py	79.54% <79.54%> (ø)
src/virtualenv/app_data/na.py	81.08% <81.08%> (ø)
src/virtualenv/app_data/via_disk_folder.py	84.54% <84.54%> (ø)
... and 40 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f99353c...36ba4c5. Read the comment docs.

[gaborbernat]

This is now ready to go. I expect to merge it over the weekend and ship it on Monday morning.

[dstufft]

If I'm reading this correctly, this will download the wheel but hold onto it for 28 days or so before using it? Is that right?

[gaborbernat]

If it's downloaded automatically hold onto it until it becomes 28 days old release (one hour after download). You can always make it use right away by manually invoking the upgrade process via the flag.

[dstufft]

Would it be useful if the simple index included the release date to use that instead of downloading and holding onto it? If not that’s fine, just thought it might be a easy optimization.

…

Sent from my iPhone

On Jun 22, 2020, at 7:10 PM, Bernát Gábor ***@***.***> wrote:  If it's downloaded automatically hold onto it until it becomes 28 days old release (one hour after download). You can always make it use right away by manually invoking the upgrade process via the flag. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[gaborbernat]

Would it be useful if the simple index included the release date to use that instead of downloading and holding onto it?

It would be useful, though I'd guess we'd still hold onto it, but we'd not need to go to the PyPi API to fetch it. That being said we'd likely need to keep around the current implementation until the simple index is able to provide this information. Alternatively, it would also work if we could extract this information from the wheel/sdist itself somehow.

[asottile]

I'm seeing breaking (presumably) due to this change. virtualenv is no longer reproducible and I've only just noticed setuptools>=50 despite not changing my virtualenv version.

I think this should be reverted back to when virtualenv was reproducible by default, we're not making the situation better for edge-breaking and in may ways it's more confusing now that it's delayed by several weeks.

[gaborbernat]

This topic has been over discussed, and both sides made their point. There has been a majority desire both on pypa and general public side of some form of auto upgrade, which has been implemented here. There are configurations available to achieve the old behavior, but we don't plan to revert this feature. (For what it's worth I was also against this feature personally, but I consider it a decent middle ground over the old always update behavior).