Winget upgrade lists and installs Python 3 twice (and any time I start the upgrade)

[ChristianBeier]

Brief description of your issue

When I start winget upgrade, it lists python 3 twice and when I do upgrade --all it is downloading and installing python-3.9.6-amd64.exe twice. AFAIK it did this even three weeks ago before my vacation.

Steps to reproduce

Execute "winget upgrade" in PowerShell

Expected behavior

Each package should be listed only once and in order matching possible dependencies. After I finished the upgrade, the package should not be listed any longer.

Actual behavior

Python 3 (and I think .net before) is listed, downloaded, and upgraded several times what wastes bandwidth and compute power.

Environment

Surface Laptop 3, Win 11,
Windows Package Manager v1.0.11694
Windows: Windows.Desktop v10.0.22000.160
Package: Microsoft.DesktopAppInstaller v1.15.11694.0

[denelon]

This looks like the same behavior reported in #1212. The upgrade command is checking the list of installed packages and seeing more than one installed. It attempts to upgrade both packages, but they may be installed side-by-side so the new version is installed twice.

[ghost]

@ChristianBeier this issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.

[ghost]

@ChristianBeier this issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.

[mavaddat]

I see the exact same issue with the twist that I can neither uninstall the stale 3.9.5150.0 version of Python nor upgrade it.

Brief description of your issue

Upgrading from cmd, PowerShell user or admin using winget upgrade --all or winget upgrade will always declare that there is an update to Python3 because winget has installed two versions of Python while it holds back the older 3.9.*.* version.

winget list --name Python

Name	Id	Version	Available	Source
Python Launcher	{463B0974-B1E1-401E-8F59-B0F9F81258E4}	3.10.7581.0
Python 3	Python.Python.3	3.10.150.0		winget
Python 3	Python.Python.3	3.9.5150.0	3.10.150.0	winget

Steps to reproduce

Try any or all of these:

1. winget upgrade --all
2. winget upgrade --id Python.Python.3 --version 3.9.5150.0
3. winget upgrade --id Python.Python.3
4. winget list --name Python
5. winget uninstall --id Python.Python.3 --version 3.9.5150.0
6. winget uninstall --id Python.Python.3

Expected behavior

Able to uninstall the Python 3.9 using winget command 4, or 5 in the steps to reproduce section.

Actual behavior

Cannot upgrade or uninstall the stunted version of Python using any of the winget commands in the steps to reproduce section.

Environment

OS Name	Platform	ServicePack	Version	VersionString
Microsoft Windows 11 Pro Insider Preview	Win32NT		10.0.22504.0	Microsoft Windows NT 10.0.22504.0

pyenv 2.64.11

$ where.exe python
C:\Users\mavad\AppData\Local\Programs\Python\Python310\python.exe
C:\Users\mavad\.pyenv\pyenv-win\shims\python
C:\Users\mavad\.pyenv\pyenv-win\shims\python.bat

[denelon]

@mavaddat can you confirm if the latest version of python is being installed? This looks like the scenario for packages that support "side by side" installation, and the earlier version isn't removed.

The Windows Package Manager is checking to see what is installed, and it finds a match to Python.Python.3 in this case. It attempts to "upgrade" to the latest version of the package, but the upgrade isn't removing the previous version, or upgrading it in place. When you run winget upgrade again, the same thing is happening over again.

We've got an issue on the backlog to see if we can detect that the latest version of a package is already installed on a machine so we will skip this package for winget upgrade --all.

• winget upgrade --all shouldn't install the same package more than once #1105

[mavaddat]

@mavaddat can you confirm if the latest version of python is being installed?

@denelon Yes, I am installing the latest Python 3 stable, which is 3.10.150.0 at the time of writing.

This looks like the scenario for packages that support "side by side" installation, and the earlier version isn't removed.

Right. I also confirm that I have two system installs of Python 3¹, which are 3.9.5150.0 and 3.10.150.0 — both of which were installed using winget.

That is, I used winget to install the latest Python 3 a few weeks ago (when 3.9 was the latest stable), so winget correctly installed Python 3.9.5150.0; however, when I tried to update my packages using winget upgrade —all, winget incorrectly installed a parallel newer version of Python 3 (being 3.10.150.0). This behaviour is incorrect, because winget upgrade —all should only upgrade existing packages, not install parallel packages.

Currently, I am unable to use winget to uninstall either version of Python 3 (see screen recording above).

The Windows Package Manager is checking to see what is installed, and it finds a match to Python.Python.3 in this case. It attempts to "upgrade" to the latest version of the package, but the upgrade isn't removing the previous version, or upgrading it in place. When you run winget upgrade again, the same thing is happening over again.

It’s worse than that, because I cannot even remove the Pythons that winget installed.

We've got an issue on the backlog to see if we can detect that the latest version of a package is already installed on a machine so we will skip this package for winget upgrade --all.

• winget upgrade --all shouldn't install the same package more than once #1105

That makes sense. An additional question is, “why doesn’t winget ask the user before making a parallel install?”

[1]: I also have pyenv installs of Python 3, but these are in %USERPROFILE% and not on the path.

[denelon]

@mavaddat, MSIX and MSI installers will typically "upgrade" the existing package. That doesn't hold true for .exe packages. Sometimes they upgrade, and sometimes they install side by side. I've even seen examples where this behavior changes, and doesn't align with semantic versioning.

We don't have solid deterministic logic to figure out what they are going to do in advance. Some users want the "upgrade" scenario to remove and install the latest, while other want the side-by-side install. We've been collecting feedback, and looking at how users expect the product to behave. We need to have rational default behaviors, and then support customized behaviors for "power users". We also don't want a bunch of messy conditional logic based on the package, as that would become unmaintainable over time.

We are building up additional logic and behavior in or validation infrastructure. The goal is to be able to observe what happens in the upgrade case, and ensure the manifest correctly identifies the behavior so the user can be informed and make choices.

[mavaddat]

Would a package maintainer (person, organization, or entity who first publishes a package to winget) not be responsible for the scripts that install or update the package?

Here are my suggestions:

• The package maintainer should be responsible for building the install, uninstall and upgrade *.ps1 scripts' logic.
  • Without these scripts specified in the package, winget will make a default best-effort to install/upgrade/uninstall
  • The default effort will not guarantee that it uninstalls or upgrades correctly (i.e., may result in parallel installs of same package with different versions).
• winget repository should provide package script templates with typical pre-defined install/update/uninstall procedures for package publishers.
• winget-cli should be able to uninstall a package.
  • Uninstalling should minimally consist of removing the program from system environment variables and removing all unshared files and folders created upon install.
  • winget repository should calculate the consequences of a package's uninstall script in a containerized setting upon package submission.
  • A package whose uninstall script cannot be shown to meet the minimum specification will rely on the default winget-cli behaviour instead.

[ChristianBeier]

Sorry for the late reply. It slipped through the last months.
As I had a clean installation of my system recently, I think I can reproduce the issue. Most likely, python version 3.9.5150.0 got installed when installing vsode or other tools on my laptop. I was also not able to do an uninstall via winget, removing it via apps & features worked and now I have only python 3.10.150.0 installed.

[denelon]

@mavaddat,

When we think about a "package" in Windows, it's generally not just a set of files to be placed on the system. The state of the art has been evolving over many years. Publishers or Independent Software Vendors (ISVs) generally build an installer to handle various aspects of registering the software with Apps & Features, writing registry entries, and file placement. With the MSIX package format, things become a bit easier to manage on the uninstall and upgrade behaviors, but that also comes with interesting caveats related to how they are installed and virtual registry entries are recorded per user.

The Windows Package Manager is attempting to meet developers and ISVs where they are. We don't want to require anyone to rewrite their packages or installers. Some of our design goals are listed below.

1. No scripts during install – something that we completely agreed with and don’t allow with MSIX
2. Rich manifest definition within GitHub – the power of being open combined with rich declarative meta data about the app is so important to meet goal 1
3. Support all types of Windows applications installers
4. Seamless updates for applications in the repository

We are adding additional enhancements to our automated validation system. We do want to verify what happens with uninstall and upgrade scenarios to ensure the manifest has the best or "expected" behaviors defined. Unfortunately, there are many edge cases with legacy installers. We've been discussing ways to somehow rate or identify different issues with packages, so settings could be applied to inform users about findings during validation, and possibly settings to filter out packages with certain limitations or undesired behaviors for certain scenarios.

[yonizaf]

I think the problem here is that 'minor' Python 3 versions (3.8, 3.9 etc.) should be treated as different packages (that they are by design), the same way Python 2 is a separate package and does not upgrade into Python 3.

For example, if I have Python 3.8.7 installed, I probably want to upgrade it to 3.8.13, rather than 3.10.x

Similar (dev related) software by Microsoft already follow that logic, for example dotnet runtimes or VC redist, they don't try to just install the latest but rather upgrade each minor version to its latest patch.

[NJT145]

The problem here is that package with id=Python.Python.3 should be a meta package that will points to the latest version of Python.Python.3.*. I don't know if there is this feature, but this is a must for a package manager.

[denelon]

@ChristianBeier We released WinGet 1.4 and it's been a while since this issue was reported. Since then, we've added substantial improvements to matching installed packages and versions. Python has also been split into minor versions:

We've also added some logic to detect when multiple packages are installed side-by-side and the latest is installed, we don't attempt to download and install the latest version for each earlier version.

Can you confirm if this Issue is still actively causing trouble?

@NJT145 the meta-package idea along with a couple of others is being tracked at:

• Handlers schema for other features #1902

[GalaxySnail]

I'm not sure what's happening, but this issue is still here.

$ winget --version
v1.4.10173
$ winget source update && winget upgrade
正在更新所有源...
正在更新源: msstore...
完成
正在更新源: winget...
完成
名称                      ID                       版本             可用             源
-------------------------------------------------------------------------------------------
Python 3.10.8 (32-bit)    Python.Python.3.11       < 3.11.0         3.11.1           winget
Python 3.10.9 (64-bit)    Python.Python.3.11       < 3.11.0         3.11.1           winget
Python 3.9.13 (64-bit)    Python.Python.3.11       < 3.11.0         3.11.1           winget
3 升级可用。

[denelon]

@GalaxySnail, thanks for sharing.

I'll keep the issue open.

[Trenly]

[Policy] Area-Matching
[Policy] Side-By-Side

[denelon]

We've been working on the side-by-side scenarios. You could try the latest release and enable the experimental feature on:

• https://github.com/microsoft/winget-cli/releases/tag/v1.8.924-preview

Please provide feedback at:

• Side by Side Experimental Feature Feedback #4281

[denelon]

Is this still a problem with WinGet 1.8?

[GalaxySnail]

It has been fixed in winget v1.8.1791. Thanks for your efforts!

[Trenly]

Close with reason: Resolved;