packaging - Add auto-pinning feature for packages with self-updaters

[vertigo220]

As discussed in issue #1573, consider the addition of a method for packages to be marked as self-updating software (programs that have a built-in auto-update feature), and auto-pin those packages to prevent sync issues between what Chocolatey thinks is installed and what is actually installed. This could be done by the package creators/maintainers or perhaps even by users via flagging them (though obviously in a way that would prevent a few users from incorrectly doing so).

[bcurran3]

I believe the reverse should be true. If you want true package management from Chocolatey, then the packages should be installed via Chocolatey. That way Chocolatey AKA your package manager, knows what's installed and can manage the package (program) the way it's intended.

Don't get me wrong. I understand your point 100%. The program auto-updater will most likely always beat Chocolatey to the upgrade. I think both strategies are good and this is an issue that there will always be a debate which is better and the "better" is probably to make both sides happy.

If you believe in Chocolatey being your package manager and controlling updates, you can reach out to the package maintainer's of programs that auto-update and ask them for a package parameter to disable the auto-updater program.

I do believe it's against "Chocolatey law" to run a choco command within an installation/uninstallation script (and it can currently break things).

I have been toying with the idea for sometime to make a program related to this. I just probably won't have the free time for a while.

[vertigo220]

Well, I like the idea of Chocolatey for helping keep programs up-to-date more than I would myself (if they don't auto update, I often won't update them for months or even a year or two). But I don't think that should mean using a version that's months old and has been updated multiple times since the version in Chocolatey. Obviously, the best scenario would be to ensure all packages are updated in a timely manner, but obviously that's not the reality of the situation. Perhaps it could be an option whether to leave everything up to Chocolatey or to always have the latest versions, which means allowing apps to auto-update. Another option would be to make Chocolatey able to see the current installed version, instead of just assuming it's the last version it installed, so if they don't match it could realize the program was updated externally and take various actions, but that ability is limited to licensed versions only.

[tylerszabo]

I've submitted a PRs to skip downloads of an installer if the package is already updated for packages that I used and did redundant downloads. I've found maintainers to be quite receptive if you implement it according to the way they're writing their scripts. I think the best way is to use Get-UninstallRegistryKey if the software is reliably updating the version there, but sometimes scraping other sources might work.

Examples:

• Origin
• Twitch

I think by pinning these packages you're missing out on possible important changes to scripts (such as fixing a broken uninstaller or working around a buggy patch from the packaged software).

With that said, I think there is value in having an auto-pin feature for other purposes (or even for this purpose if a maintainer isn't responsive and you don't want to re-package yourself).

[AdmiringWorm]

@tylerszabo I think you mixed the links for the twitch and origin packages.

BTW, I do find the idea of auto-pinning to be an interesting idea, not completely sure if it's really doable without having the maintainer supplying extra data to the functions though (well, not reliably at least).

[tylerszabo]

@AdmiringWorm, thanks - fixed :)

[bcurran3]

I can see the future - this will be commented on by Rob as saying "unsupported," "not in public API," etc. etc., yadda yadda...

Theoretical way to implement auto-pinning as a maintainer:

if ($pp["PIN"] -eq 'yes')
{
 Write-Output "" >"$env:ChocolateyInstall\.chocolatey\$packageName.$packageVersion\.pin"
 # e.g. C:\ProgramData\chocolatey\.chocolatey\GoogleChrome.68.0.3440.7500\.pin
}

The .pin file in that directory is the flag that causes the package to be pinned.

Where there's a will(iam?), there's a way.

[bcurran3]

@AdmiringWorm What's up? I KNOW you get it! Why confused emoji? Questions?

I'll expound for anyone who doesn't understand: (This info is for package maintainers.)

When you choco pin add -n=packagename --version packageversion, Chocolatey creates a .pin file in the C:\ProgramData\chocolatey\.chocolatey\packagename.packageversion\ folder. This .pin file is checked during upgrades and uninstalls. If the .pin file is found, neither an upgrade nor an uninstall will occur. The .pin file is blank and the contents are ignored. NOTE: All this is undocumented, unsupported, and could change in future versions. (There are talks of Chocolatey using a database for package information in the future - thus this pinning mechanism would probably go bye-bye... There, I saved @ferventcoder some typing.)

To implement an auto-pinning feature in a package:

Your packagename.nuspec file needs:

    <dependencies>
      <dependency id="chocolatey-core.extension" version="1.1.0" />
    </dependencies>

Add the following to your chocolateyInstall.ps1

if ($pp["PIN"] -eq 'yes')
{
 #creates an empty .pin file in the super secret Chocolatey area with info about packages, thus pinning it
 Write-Output "" >"$env:ChocolateyInstall\.chocolatey\$packageName.$packageVersion\.pin"
 #informs installing user that the package has been auto-pinned
 Write-Host "$packageName auto-pinned by request." -foreground yellow
}

Then when a user does cinst packagename --params "'/PIN:yes'" the package is auto-pinned. Easy to implement and COMPLETELY up to the installing user to use or not.

...and "because 95% of packages are on internal repositories" (@ferventcoder 2018 ad infinitum), enterprise Chocolatey users with internalized packages would just skip the if statement part and make it default to pin packages of their choice. (I wanted to make sure I covered everyone...well, I guess, the majority.)

A safe way to implement for those worried about future compatibility problems would be to add:

<dependency id="chocolatey" version="[0.9.9,0.10.12.0)" />

The above would make the package dependent on Chocolatey v0.9.9 (Implementation of pinning.) through the current version but not work with the next release as it would have to be tested if the auto-pinning was still functional or not.

I hope that covers all the bases. I think so.

Where there's a will(iam?), there's a way. :)

[bcurran3]

Since I'm engrossed in this subject matter at the moment and want to show the opposing side...

Some examples that could be done to disable self-updating programs in packages by implementing a --params "'/DISABLEUPDATE:yes' parameter:

adobereader - SchTasks /Delete /TN "Adobe Acrobat Update Task" /F
flashplayerplugin - SchTasks /Delete /TN "Adobe Flash Player Updater" /F
ccleaner - SchTasks /Delete /TN "CCleaner Update" /F
dropbox - SchTasks /Delete /TN "DropboxUpdateTaskMachineCore" /F
dropbox - SchTasks /Delete /TN "DropboxUpdateTaskMachineUA" /F
(not sure if both are needed for dropbox - I have the first disabled)

The Google updater would need some testing as I show 4 Google scheduled tasks. GoogleUpdateTaskMachineCore and GoogleUpdateTaskMachineUA run from C:\Program Files (x86)\Google\Update\ but there are also two that run in user space where the task name includes a CLSID and the updater runs from C:\Users\username\AppData\Local\Google\Update\

All of this can be implemented at the package level. You just need to voice yourself to the package maintainers that you want the option. Ask nicely, this will take some extra time to research, test, and debug. :)

[tylerszabo]

@vertigo220 if there are specific packages that are giving you issues please feel free to reach out to me by email. I've made a mini mission of mine to get the software I use that auto-updates to play nice with chocolatey (and vice versa). I don't want to take this thread out of scope by getting into particular packages, though.

[SolidKnight]

I would support the option for packages to be auto-pinned by the maintainer. The reasons (this is mostly in regards to corporate scenarios):

1. Some software releases irregularly, frequently, and sometimes with no notification.
2. Some software cannot have their automatic updater turned off so you're always competing with that.
3. Some software will register with a service where the update cycle is controlled there in an admin portal--chocolatey's role in this scenario is just to do the initial install.
4. Some products are a combination of all the above thus making it a nightmare to keep packaging and delivering all versions through chocolatey.

Let the decision and fallout of auto-pinning be my problem. If I could specify that a package will automatically pin itself, it means one less script I have to run to ensure that it's being done and one less potential point of failure.

[bcurran3]

@SolidKnight

In the scenario where an enterprise ONLY uses Chocolatey for initial install, I would recommend deleting the lib\packagename directory and the result is that Chocolatey will no longer manage that package/program.

I do this is my setup scripts, though for a different reason. A few packages I install are user specific and I want the package to be easily run again under a different user account:

choco install desktopicons-winconfig --params "'/AllIcons:YES /RenameComputer:YES'" 
rd %ChocolateyInstall%\lib\desktopicons-winconfig /s /q

Of course if you WANT Chocolatey to at the very least know the package/program is installed, this isn't helpful, but if Chocolatey isn't managing the package and thus doesn't know the correct version of the program installed, that's not very helpful either. :)

This is probably all going to be addressed in the forthcoming Chocolatey Enterprise Console (or whatever its real name is). Chocolatey Staff can now step in and plug away...

[ferventcoder]

Related to #798

[ferventcoder]

I think there is some value in this. Some packages should support the idea of being pinned. I see it as something like a .pin file inside of the Chocolatey package that notes that the package should be pinned upon installation.

[bcurran3]

Bad news, for me.

I implemented my conceptual theory in to what I thought was a proof of concept in chocolatey-misc-helpers.extension v0.0.3's Enable-AutoPin cmdlet. Sadly when I went to use it in a real world scenario I found out that choco.exe deletes the .pin file after chocolateyinstall.ps1 runs if the .pin file didn't exist previously, so retired the function.

Good news, for me - @ferventcoder is on board with implementing the idea natively into choco.exe!

[musm]

+1 for auto pinning packages with auto updaters like dropbox

[vertigo220]

Just wanted to add something I've found in installing apps with chocolatey but then updating them either via self-updaters or by manually downloading and installing them. This may already be common knowledge, but since chocolatey installs in its own location (ProgramData\chocolatey\lib) instead of in Program Files, which the self-updaters and manual installers use, this means that once something is installed by chocolatey then updated externally, the links to the programs have to be updated in order to launch the newer versions vs the chocolatey ones, and then even if chocolatey were used to update them, that new, updated version would no longer be the one being used. So essentially, there is absolutely no inter-operability. I guess I just thought at first that it was simply an issue of chocolatey not realizing it was updated, whereas it's actually an issue of the chocolatey version not being updated.

I'm sure there's a very good reason for using a different install location, but I wonder if giving the option to use the app's standard location would eliminate this issue?

[kmindi]

@vertigo220 That's very interesting, I aslo experienced that and what you described may be the cause (for still starting applications I thought were updated).

[vertigo220]

Another interesting finding: I uninstalled GOG Galaxy in chocolatey, since I've been using its self-updater for multiple versions now, including recently updating to the v2 beta, which doesn't even show as being available in chocolatey (due to being a beta, obviously), so I figured that since there was the chocolatey version and the non-chocolatey version, I would just remove the former and leave the latter; however, it uninstalled it completely (the newer, beta, non-chocolately version). I'm not sure how or why chocolatey would affect that. Then I tried another program, this time HxD, and this time the uninstall failed, saying it isn't installed, even though it is (and still working after the uninstall attempt in chocolatey). Both programs have almost empty directories in the chocolatey directory after being externally updated, with neither containing the actual application, so I'm guessing the external update somehow removes them from the chocolatey directory, and then chocolatey inconsistently sometimes follows them to the new directory to uninstall (yet seemingly not for gathering info related to current version) and sometimes doesn't (which inconsistently leads it to both show the application as installed yet says it's not when trying to uninstall it). Does this make sense?

[gep13]

@vertigo220 one of the paid features of Chocolatey attempts to address some of the things that you mention:

https://chocolatey.org/docs/features-synchronize

[vertigo220]

@gep13 Thanks. I think that was mentioned before, but while I don't mind paying for chocolatey if it works well and does what I need (though I do wish there were something like a "home" level at a slightly lower cost, since I don't need the package builder and, probably, the advanced PowerShell features, but it's not that big a deal), since in that case it would be very useful, but I'm not at the point yet I'm sure it does. I'm still debating whether to make the switch. In fact, that's why I'm back right now, because I'm setting up a couple other computers and trying to decide whether to do so with chocolatey or normally. I used chocolatey to do some of the initial setup on my computer a while back, but then due to the issues I had ended up not using it, hence some of my problems mentioned in this issue. I really want to use it, and wouldn't mind paying if it meant it would work as needed, I'm just concerned about getting invested in it and then having it not perform as expected, especially since you said those features "attempt to address" these issues. Also, since this issue is still open, I assume the changes discusses would in fact help, meaning even the paid version may not be enough. What am I missing?

Also, slightly OT, but looking at those additional features, I found the malware protection one curious. I always assumed there was some sort of "built-in" protection already, since I didn't think you guys would want to be "responsible" for people getting infected.

[kmindi]

I have the paid version, yet I am also unsure if that feature (synchronize) works (there is no way to trigger it manually as another caveat), because the versions listed in chocolatey still show the old versions, despite that the programs updated themselves.

[vertigo220]

@gep13 - I don't mean to belabor this issue, and I apologize if it comes across that way, but I'd really like your thoughts on it. As I said, I'm working on setting up a couple computers, and right now I'm held up by this, because I'm trying to decide whether to set them up with Chocolatey or not. I really want to, but I'm worried that if I do, I'll end up with the same situation I'm in now with my computer, even with the paid version, which is that apps will self-update or cause the user to manually update them, and that will result in multiple versions on the same system, which just makes things more complicated, not less.

[soredake]

Any progress on this?

[TheCakeIsNaOH]

I think this is a duplicate of #242, although I like the suggestion of using the pre-existing pinning functionality suggested in this issue.

[TheCakeIsNaOH]

As of Chocolatey CLI v1.2.0, there is a --pin-package option for choco install.
This is not automatic pinning, but it does allow installation and pinning of a package in one step, instead of having to call choco.exe twice.