PR: Create UpdateManager plugin to handle updates for the conda-based installers

[mrclary]

Description of Changes

General Changes

• The existing update mechanism is removed from the Application plugin to its own UpdateManager plugin. This allows a simplified paradigm where the plugin container can take the role of signal dispatcher between the main UpdateManagerWidget and the UpdateManagerStatus status bar widget.
• A preference pane was created for the Update Manager that contains "Check for updates at startup" and "Check for stable releases only" preferences. The check at startup preference was removed from the "Application" preference pane.
• The update urls and messaging are essentially unchanged from the recent 5.x PR (PR: Improve mechanism that checks for updates and message that informs about them #21402, PR: Use requests instead of urllib for updates #21423, and PR: Follow-up to using the requests library for updates #21433).

Update Flow

• Upon commencing with the install, Spyder will quit and the installation/update process will proceed.
• If the update downloaded a conda-based installer, an interactive graphical uninstaller will launch (Windows) or an interactive command-line uninstaller is executed at a shell terminal (macOS, Linux). For all platforms, the interactive graphical installer is launched upon successful completion of the uninstall process.
• If the update did not require downloading a conda-based installer, a shell terminal will inform the user that Spyder is being updated via conda, and conda's standard output is displayed.
• In all scenarios, Spyder will start/restart upon successful install/update.
• For current versions that are not conda-based installs, only downloading the conda-based installer will result in installing an update. If the user rejects this option, the user is only instructed how to perform the update, but no automatic update will be peformed.

Class Mermaid Map

classDiagram
  class ProgressDialog { }
  class UpdateManager { }
  class UpdateManagerConfigPage { }
  class UpdateManagerContainer { }
  class UpdateManagerStatus { }
  class UpdateManagerWidget { }
  class WorkerDownloadInstaller { }
  class WorkerUpdate { }
  UpdateManagerConfigPage --* UpdateManager : CONF_WIDGET_CLASS
  UpdateManagerContainer --* UpdateManager : CONTAINER_CLASS
  UpdateManagerStatus --* UpdateManagerContainer : update_manager_status
  ProgressDialog --* UpdateManagerWidget : progress_dialog
  UpdateManagerWidget --* UpdateManagerContainer : update_manager
  WorkerDownloadInstaller --* UpdateManagerWidget : download_worker
  WorkerUpdate --* UpdateManagerWidget : update_worker

Loading

Issue(s) Resolved

Fixes #20831
Supersedes #20900

Interactive Testing

Interactive testing is most easily performed using the following script. Just copy and save this somewhere and run it from a Spyder developer environment.

interactive_test.py

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import argparse
import logging
from unittest.mock import patch
from spyder.utils.qthelpers import qapplication
from spyder.plugins.updatemanager import workers
from spyder.plugins.updatemanager.widgets import update
from spyder.plugins.updatemanager.widgets.update import UpdateManagerWidget

logging.basicConfig()
_logger = logging.getLogger('spyder.plugins.updatemanager')
_logger.setLevel(10)

parser = argparse.ArgumentParser()
parser.add_argument('-v', '--version', type=str, help="Spyder current version")
parser.add_argument('-s', '--stable-only', action='store_true', default=False,
                    help="Check for stable releases only")
parser.add_argument('-e', '--env', choices=['conda', 'app', 'pip'],
                    default='conda', help='Environment type')
parser.add_argument('-a', '--anaconda', action='store_true', default=False,
                    help="Anaconda pkg installed")
parser.add_argument('-c', '--channel', default='conda-forge',
                    choices=['conda-forge', 'main', 'develop'],
                    help="Spyder conda channel")

cli_opts = parser.parse_args()

if cli_opts.env == 'app':
    is_conda = True
    is_anaconda_pkg = False
    is_conda_based_app = True
    spyder_conda_channel = ('conda-forge',
                            'https://conda.anaconda.org/conda-forge/osx-64')

if cli_opts.env == 'conda':
    is_conda = True
    is_anaconda_pkg = cli_opts.anaconda
    is_conda_based_app = False
    spyder_conda_channel = {
        'conda-forge': ('conda-forge',
                        'https://conda.anaconda.org/conda-forge'),
        'main': ('pkgs/main', 'https://repo.anaconda.com/pkgs/main'),
        'develop': ('pypi', None),  # local install
    }[cli_opts.channel]

if cli_opts.env == 'pip':
    is_conda = False
    is_anaconda_pkg = False
    is_conda_based_app = False
    spyder_conda_channel = (None, None)


def mock_get_conf(self, option, *args, **kwargs):
    if option == 'check_stable_only':
        return cli_opts.stable_only
    else:
        return super(UpdateManagerWidget, self).get_conf(option, *args, **kwargs)


for mod in [workers, update]:
    for attr, new in [
        ('__version__', cli_opts.version),
        ('is_conda_based_app', lambda: is_conda_based_app),
        ('is_anaconda', lambda: is_conda),
    ]:
        patch.object(mod, attr, new=new).start()
patch.object(update, 'is_anaconda_pkg', new=lambda: is_anaconda_pkg).start()
patch.object(workers, 'get_spyder_conda_channel',
             new=lambda: spyder_conda_channel).start()
patch.object(UpdateManagerWidget, 'get_conf', new=mock_get_conf).start()

app = qapplication()
um = UpdateManagerWidget(None)
um.start_check_update()
app.exec()

However, this will only be interactive with the plugin. To test the full interactivity with Spyder, see the interactive instructions in #20900.

[pep8speaks]

Hello @mrclary! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2023-12-15 21:17:21 UTC

[dalthviz]

Checked this locally and not totally sure if I'm doing things correctly but seems like I'm unable to trigger a major release update 🤔 I was able to trigger a minor release update manually though 👍

Steps followed (extrapolating from the steps at #20900 (comment))

1. Install latest 6.0.0a2 (via conda-installer)
2. Update conda-based installation.
   1. Checkout the https://github.com/mrclary/spyder/tree/cbi-updater-plugin branch locally.
   2. Modify the following files in the Spyder repository:
      • spyder/__init__.py. Change the version on line 32 to (5, 4, 4, "dev0").
      • spyder/plugins/updatemanager/widgets/update.py. Change the delay interval on line 197 from 60000 ms to 10000 ms.
      • spyder/plugins/updatemanager/scripts/install.bat. Remove -y from the conda install command on line 66 (.bat) to allow you to cancel a conda update without clobbering your existing install.
   3. Install superqt and qdarkstyle 3.2 over the spyder-runtime env created by the Spyder 6.0.0a2 conda-installer
   4. Run the following command in a terminal/cmd window to update the conda-based Spyder.

      conda run --live-stream -p "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime" python "E:\Acer\Documentos\Spyder\Spyder otros\mrclary\spyder\install_dev_repos.py" --not-editable

Update Flow that partially worked (minor version, check via manual trigger)

1. Start Spyder.
2. The updater should run at Spyder launch (I had to trigger the Check updates... action manually otherwise I got the status bar just displaying Spyder 5.4.4.dev0).
   1. Testing for a minor update, Spyder will be updated in-place using conda mechanisms (worked as described 👍).
      • You should be alerted that Spyder 5.4.5 is available and whether you want to proceed to install it. Click "Yes".
      • Spyder will quit.
      • A terminal/cmd window appears and displays the progress of the conda update. After the environment solve, do not confirm the changes: type "n" and hit return. You do not want to proceed with the conda update.
      • After the terminal/cmd window closes, Spyder will launch.

Update Flow that didn't work (major version - the update available dialog showed me 5.5.0 always)

1. Testing for a major update, a conda-based installer will be downloaded and launched.
   • You should be alerted to Spyder 5.5.0 being available and whether you want to proceed to install it (I had to trigger the Check updates... action manually otherwise I got the status bar just displaying Spyder 5.4.4.dev0 as before). Click "No".
   • Open Preferences->Update Manager and uncheck Check for stable releases only.
   • Click Check for updates... in the Help menu (or clicking over the status bar).
   • You should be alerted that version Spyder 6.0.0a2 is available and whether you want to download and install it (in my tests I always got 5.5.0 🤔 ). Click "Yes". This will initiate downloading the conda-based installer.
   • After the download is complete, a message box will appear asking if you want to proceed with the installation. Click "Yes".
   • Spyder will quit.
   • Spyder's uninstall script will launch interactively in a terminal asking if you wish to uninstall Spyder (macOS or Linux), or a Windows Uninstaller will launch (Windows). Cancel the uninstall operation.
   • After the uninstall terminal/window is closed, the installer should launch. Cancel the installer.

It's possible that I'm missing setting up something (for example I didn't change the updates worker code since seems like the url should point to the GitHub release page which has the alpha release?) so will try to check again

[mrclary]

@dalthviz, thanks for looking at this! I think I found the issue. When getting a list of updates from GitHub, the versions are listed in chronologically released order, rather than release order, so 5.5.0 always shows up last:
['4.2.4', '4.2.5', '5.0.0', '5.0.1', '5.0.2', '5.0.3', '5.0.4', '5.0.5', '5.1.0', '5.1.1', '5.1.2', '5.1.3', '5.1.4', '5.1.5', '5.2.0', '5.2.1', '5.2.2', '5.3.0', '5.3.1', '5.3.2', '5.3.3', '5.4.0', '5.4.1', '5.4.2', '5.4.3', '6.0.0a1', '5.4.4', '5.4.5', '6.0.0a2', '5.5.0']

I'll fix this and push it up.

[mrclary]

@dalthviz, I've pushed up fixes.

[dalthviz]

Checking again seems like once a check for updates is started things are working as expected (at least for the most straightforward flow where you say Yes to everything) 👍 Just in case, tested setting the version without "dev0" but still seems like the check for updates is not being done automatically even if the Check for updates on startup option is enabled 🤔 Couple of screenshots showing how things are looking on Windows with the conda-installers:

Minor release update:

• Launch Spyder and get the update dialog message (still needs to be lauched manually though 🤔):

• Click Yes and Spyder closes and a cmd showing the update process shows up:

• Due to testing setup then you get prompt to continue or not with the installation:

• Spyder is relauched when the cmd closes:

Major release update (with the option to get only stable release unchecked):

• Launch Spyder and get the update dialog message (still needs to be lauched manually though 🤔):

• Click Yes and then the download dialog starts:

• After finishing the install dialog appears:

• Click Yes, Spyder closes and a cmd shows as well as the uninstall installer wizard:

• When the uninstaller closes the installer is launched (regarding this, should we somehow check if Spyder is actually uninstalled and depending on that show the installer or just relaunch Spyder?):

Will try now to test other actions combinations (like for example cancel the download or select to install the update after closing Spyder) and document anything if I manage to see any weird behaviors or issues

[mrclary]

@dalthviz, thanks for following up. I'll investigate why Spyder does not check for updates at startup.

should we somehow check if Spyder is actually uninstalled and depending on that show the installer or just relaunch Spyder?

I agree that this should be done at some point. We can discuss whether we should defer until later or not, depending on how involved such a check would be.

[mrclary]

@dalthviz, I found the issue and pushed up the fix. It was just a typo preventing checking for update at startup.

[dalthviz]

Awesome! Indeed, seems like now the automatic check for updates on startup works 👍

Started checking some other actions combinations and noticed the following when doing a major release update:

• If you click Yes, wait for the installer download to finish, then click After closing and the click over the status bar you get prompted to download again the installer:

Maybe we need to save the downloaded installer somehow and skip that step when clicking the status bar if that was already done?

Also, I think I got to see the cmd with the output for an update to Spyder 5.5.0 after closing Spyder when the update dialog showed me a major update to 6.0.0a2 although that only happened one time 🤔. I will try to see what could be triggering that or if was just a tests setup error on my side

[ccordoba12]

@mrclary, thanks for your work to address my review promptly! I took a quick look at it and things seem fine to me now.

@dalthviz, could you give this PR a new review to check that @mrclary's latest changes didn't break what was already working? Thanks!

[dalthviz]

Giving this a new manual check locally, seems like there is an issue related with the settings (the report error dialog pop-up when unchecking the Check for stable releases only option and clicking the preference dialog Apply or Ok buttons):

Traceback (most recent call last):
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\lib\site-packages\spyder\plugins\preferences\widgets\configdialog.py", line 185, in button_clicked
    configpage.apply_changes()
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\lib\site-packages\spyder\plugins\preferences\widgets\config_widgets.py", line 119, in apply_changes
    self.apply_callback()
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\lib\site-packages\spyder\plugins\preferences\widgets\config_widgets.py", line 154, in <lambda>
    apply_callback=lambda: self._apply_settings_tabs(
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\lib\site-packages\spyder\plugins\preferences\widgets\config_widgets.py", line 193, in _apply_settings_tabs
    self.apply_settings(options)
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\lib\site-packages\spyder\plugins\application\confpage.py", line 286, in apply_settings
    um = self.plugin.get_plugin(Plugins.UpdateManager, error=False)
  File "C:\Users\dalth\AppData\Local\spyder-6\envs\spyder-runtime\lib\site-packages\spyder\api\plugins\new_api.py", line 444, in get_plugin
    raise SpyderAPIError(
spyder.api.exceptions.SpyderAPIError: Plugin "update_manager" not part of REQUIRES or OPTIONAL requirements!

Also, I noticed that there are a couple of ECHO esta desactivado (ECHO is deactivated) lines being shown in the cmd when updating a minor version:

[mrclary]

Thanks @dalthviz. I'll add UpdateManager to the OPTIONAL list in Application plugin.
Regarding ECHO is deactivated, I'm not sure what that is. @echo off is used at the top of the script, but that just disables echoing each command that executes 🤷🏼

[mrclary]

Also, I noticed that there are a couple of ECHO esta desactivado (ECHO is deactivated) lines being shown in the cmd when updating a minor version:

This should be fixed now also.

[dalthviz]

Checked again locally and seems like everything is working 🎉 so this LGTM 👍

[ccordoba12]

Thanks @mrclary for your work on this!

[mrclary]

Thanks @dalthviz and @ccordoba12 for all the feedback!