Web / Intermittent Failures to Import Toga on Load Due to importlib.metadata Error

[rmartin16]

Describe the bug

Pyodide is occasionally failing to load toga in to the python web runtime. Occurs for simple page reloads as well as reloads that bypass the cache (shift+F5).....as long as you reload enough times.

Traceback (most recent call last):
  File "/lib/python3.10/site-packages/toga/__init__.py", line 97, in _package_version
    from setuptools_scm import get_version
ModuleNotFoundError: No module named 'setuptools_scm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 435, in eval_code
    .run(globals, locals)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 304, in run
    coroutine = eval(self.code, globals, locals)
  File "<exec>", line 2, in <module>
  File "/lib/python3.10/runpy.py", line 209, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/lib/python3.10/site-packages/helloworld/__main__.py", line 1, in <module>
    from helloworld.app import main
  File "/lib/python3.10/site-packages/helloworld/app.py", line 4, in <module>
    import toga
  File "/lib/python3.10/site-packages/toga/__init__.py", line 118, in <module>
    __version__ = _package_version(__file__, __name__)
  File "/lib/python3.10/site-packages/toga/__init__.py", line 115, in _package_version
    return importlib_metadata.version(package)
  File "/lib/python3.10/importlib/metadata/__init__.py", line 955, in version
    return distribution(distribution_name).version
  File "/lib/python3.10/importlib/metadata/__init__.py", line 928, in distribution
    return Distribution.from_name(distribution_name)
  File "/lib/python3.10/importlib/metadata/__init__.py", line 518, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for toga-core

Steps to reproduce

briefcase new --no-input && cd helloworld && briefcase run web

Expected behavior

Pyodide should successfully import toga.

Screenshots

No response

Environment

• Operating System: pop os 22.04
• Python version: 3.10.9
• Software versions:
  • Briefcase: 0.3.13.dev28+gea7c825c
  • Toga: 0.3.0
  • Chrome: 109.0.5414.119
  • Firefox: 109.0.1

Logs

Chrome console

logger.ts:39 [pyscript/main] searching for <py-config>
logger.ts:39 [py-config] loading  /pyscript.toml
logger.ts:39 [pyscript/main] config loaded:
{
  "name": "Hello World",
  "description": "My first application",
  "version": "0.0.1",
  "type": "app",
  "schema_version": 1,
  "runtimes": [
    {
      "src": "https://cdn.jsdelivr.net/pyodide/v0.21.3/full/pyodide.js",
      "name": "pyodide-0.21.3",
      "lang": "python"
    }
  ],
  "packages": [
    "/static/wheels/helloworld-0.0.1-py3-none-any.whl",
    "/static/wheels/toga_core-0.3.0-py3-none-any.whl",
    "/static/wheels/toga_web-0.3.0-py3-none-any.whl",
    "/static/wheels/travertino-0.1.3-py3-none-any.whl"
  ],
  "fetch": [],
  "plugins": [],
  "terminal": false,
  "splashscreen": {
    "autoclose": true
  },
  "pyscript": {
    "version": "2022.12.1",
    "time": "2023-02-09T17:33:28.691Z"
  }
}
logger.ts:39 [py-splashscreen] add py-splashscreen
logger.ts:39 [pyscript/main] Initializing runtime
logger.ts:39 [pyscript/pyodide] Runtime config: Object
logger.ts:39 [pyscript/main] Downloading pyodide-0.21.3...
logger.ts:39 [pyscript/main] Python startup...
logger.ts:39 [pyscript/pyodide] Loading pyodide
stdio.ts:37 Python initialization complete
logger.ts:39 [pyscript/pyodide] Found packages in configuration to install. Loading micropip...
logger.ts:39 [pyscript/pyodide] pyodide.loadPackage: micropip
logger.ts:39 [pyscript/pyodide] Loading micropip, pyparsing, packaging, distutils
logger.ts:39 [pyscript/pyodide] Loaded pyparsing, packaging, distutils, micropip
logger.ts:39 [pyscript/pyodide] pyodide loaded and initialized
logger.ts:39 [pyscript/main] Python ready!
logger.ts:39 [pyscript/main] Setting up virtual environment...
logger.ts:39 [pyscript/main] importing pyscript
stdio.ts:41 /lib/python3.10/site-packages/pyodide/__init__.py:74: FutureWarning: pyodide.create_proxy has been moved to pyodide.ffi.create_proxy Accessing it through the pyodide module is deprecated.
stdio.ts:41   warn(
logger.ts:39 [pyscript/main] Packages to install:  Array(4)
logger.ts:39 [pyscript/pyodide] micropip install /static/wheels/helloworld-0.0.1-py3-none-any.whl,/static/wheels/toga_core-0.3.0-py3-none-any.whl,/static/wheels/toga_web-0.3.0-py3-none-any.whl,/static/wheels/travertino-0.1.3-py3-none-any.whl
logger.ts:39 [pyscript/main] Paths to fetch:  Array(0)
logger.ts:39 [pyscript/main] All paths fetched
logger.ts:39 [pyscript/main] Python plugins to fetch:  Array(0)
logger.ts:39 [pyscript/main] All plugins fetched
logger.ts:39 [py-script] py-mount: found 0 elements
logger.ts:39 [pyscript/main] Executing <py-script> tags...
logger.ts:39 [pyscript/main] Initializing web components...
logger.ts:39 [pyscript/main] Startup complete
logger.ts:39 [py-splashscreen] Closing
logger.ts:39 [pyscript/main] PyScript page fully initialized
logger.ts:39 [pyexec] Python exception:
Traceback (most recent call last):
  File "/lib/python3.10/site-packages/toga/__init__.py", line 97, in _package_version
    from setuptools_scm import get_version
ModuleNotFoundError: No module named 'setuptools_scm'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 435, in eval_code
    .run(globals, locals)
  File "/lib/python3.10/site-packages/_pyodide/_base.py", line 304, in run
    coroutine = eval(self.code, globals, locals)
  File "<exec>", line 2, in <module>
  File "/lib/python3.10/runpy.py", line 209, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/lib/python3.10/site-packages/helloworld/__main__.py", line 1, in <module>
    from helloworld.app import main
  File "/lib/python3.10/site-packages/helloworld/app.py", line 4, in <module>
    import toga
  File "/lib/python3.10/site-packages/toga/__init__.py", line 118, in <module>
    __version__ = _package_version(__file__, __name__)
  File "/lib/python3.10/site-packages/toga/__init__.py", line 115, in _package_version
    return importlib_metadata.version(package)
  File "/lib/python3.10/importlib/metadata/__init__.py", line 955, in version
    return distribution(distribution_name).version
  File "/lib/python3.10/importlib/metadata/__init__.py", line 928, in distribution
    return Distribution.from_name(distribution_name)
  File "/lib/python3.10/importlib/metadata/__init__.py", line 518, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for toga-core

fn @ logger.ts:39

Briefcase run Log

Date/Time:       2023-02-09 12:35:13 
Command line:    /home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/briefcase run web --log

OS Release:      Linux 6.0.12-76060006-generic
OS Version:      #202212290932~1674139725~22.04~ca93ccf SMP PREEMPT_DYNAMIC Thu J
Architecture:    x86_64
Platform:        Linux-6.0.12-76060006-generic-x86_64-with-glibc2.35

Python exe:      /home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/python
Python version:  3.10.9 (main, Jan 27 2023, 17:43:38) [GCC 11.3.0]
Virtual env:     True
Conda env:       False

Briefcase:       0.3.13.dev28+gea7c825c
Target platform: web
Target format:   static

Environment Variables:
    COLORTERM=truecolor
    CONDA_DEFAULT_ENV=base
    CONDA_EXE=/home/russell/anaconda3/bin/conda
    CONDA_PREFIX=/home/russell/anaconda3
    CONDA_PROMPT_MODIFIER=(base) 
    CONDA_PYTHON_EXE=/home/russell/anaconda3/bin/python
    CONDA_SHLVL=1
    DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
    DEFAULTS_PATH=/usr/share/gconf/pop.default.path
    DESKTOP_SESSION=pop
    DISPLAY=:1
    GDMSESSION=pop
    GIO_LAUNCHED_DESKTOP_FILE_PID=76269
    GNOME_DESKTOP_SESSION_ID=this-is-deprecated
    GNOME_SHELL_SESSION_MODE=pop
    GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
    GTK_IM_MODULE=ibus
    GTK_MODULES=gail:atk-bridge
    HOME=/home/russell
    IBUS_DISABLE_SNOOPER=1
    INVOCATION_ID=a4683b689d9b4082a831e7afd5216a1c
    JOURNAL_STREAM=8:66848
    LANG=en_US.UTF-8
    LESSCLOSE=/usr/bin/lesspipe %s %s
    LESSOPEN=| /usr/bin/lesspipe %s
    LIBVIRT_DEFAULT_URI=qemu:///system
    LOGNAME=russell
    LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
    MANAGERPID=3899
    MANDATORY_PATH=/usr/share/gconf/pop.mandatory.path
    OLDPWD=/home/russell/tmp/beeware
    PATH=/home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin:/home/russell/.pyenv/plugins/pyenv-virtualenv/shims:/home/russell/.pyenv/shims:/home/russell/anaconda3/bin:/home/russell/anaconda3/condabin:/home/russell/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/home/russell/.local/share/JetBrains/Toolbox/scripts
    PS1=
\[�[1;36m\]~/tmp/beeware\[�[0m\] via \[�[1;33m\]🐍 \[�[0m\]\[�[1;33m\]\[�[0m\]\[�[1;33m\]v3.10.9\[�[0m\]\[�[1;33m\] \[�[0m\]\[�[1;33m\](\[�[0m\]\[�[1;33m\]venv-3.10-briefcase\[�[0m\]\[�[1;33m\]) \[�[0m\]
\[�[1;32m\]❯\[�[0m\] 
    PWD=/home/russell/tmp/beeware/helloworld
    PYENV_ROOT=/home/russell/.pyenv
    PYENV_SHELL=bash
    PYENV_VIRTUALENV_INIT=1
    QT_ACCESSIBILITY=1
    QT_IM_MODULE=ibus
    SESSION_MANAGER=local/jupiter:@/tmp/.ICE-unix/4173,unix/jupiter:/tmp/.ICE-unix/4173
    SHELL=/bin/bash
    SHLVL=1
    SSH_AGENT_LAUNCHER=gnome-keyring
    SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
    STARSHIP_SESSION_KEY=********************
    STARSHIP_SHELL=bash
    SYSTEMD_EXEC_PID=4343
    TERM=xterm-256color
    TERMINATOR_DBUS_NAME=net.tenshu.Terminator23558193cd9818af7fe4d2c2f5bd9d00f
    TERMINATOR_DBUS_PATH=/net/tenshu/Terminator2
    TERMINATOR_UUID=urn:uuid:c001956e-61ef-4b29-ab9d-54fa180634df
    USER=russell
    USERNAME=russell
    VIRTUAL_ENV=/home/russell/github/beeware/briefcase/venv-3.10-briefcase
    VIRTUAL_ENV_PROMPT=(venv-3.10-briefcase) 
    VTE_VERSION=6800
    WINDOWPATH=2
    XAUTHORITY=/run/user/1000/gdm/Xauthority
    XDG_CONFIG_DIRS=/etc/xdg/xdg-pop:/etc/xdg
    XDG_CURRENT_DESKTOP=pop:GNOME
    XDG_DATA_DIRS=/usr/share/pop:/usr/share/gnome:/home/russell/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share/:/usr/share/:/var/lib/snapd/desktop
    XDG_MENU_PREFIX=gnome-
    XDG_RUNTIME_DIR=/run/user/1000
    XDG_SESSION_CLASS=user
    XDG_SESSION_DESKTOP=pop
    XDG_SESSION_TYPE=x11
    XMODIFIERS=@im=ibus
    _=/home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/briefcase
    _CE_CONDA=
    _CE_M=

Briefcase Log:
[12:35:03]                                                                                                                                                             create.py:750
           [helloworld] Generating application template...                                                                                                             create.py:750
           Using app template: https://github.com/beeware/briefcase-web-static-template.git, branch v0.3.13                                                            create.py:249
           Template branch v0.3.13 not found; falling back to development template                                                                                     create.py:264
[12:35:04] Using existing template (sha 6caadc569a43b1e5ebb4c4d22b2e3e684feedd2f, updated Thu Feb  9 10:00:53 2023)                                                      base.py:704
                                                                                                                                                                       create.py:753
           [helloworld] Installing support package...                                                                                                                  create.py:753
           No support package required.                                                                                                                                create.py:301
                                                                                                                                                                       create.py:760
           [helloworld] Installing application code...                                                                                                                 create.py:760
           Installing src/helloworld... done                                                                                                                           create.py:541
                                                                                                                                                                       create.py:763
           [helloworld] Installing requirements...                                                                                                                     create.py:763
           Writing requirements file... done                                                                                                                           create.py:395
                                                                                                                                                                       create.py:766
           [helloworld] Installing application resources...                                                                                                            create.py:766
           Unable to find src/helloworld/resources/helloworld-32.png for 32px application icon; using default                                                          create.py:630
                                                                                                                                                                       create.py:769
           [helloworld] Removing unneeded app content...                                                                                                               create.py:769
           No app content clean up required.                                                                                                                           create.py:725
                                                                                                                                                                       create.py:772
           [helloworld] Created web/static/Hello World                                                                                                                 create.py:772
                                                                                                                                                                       static.py:118
           [helloworld] Building web project...                                                                                                                        static.py:118
           Removing old wheels... done                                                                                                                                 static.py:121
                                                                                                                                                                   subprocess.py:663
           >>> Running Command:                                                                                                                                    subprocess.py:664
           >>>     /home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/python -u -m wheel pack '/home/russell/tmp/beeware/helloworld/web/static/Hello   subprocess.py:665
           World/app' --dest-dir '/home/russell/tmp/beeware/helloworld/web/static/Hello World/www/static/wheels'                                                                    
           >>> Working Directory:                                                                                                                                  subprocess.py:672
           >>>     /home/russell/tmp/beeware/helloworld                                                                                                            subprocess.py:673
[12:35:05] Repacking wheel as /home/russell/tmp/beeware/helloworld/web/static/Hello World/www/static/wheels/helloworld-0.0.1-py3-none-any.whl...OK                 subprocess.py:637
           >>> Return code: 0                                                                                                                                      subprocess.py:700
           Building app wheel... done                                                                                                                                  static.py:126
                                                                                                                                                                   subprocess.py:663
           >>> Running Command:                                                                                                                                    subprocess.py:664
           >>>     /home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/python -u -m pip wheel --wheel-dir                                               subprocess.py:665
           '/home/russell/tmp/beeware/helloworld/web/static/Hello World/www/static/wheels' -r '/home/russell/tmp/beeware/helloworld/web/static/Hello                                
           World/requirements.txt'                                                                                                                                                  
           >>> Working Directory:                                                                                                                                  subprocess.py:672
           >>>     /home/russell/tmp/beeware/helloworld                                                                                                            subprocess.py:673
[12:35:06] Collecting toga-web~=0.3.0                                                                                                                              subprocess.py:637
             Using cached toga_web-0.3.0-py3-none-any.whl (11 kB)                                                                                                  subprocess.py:637
           Collecting toga-core==0.3.0                                                                                                                             subprocess.py:637
             Using cached toga_core-0.3.0-py3-none-any.whl (499 kB)                                                                                                subprocess.py:637
           Collecting travertino>=0.1.3                                                                                                                            subprocess.py:637
             Using cached travertino-0.1.3-py3-none-any.whl (15 kB)                                                                                                subprocess.py:637
           Saved ./web/static/Hello World/www/static/wheels/toga_web-0.3.0-py3-none-any.whl                                                                        subprocess.py:637
           Saved ./web/static/Hello World/www/static/wheels/toga_core-0.3.0-py3-none-any.whl                                                                       subprocess.py:637
           Saved ./web/static/Hello World/www/static/wheels/travertino-0.1.3-py3-none-any.whl                                                                      subprocess.py:637
                                                                                                                                                                   subprocess.py:637
           [notice] A new release of pip available: 22.3.1 -> 23.0                                                                                                 subprocess.py:637
           [notice] To update, run: pip install --upgrade pip                                                                                                      subprocess.py:637
[12:35:07] >>> Return code: 0                                                                                                                                      subprocess.py:700
           Installing wheels for requirements... done                                                                                                                  static.py:146
           Writing Pyscript configuration file... done                                                                                                                 static.py:167
           Compile static web content from wheels                                                                                                                      static.py:184
             Processing helloworld-0.0.1-py3-none-any.whl...                                                                                                           static.py:197
             Processing toga_core-0.3.0-py3-none-any.whl...                                                                                                            static.py:197
             Processing toga_web-0.3.0-py3-none-any.whl...                                                                                                             static.py:197
               Found toga_web/static/toga.css                                                                                                                          static.py:103
             Processing travertino-0.1.3-py3-none-any.whl...                                                                                                           static.py:197
           Compiling static web content from wheels... done                                                                                                            static.py:185
                                                                                                                                                                         build.py:73
           [helloworld] Built web/static/Hello World/www/index.html                                                                                                      build.py:73
                                                                                                                                                                       static.py:273
           [helloworld] Starting web server...                                                                                                                         static.py:273
           Web server open on http://127.0.0.1:8080                                                                                                                    static.py:289
                                                                                                                                                                       static.py:294
           [helloworld] Web server log output (type CTRL-C to stop log)...                                                                                             static.py:294
           ===========================================================================                                                                                 static.py:298
[12:35:13] Shutting down server... done                                                                                                                                static.py:331

Additional context

No response

[freakboy3742]

Thanks for formalizing this; I've been trying to narrow this down to a more specific test case so I can report it upstream.

FWIW, the workaround is to explicitly set the formal_name and app_id arguments in the constructor of the App instance (in app.py::main()). If they're explicitly provided, they override the importlib approach, and the app will start.

As best as I can make out, it's a race condition on startup; the app will sometimes start, but not reliably (and in my experience, it crashes much more often than it succeeds). However, the problem is deep in the pyodide part of the stack.

[rmartin16]

Ohh....since the error was looking different in the browser for me, I failed to put together this was the previous race condition issue already identified. Thanks for reminding me, though.

[freakboy3742]

From pyscript/pyscript#1204: This is due to a bug in Pyodide; you can work around it by putting the following at the bottom of the pyscript.toml file generated by Briefcase.

[[runtimes]]
src = "https://cdn.jsdelivr.net/pyodide/v0.22.1/full/pyodide.js"
name = "Python runtime"
lang = "python"

Longer term, we will likely need to have the ability to customise the specific runtime (both the Pyodide version and the PyScript version); however, we probably want to hold off on this until the configuration options for a micropython-based Pyscript are clear, as this will likely be a common use case.