Using template as a key in the cookiecutter context has special meaning in versions >=2.2.0

[rmartin16]

Workaround

Until Briefcase v0.3.15 is released, run this command after Briefcase is installed:

python -m pip install cookiecutter==2.1.1

Impact

The briefcase new command is unusable with cookiecutter>=2.2.0.

Describe the bug

The release of cookiecutter 2.2.0 (and 2.2.1) introduces special meaning for template in a cookiecutter context. This was to add support for nested templates in cookiecutter/cookiecutter#1770.

Briefcase is currently passing a URL or filepath here from briefcase new so the source of the template is included in pyproject.toml of the rolled out project.

Example trace from running briefcase new:

Traceback (most recent call last):
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/bin/briefcase", line 8, in <module>
    sys.exit(main())
  File "/home/russell/github/beeware/briefcase/src/briefcase/__main__.py", line 25, in main
    command(**options)
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/new.py", line 531, in __call__
    return self.new_app(
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/new.py", line 497, in new_app
    self.generate_template(
  File "/home/russell/github/beeware/briefcase/src/briefcase/commands/base.py", line 891, in generate_template
    self.tools.cookiecutter(
  File "/home/russell/github/beeware/briefcase/venv-3.10-briefcase/lib/python3.10/site-packages/cookiecutter/main.py", line 116, in cookiecutter
    ).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

Steps to reproduce

1. Install latest cookiecutter: python -m pip install -U cookiecutter==2.2.0
2. Run briefcase new --no-input

Expected behavior

The template is rolled out properly without an error.

Screenshots

No response

Environment

• Operating System: pop os 22.04
• Python version: 3.10.12
• Software versions:
  • Briefcase: 0.3.14 and 0.3.15.dev385+g5ed30106.d20230706

Logs

0.3.14
briefcase.2023_07_06-13_37_09.new.log

0.3.15.dev385+g5ed30106.d20230706
briefcase.2023_07_06-13_39_17.new.log

Additional context

It isn't at least immediately clear to me if cookiecutter considered existing uses of template in a context; however, I am also not particularly sure if Briefcase's use of cookiecutter is especially different from intended use cases.

Nonetheless, it doesn't look like Briefcase's use of template in context should be accommodated in cookiecutter....and instead, we should just change template to template_source or something.

This also means any new installation of Briefcase will encounter this exception. A new release is probably necessary to mitigate this issue.

[rmartin16]

This issue provides some support for the idea that Briefcase should use a dedicated virtual environment for projects so we can lock down the versions for dependencies for Briefcase itself.

[freakboy3742]

Agreed that using template_source is probably the best approach here.

Also agreed that we're overdue for a release. I was hoping to do one next week, as I'm about to head to EuroPython to talk about Briefcase, so it would be good to clear the backlog of features that have accumulated into production.

Using a dedicated venv for projects is an interesting idea; the problem we'll hit there is that not everyone agrees that venv is the right way to build a project, and we could end up in a situation where we have an conda base environment creating an environment where Briefcase is installed creating a venv that is the runtime environment. That said, I guess it would make the "installed package X but my app is failing" bugs manifest sooner...

[rmartin16]

Agreed that using template_source is probably the best approach here.

👍🏼 I'll probably look at this tomorrow if you don't knock it out first.

Another thought that occurred to me was we could actually use this new nested template functionality in cookiecutter. Instead of having the separate repos for each template, we could have one repo that contains all the templates. I haven't thought through all the pros and cons....but throwing it out there for consideration.

[freakboy3742]

I haven't looked into the nested template feature yet, so I'm not really across what it could potentially do.

That said: I'm leaning away from the the idea of using git repositories for templates at all, in favor of packing templates as data in Python packages with registered configuration endpoints. We would still manage templates as git repos, but we could consolidate the templates into a single repository (possibly the Briefcase repository?). When we publish a new briefcase version, we can build the binary artefacts and include them in the binary packages. This also allows third parties to develop and install their own templates - so, for example, rather than Briefcase itself needing to maintain PySide templates, PySide can provide a plugin that exposes the Pyside template (or the modifications needed).

That still means using cookiecutter - but perhaps the sub template idea might be a way of pyside providing it's custom app.py without needing to control the rest of the template?

[rmartin16]

FYI. Found another backwards incompatible change. Putting this here until there's time to address this properly....

Previously, when a True/False value was passed in to cookiecutter via context, it would be exposed in the template generating code as a string....now it's the actual symbol True and False.

This changes how I was handling using the brutus user or not in Docker files.

It'll also probably affect how use_full_install_path works for Windows templates.

• Added: Boolean variable support in JSON cookiecutter/cookiecutter#1626

sigh

[rmartin16]

To work around this until Briefcase v0.3.15 is released, run this command after installing Briefcase:

python -m pip install cookiecutter==2.1.1