Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The PDM_OVERRIDE environment variable does not work. #3182

Closed
1 task done
dairiki opened this issue Sep 24, 2024 · 2 comments
Closed
1 task done

The PDM_OVERRIDE environment variable does not work. #3182

dairiki opened this issue Sep 24, 2024 · 2 comments
Labels
🐛 bug Something isn't working

Comments

@dairiki
Copy link

dairiki commented Sep 24, 2024

Synopsis

Using the --override argument to pdm lock to pass a constraint file works, but using the PDM_OVERRIDE environment variable to do the same does not.

  • I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

$ pdm init -n
Creating a pyproject.toml for PDM...
INFO: Using the first matched interpreter.
Virtualenv is created successfully at 
[...]/.venv
Project is initialized successfully

# Need at least one dependency.  (I don't think it matters what it is.)
$ pdm add markupsafe

$ touch constraints.txt

# This works:
$ pdm lock --override constraints.txt
Changes are written to pdm.lock.
  0:00:00 🔒 Lock successful. 

# This does not:
$ PDM_OVERRIDE=constraints.txt pdm lock -v
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   markupsafe>=2.1.5
pdm.termui:   Adding requirement markupsafe>=2.1.5
Traceback (most recent call last):
  File "/home/dairiki/.local/bin/pdm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/core.py", line 351, in main
    return core.main(args or sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/core.py", line 269, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/core.py", line 264, in main
    self.handle(project, options)
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/core.py", line 196, in handle
    command.handle(project, options)
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/cli/commands/lock.py", line 123, in handle
    actions.do_lock(
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/cli/actions.py", line 145, in do_lock
    mapping = _lock_for_env(
              ^^^^^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/cli/actions.py", line 189, in _lock_for_env
    mapping, *_ = resolve(
                  ^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/resolver/core.py", line 43, in resolve
    result = resolver.resolve(requirements, max_rounds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/resolvelib/resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/resolvelib/structs.py", line 127, in __bool__
    next(iter(self))
         ^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/resolvelib/structs.py", line 136, in __iter__
    self._factory() if self._iterable is None else self._iterable
    ^^^^^^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/resolver/providers.py", line 217, in matches_gen
    elif identifier in self.overrides:
                       ^^^^^^^^^^^^^^
  File "/home/dairiki/.pyenv/versions/3.12.3/lib/python3.12/functools.py", line 995, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/resolver/providers.py", line 165, in overrides
    parser.parse_file(override_file)
  File "/home/dairiki/.local/share/pdm/venv/lib/python3.12/site-packages/pdm/formats/requirements.py", line 104, in parse_file
    with open(filename_or_url, encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'c'

Actual behavior

pdm lock craps out if the PDM_OVERRIDE environment variable is set.

Expected behavior

pdm lock should complete successfully, respecting the constraints specified in the file referenced by PDM_OVERRIDE.

Here's the Issue

The click.Optionargument for the --override option is declared with action="append". This means that, if the --override option is specified on the command line, override will be a list of strings.
This conflicts with the type of the default value for the option, os.getenv("PDM_OVERRIDE"), which is just a string (rather than a sequence of strings).

pdm/src/pdm/cli/options.py

Lines 514 to 521 in d579492

override_option = Option(
"--override",
default=os.getenv("PDM_OVERRIDE"),
action="append",
help="Use the constraint file in pip-requirements format for overriding. [env var: PDM_OVERRIDE] "
"This option can be used multiple times. "
"See https://pip.pypa.io/en/stable/user_guide/#constraints-files",
)

PDM interprets override as a sequence of file names of constraint files. When the value comes from PDM_OVERRIDE, interpreting the string as a sequence erroneously results in a series of one-character-long file names.

It's a trivial issue, but I'm unsure of the cleanest fix. Perhaps using click's envvar arg would do the trick? (This would allow the specification of multiple constraint files via PDM_OVERRIDE.)

If you would like, I would be happy to attempt a PR. (But maybe this is simple enough that you'd rather just fix it yourself.) Let me know.

[Edited: Originally, I did not look close enough to realize that the Option class was not a click.Option. ]

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:
  2.18.2
Python Interpreter:
  /home/dairiki/git/web/lektor-packages/lektor/junk/.venv/bin/python (3.12)
Project Root:
  /home/dairiki/git/web/lektor-packages/lektor/junk
Local Packages:
  
{
  "implementation_name": "cpython",
  "implementation_version": "3.12.3",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.8.0-45-generic",
  "platform_system": "Linux",
  "platform_version": "#45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Sep 11 15:25:05 UTC 2",
  "python_full_version": "3.12.6",
  "platform_python_implementation": "CPython",
  "python_version": "3.12",
  "sys_platform": "linux"
}
@dairiki dairiki added the 🐛 bug Something isn't working label Sep 24, 2024
@frostming
Copy link
Collaborator

PR welcome

@frostming
Copy link
Collaborator

Fixed by e95520e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants