pdm takes forever to resolve s3fs==2023.3.0 and boto3

[ianliu]

• I have searched the issue tracker and believe that this is not a duplicate.

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

PDM is taking forever to resolve the dependencies s3fs==2023.3.0 and boto3.

• Solving with pip takes 60 seconds with caching disabled.
• I left pdm solving for more than 5 minutes and it still didn't finish.

The solved dependency is for boto3==1.24.59, which is 129 versions behind the latest version 1.26.84.

Below I compare the logs of pip and pdm while solving the dependency. The logs are piped to ts -s, which prepends the elapsed time for each output log line.

Resolving with pip

$ virtualenv venv
$ ./venv/bin/python -m pip install --no-cache-dir s3fs==2023.3.0 boto3 | ts -s | tee pip.log

pip.log

Resolving with pdm

This test was cancelled after a minute of pdm trying to solve the dependencies. From the logs, one can see that pdm is still analysing boto3 1.26.74, which are the first 11 versions. So, what took pip to analyse 129 boto3 versions, pdm analysed only 11. Extrapolating, this means pdm would take 10 minutes to solve this dependency.

$ pdm init -n
$ pdm add -v 's3fs==2023.3.0' boto3 | ts -s | tee pdm.log

pdm.log

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:
  2.4.6
Python Interpreter:
  /home/ian/src/pypkgs-tester/pdm_test/.venv/bin/python (3.10)
Project Root:
  /home/ian/src/pypkgs-tester/pdm_test
Project Packages:
  None

PDM 2.4.6 is installed, while 2.4.7 is available.
Please run `pdm self update` to upgrade.
Run `pdm config check_update false` to disable the check.
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.9",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.1.12-arch1-1",
  "platform_system": "Linux",
  "platform_version": "#1 SMP PREEMPT_DYNAMIC Tue, 14 Feb 2023 22:08:08 +0000",
  "python_full_version": "3.10.9",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "linux"
}

PDM 2.4.6 is installed, while 2.4.7 is available.
Please run `pdm self update` to upgrade.
Run `pdm config check_update false` to disable the check.

[frostming]

This can be resolved by sarugaku/resolvelib#113 which is merged recently but not released yet

[frostming]

It turns out PDM uses a different preference on which package should be resolved first and walks a longer path(with more failed trials) than pip. But nothing works wrong.

[ai-nikolai]

@frostming I am encounter this issue in 2024 again. It is now running for over 30 minutes.

How can we overcome this issue?