pdm lock hangs with no further debug output
#1609
Labels
🐛 bug
Something isn't working
Comments
|
It looks as if it might be fetching hashes still, if yes it would be great if there were some output so one knows it is still doing something :) |
|
It is indeed still fetching hashes. That said fetching wheels like: Which brings me back to #1128 -- I think it would be really helpful (speedwise) to parse wheel tags and ignore wheels that do not match the require target pythons, I am downloading hundreds of MB for python versions that will never be used |
|
Weird, PDM progresses really slowly through those files which large (?) waits in between. Is there rate limiting somewhere in PDM? |
|
Ok downloads are stalling here, this is something I'll have to fix for myself. That said it would be nice of pdm could log what it is doing. I will leave this open for you to decide whether logging should get improved. |
frostming
added a commit
that referenced
this issue
Jan 10, 2023
frostming
added a commit
that referenced
this issue
Jan 10, 2023
frostming
added a commit
that referenced
this issue
Jan 10, 2023
|
@frostming thank you for the added logline. What is bugging me now is that my projects still downloads gigabytes of data and I am not sure why. It seems to download all wheel files for the resolved version even though the required python version would not match it. I know you said that having all hashes in #1128 (reply in thread) would be okay. I'd like to ask you if it were possible to reconsider this stance -- as it stands now out of the 2 gigabytes of data downloaded, only a fifth of it or so is actually useful for the project, the rest is for older versions. Even though this occurs only on lock when the cache doesn't exist, this is really wasteful if you ask me. I know that Fastly provides the CDN to PyPI for free basically but I think it would be a good idea to not waste resources we do not need. |
Your observation is different from what is expected because fetching hashes doesn't even need to download any file. Can you find out what files it is downloading and the size of these files? |
|
Yes, it is wasteful, but not too much. Packages often release wheels for Python versions they do support, which in most cases are 2-3 minor versions. Even if your project only supports one minor version, that is a 2/3 waste. Most projects will specify a wider |
|
Hi @frostming, I have now used a minimal pyproject file: and ran pdm from the master branch. It does download files according to your log lines: It seems like the hashes from unearth are not moved over. While |
|
Just to be clear, the lines I am refering to are: |
|
Interesting, any idea what could be causing the difference? I am running with python 3.11 and pdm says: |
|
Oha, I have added a so there are indeed no hashes, adding so at least |
|
Okay @frostming, I am a step further. I do get your result if I comment out those two lines in unearth: It seems to me that response from the json api do not populate |
|
Ok, so I can confirm the following: When the simple index page returns HTML then the URLs look like this: and When the index returns a JSON response then the URL is: and This begs two new questions:
|
frostming
added a commit
that referenced
this issue
Jan 11, 2023
|
Fixed in the main branch |
|
Thanks! I wonder if it would make sense to fix in unearth as well. Ie return one of the hashes for |
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Make sure you run commands with
-vflag before pasting the output.Steps to reproduce
I am running
pdm lock -von this pyproject.toml (https://gitlab.com/rocketduck/csi-plugin-nfs/-/blob/locking-issues/pyproject.toml):Actual behavior
Locking hangs with the following output:
Running strace on the pdm process yields:
Ending the process with CTRL-C:
so it seems to be waiting for something?
Expected behavior
Locking should finish
Environment Information
PDM version: 2.3.4 Python Interpreter: /home/florian/sources/csi-plugin-nfs/.venv/bin/python (3.11) Project Root: /home/florian/sources/csi-plugin-nfs Project Packages: None { "implementation_name": "cpython", "implementation_version": "3.11.1", "os_name": "posix", "platform_machine": "x86_64", "platform_release": "6.0.16-300.fc37.x86_64", "platform_system": "Linux", "platform_version": "#1 SMP PREEMPT_DYNAMIC Sat Dec 31 16:47:53 UTC 2022", "python_full_version": "3.11.1", "platform_python_implementation": "CPython", "python_version": "3.11", "sys_platform": "linux" }The text was updated successfully, but these errors were encountered: