How to speed up resolving dependencies when installing a package

[gagarine]

Poetry can be slow to install package. The #2094 is about improving the situation on poetry. This one is about what can be done now without changing poetry.

I taught it may be interesting to discuss the different way of improving this using concrete examples.

Doing a poetry add newspaper3k on a fresh project take 1-2 min to solve dependencies. With the cache, it took only 6s.

=> what can I do to make newspaper3k (or another package) resolve dependencies faster on my side?
=> what can I do to make newspaper3k (or another package) resolve dependencies faster on the newspaper3k side (i.e. better dependency declaration)?

[abn]

@gagarine without cache, it would seem poetry add newspaper3k works fairy quick. The first attempt did it under 10s.

Resolving dependencies... (9.7s)

Taking a look at https://pypi.org/project/newspaper3k/ the wheel looks to be in order and has the required metadata to work, and so does https://pypi.org/pypi/newspaper3k/json.

Within the constraints of your issue description, I suspect there is not much else that can be done. If it takes 1-2 minutes for the resolution for this particular package, it might be due to other factors impacting the performance on your environment or due to the other dependencies at play.

As for your questions, it is not clear what you mean by "my side" or "the newspaper3k side". The general suggestions would be;

• Have wheels that declare dependencies correctly.
• Package the right wheels for any python version or platform variations.
• If using an sdist, provide egg-info directory.
• Publish and maintain complete metadata to the PyPI (requires_dist)

Obviously, the project you are adding as well as all it's dependencies need to adhere to the above for optimal performance.

Apart from the cache, I cannot think of any obvious things that can be done to improve dependency resolution performance without a modification in poetry.

As was mentioned in #2094 (comment), this is an area that is being acively worked on. Additionally, the discussion around the improvement of how dependencies are declared and this information is made available to tools like poetry (referenced in #2094 (comment)) has a lot of potential impact on how this experience will be improved.

[gagarine]

Thanks for your help, I start to understand. I'm very familiar to PHP composer and npm.

I taught poetry was similar. It is not because the underlying package format and repositories (packagist vs PyPi) are very different.

I don't understand why newspaper3k take long to install on my side after running poetry cache clear pypi --all.

[abn]

Poetry is not entirely disimilar to composer or npm.

I'd reccomend you take a look at your bandwidth and CPU utilisation during this process and also perhaps try poetry add -vvv.

I am closing this issue as, I believe the original concern has been resolved. You can join the Discord Community if you would like help debugging the issue at hand. Someone there might be able to assist.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.