Questions about poetry - features and future enhancements

[lwasser]

Hi there @radoering / poetry maintainers! i'm working on a packaging guide for pyOpenSci - an organization that supports diverse community around scientific open source software. I'm trying to clarify what the packaging landscape looks like through a packaging guide. We also will use this as a resource to support our peer review of open source tools.

I have a few specific questions that i just wanted some clarification around regarding poetry features. I want to represent it correctly in our guide! I also mentioned i'll ping you when i have a new cleaned up PR to review to ensure i get this right.

questions are as follows:

1. Poetry now adds dependencies to the pyproject.toml file a bit differently than the standards advise us to. I had heard that you might be adjusting this to follow standards in a near future release as that PEP didn't exist when poetry was created. Is that true?
2. Poetry's support for more complex (non pure python builds) exists but is not documented. And in this issue at the top, it does mention it's not supported BUT that was from 2020. Is this a supported feature or a TO BE supported feature in the future? Many thanks for any clarification here. I am trying to divide tools by which ones are pure python vs which support the complex builds that our scientific ecosystem often runs into. there is a place for both in our ecosystem I just am unsure of which category to place poetry!
3. Can poetry support other build back ends? PDM does. Hatch right now does not. Specifically i've been playing with meson-python and PDM as it's a tool being adopted by many of the core scientific ecosystem packages right now.
4. Upper bounds pinning ^. I know this is a controversial topic but i wanted to ask here. Many in our community have strong feelings about upward bounds causing build issues given many packages don't follow strict SEMVER and some use CALVER. I just wondered if there is any plans to accommodate a user who wants to not use upward bounds ^ pinning?

Many thanks for the work that you do here. i am sorting through a lot of feedback on tools and figure the best source is from the maintainers. our goal is to make packaging ecosystem a bit easier to navigate for beginners. I feel like you have worked towards that with with your excellent documentation already! I think the upper bounds pinning is the one point of challenging element for some folks.

[Secrus]

ad 1. Yes, this is being worked on. However, there is a lot of code to be changed before this happens. But it will be done.
ad 2. This is a non-trivial issue to handle, not only in Poetry but in the packaging ecosystem overall. Non-pure Python builds are currently based on the old approach made popular by distutils and setuptools, which is the setup.py script. There is some work being done by people from PyPA to standardize the approach to language extension modules, but it's nowhere near being complete ATM. If some solution is made one day, especially if it is PEP-based, Poetry will implement it.
ad 3. It does not and there is currently no plan to support other backends since there are more important issues for us. Unless there is some real need, I doubt we will be making it a thing.
ad 4. This is a default choice, but we support other specifiers, like the rest of the packaging ecosystem. You can see this in Poetry's own pyproject.toml, where we use dependencies that are calver based. The custom specifier ^ is probably going to be dropped when we adapt to PEP 621 (your first question)

[lwasser]

Thank you so so much! Ok a few points of clarification.

1. related to upper bounds pinning / custom specifier ^ - this has really been a touchy point for folks in the community so i'm super glad to hear that it may be dropped and also to see that users can adapt to calver implementations. Regarding the poetry workflow, given this is the default choice NOW, do you suggest that we mention that users can simply remove the custom specifier ^ manually in their pyproject.toml as a work around?

or is there some other way to work with poetry and avoid that upper bound specifier being applied? The guidebook that i'm working on is targeted at beginners.

2. I know this is a tremendous amount of work to support PEP 621 and also remove the custom specifier as default!! Do you think your timeline is 6 months to a year out for this work? i am going to add note in our guide that this work is happening and will make a note to myself to go back and update the guide in the future as well!

again many many thanks!! your responses are super helpful. I understand that the packaging ecosystem is complex to navigate. But am also happy to hear that Poetry is following the lead of work being done by PyPA, etc as well!!

[Secrus]

1. If you feel like it's big trouble for people, you can mention it. We have this info in our documentation as well.
2. We don't want to give any kind of timeline for that, as we are all volunteers and work on the project in our free time and that comes and goes. It will be available when it's done.

[lwasser]

understood!! thank you again for the speedy replies to my questions! and for your volunteer time on this widely used project!
i'll leave things open ended but also will link to your docs around dependency pinning!! my questions are answered!

when i have a new version of this overview i'll share / ping you in case you (or anyone here) would like to review to ensure you are happy with the overview of poetry! cheers!