Can't install latest cabinetry on top of pyhf master (or pyhf 0.7.0.rc1)

[lhenkelm]

... because cabinetry 0.4.1 pins pyhf as pyhf[minuit]~=0.6.3.
It would be great if this could be relaxed to pyhf[minuit]>=0.6.3,
as the known incompatibilities do not break my usage of cabinetry.
In general, when using cabinetry as a library, I'd rather discover that I need to manually cap pyhf when upgrading if a known incompatibility affects some feature of cabinetry I rely on, than be unable to install my project from scratch.

E.g. when installing all dependencies from scratch and requesting pyhf master, cabinetry 0.4.1, pip gives:

ERROR: Could not find a version that satisfies the requirement pyhf[minuit]~=0.6.3 (from cabinetry) (from versions: 0.0.1, 0.0.3, 0.0.4, 0.0.5, 0.0.8, 0.0.12, 0.0.13, 0.0.14, 0.0.15, 0.0.16, 0.0.17, 0.1.0, 0.1.1, 0.1.2, 0.2.0, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.6.0, 0.6.1, 0.6.2, 0.6.3, 0.7.0rc1)
ERROR: No matching distribution found for pyhf[minuit]~=0.6.3

When upgrading on top of an old environment (from before the pyhf release candidate "renamed" the version pip assigns the head of the pyhf master branch from 0.6.3.devN to 0.7.0.rc2.devN ) it happens to work, but only because the pip dependency resolver does not attempt to fully resolve in the presence of old installations, and so only warns that it ignores cabinetry's requirement:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cabinetry 0.4.1 requires pyhf[minuit]~=0.6.3, but you have pyhf 0.7.0rc2.dev1 which is incompatible.
Successfully installed fonttools-4.34.0 hypothesis-6.49.1 iminuit-2.12.1 pillow-9.2.0 pyhf-0.7.0rc2.dev1 setuptools-63.1.0 typing-extensions-4.3.0 uproot-4.3.3

Related: #301

[alexander-held]

Hi @lhenkelm, sorry for the delay here! I would propose that I merge #301 and simultaneously raise the requirement to pyhf~=0.7.0rc1. That would allow you to use the new cabinetry master with the RC (and with pyhf 0.7 when it releases).

Supporting >=0.6.3 would require some version-specific code branching in cabinetry. We can consider this if there is strong motivation for keeping the requirement a bit more loose.

I think merging the #301 is safer than just relaxing the requirements on master, as that would introduce at least one bug I can spontaneously think of in yield uncertainty calculations due to the API changes that are coming in the next pyhf release.

[lhenkelm]

Thanks for the temporary fix @alexander-held!

You are probably already aware of it, but just in case, here is an excellent post discussing the downsides of pinning dependencies in libraries.

[alexander-held]

Yes, Henry's blog post is great! I view pinning to the pyhf minor version similarly to the hist / boost-histogram example: they are tightly coupled and can be released shortly after each other. pyhf 0.4 and 0.5 did not introduce breaking changes to cabinetry as far as I remember, 0.6 and 0.7 both did (but cabinetry now also uses a lot more of the pyhf API than it did when 0.5 came out). I expect that the next minor pyhf release is very likely going to introduce breaking changes as well.

Please feel free to bring up this topic again if it causes problems in the future.