Should a "prefix" be required, even for non-relocatable packages?

[bradking]

Some Linux distros have a "usr-move" initiative such that /lib is a symbolic link to /usr/lib. This can cause trouble for relocatable packages that compute their resource locations relative to the path at which the package file is found.

Consider a package installed in /usr that provides /usr/lib/cps/foo.cps with cps_path=@prefix@/lib/cps. Other fields use @prefix@ with the expectation that it be replaced by /usr, since that is the prefix to which the package is installed. If the package is loaded through the /lib -> /usr/lib cross-prefix symlink, it will be loaded as /lib/cps/foo.cps and @prefix@ will not have /usr. The resulting expansion of other fields will have incorrect paths.

The only way to have a well-defined prefix in the case of a cross-prefix symlink is for the package to not be relocatable, and to simply memorize its prefix. This means there is a trade-off between being relocatable and supporting cross-prefix symlinks. Distros with usr-move symlinks can choose the latter and hard-code the prefix.

One way for the .cps file to hard-code a prefix is by using absolute paths in all fields and not referencing @prefix@ at all. IMO that is unnecessarily restrictive. Furthermore, even if a CPS file does not use @prefix@ anywhere, it is semantically useful for clients to be able to know the prefix of the package.

[bradking]

I propose that the CPS spec be updated to guarantee that clients can always compute the proper installation prefix of a package. This means that at least one of the following fields must always be present:

• cps_path=@prefix@/... for a relocatable package.
• prefix=/absolute/path/to/prefix for a non-relocatable package.

[mwoehlke]

A lot of this is XY problem. The only plausible solution I've yet seen presented to the /lib -> /usr/lib is to not be relocatable. You already described the currently intended way to achieve that.

The real question is in your last sentence; do we need/want "prefix" as an emergent concept from a CPS, even if nothing in CPS itself cares?

The advantage of the proposal (besides that there is always some notion of "prefix") is that tools are assured of having a meaningful substitution for @prefix@. The disadvantages are that determining that substitution is slightly more complicated, and there is no way of ensuring that prefix, if specified, is actually meaningful. (For example, a non-relocatable package that does not use @prefix@ could simply claim prefix=/ in order to be compliant, even though this is almost certainly "wrong".)

I think we agree that with the proposed modification, cps_path and prefix would be mutually exclusive?

[bradking]

If a wrong prefix is specified, I consider that no different than a wrong includes or other resource path. It's a bug in the package that should be fixed by its distributor.

Alternatively, one way to ensure the prefix field is accurate is to require cps_path. Then we can check if cps_path=@prefix@/..., where @prefix@ is replaced with the value of the prefix field, matches the (real)path from which the package is loaded.

[mwoehlke]

We can't require both; one expected use case is "sidecar" CPS files for packages that don't yet ship them, e.g. the .cps in the user's HOME or some such, describing a "system" (i.e. in /usr) package. At best, cps_path in that case would give the "prefix" as where the .cps file lives.

To be clear, "the prefix can be nonsense" isn't a strong objection from me; I'm okay with not knowing it at all. Rather, it seemed worth pointing out the proposal can't actually ensure that a reasonable prefix is known.

[dcbaker]

Could we simply require that tools resolve symlinks when locating cps files?

[bradking]

Okay, for sidecar CPS files we certainly can't include cps_path at all. That's another use case for an absolute prefix. I'm not concerned about buggy .cps files specifying an incorrect prefix.

My main goal here is that the CPS spec ensures that a prefix can always be known from a well-formed .cps file. Actually this brings up the Prefix Determination step that's based on the search prefixes: "if the tool has just completed a search ... the prefix is known from the path which was searched". This will not be the case if the tool is given the path to the .cps file explicitly by a user. IMO that option should be dropped.

All of these use cases can be supported if we require at least one of cps_path or prefix to be present. We could consider an exception in cases that the prefix can be unambiguously deduced by stripping a well-defined relative path suffix from the path to the .cps file (the last step in the Prefix Determination section). However, it may be simplest to just require cps_path or prefix.

[bradking]

require that tools resolve symlinks when locating cps files?

In some cases symlinks are desirable and should be preserved. I've seen environments in which a network file system is mounted in a different physical path on different hosts, but a symlink provides a path name common to all hosts. CMake in particular goes to lengths to avoid resolving such symlinks.

Regardless, symlinks are a xy-problem for this issue. The real issue here is about always knowing the package's prefix.

[mwoehlke]

That said, I am also considering the following updated recommendation for determining the prefix from cps_path:

If ``fullpath`` is the location of the ``.cps`` file,
tools shall attempt prefix resolution
against ``dirname(fullpath)``, at minimum.
It is recommended that, if this fails,
tools also attempt prefix resolution
against ``realpath(dirname(fullpath))``
and ``dirname(realpath(fullpath))``,
where ``realpath(...)`` represents the canonicalized
(that is, with all symlinks fully expanded)
form of its argument.

Rationale: I don't want us to needlessly fall flat on our face if the user has done something silly like symlinking ~/cps to /some/long/prefix/cps and uses that to tell tools where to look. But I also agree we should prefer not resolving symlinks when possible.

[mwoehlke]

Actually this brings up the Prefix Determination step [...]

If one of cps_path or prefix is required, most of that section becomes irrelevant and I would expect it to go away.

[bradking]

Hmm. It's possible, particularly for a sidecar CPS file, that a package's artifacts are scattered across the filesystem with no well-defined prefix. On Windows the artifacts might not even be on the same drive letter, so even prefix=/ doesn't make sense. For such packages it doesn't make sense to require a prefix.

I propose we categorize packages as follows:

• Relocatable prefixed: cps_path=@prefix@/.../cps and no prefix=. @prefix@ is allowed in other fields.
• Non-relocatable prefixed: prefix=/... and optionally cps_path=@prefix@/.../cps. @prefix@ is allowed in other fields.
• Non-relocatable scattered: no prefix= and no cps_path=. @prefix@ is not allowed in any fields.

The scattered category should be documented as a last resort. If a .cps file comes with a package whose deployment models a well-defined installation prefix, that prefix must be recoverable by cps_path and/or prefix.

[mwoehlke]

Non-relocatable scattered: no prefix= and no cps_path=. @prefix@ is not allowed in any fields.

There's no good way to validate this beforehand; you'd have to parse not only the entire initial .cps file, but also all supplemental files, to know that @prefix@ is not used anywhere. In my book, ensuring that you can at least always substitute @prefix@ is by far the more valuable reason to make a change in this area. Besides, this case is not significantly different from the prefix being bogus. (Unless you've changed your mind about that being a concern?)