Namespacing packages

[dbeacham]

Has there been any thought given to namespacing packages, i.e. listed under an organisation or user. For example servant/client and servant/server rather than everything living in a global namespace?

No idea about

1. engineering effort or community pay off
2. if it has been discussed/dismissed before but couldn't immediately find anything here or in cabal repo
3. if this is more a cabal issue

but it could be helpful with

1. grouping similar packages together under some sort of organisation
2. stopping package names being taken by experimental and never updated 0.0.01 releases
3. could/would make forking packages simpler rather than having to take ownership or giving the fork a different name.

[gbaz]

There's been scattered discussion over the years but I don't recall where offhand.

multilib packages seem to cover a number of the cases here, and there's already ad-hoc namespacing-by-prefix conventions that handle grouping.

Technically, it seems like the work that would need to be done would be to enable (3) -- and I'm not sure exactly what that would entail (someone would then need to set up which namespace to pull from, etc., which seems more complicated if anything than something resembling what we have now).

I think that maybe something in cabal.project files or similar that allowed specification of global package overrides (as an extension of the backpack stuff perhaps?) would be the most direct way to get (3), and that doesn't seem to require any technical changes in terms of namespacing.

That is to say, namespacing seems to be 90% a convention issue, not a technical issue, and I'm not sure what technical changes it would make sense to implement to support any particular convention, and of those changes, I think they might be best thought of more generally than involving namespacing anyway...

[chshersh]

Multilib packages don't really replace the grouping of similar packages under the same namespace as versioning is more difficult for them (you can't change version only for a single package, you need to bump versions for all packages even when changing only a single public library).

But I see a huge benefit in lightweight forking of Haskell packages and first-class support of packages by build tools and package repositories. In the end, almost all Haskell packages are run by volunteers in their free time. Volunteers tend to burn out and free time is a limited resource. The Haskell stability group has several proposals to deal with this. But I think that adding a simple way to fork an existing package will enable successors to easily continue bringing improvements to existing projects and users to easily switch to them.

Thus being said, Hackage already kinda allows this as @gbaz mentioned. You can fork a GitHub repository of an existing package and upload it to Hackage under a custom name. For example, I can make my own version of text and call it chshersh-text and publish it to Hackage. I find chshersh/text more aesthetically pleasing than chshersh-text but I also find it a weak reason to change build-depends parsing logic and Hackage routing for the sake of a better-looking name. But it's just a matter of naming conventions.

In the end, I always found the package takeover procedure hostile to package maintainers. I'll gladly ditch the takeover in favour of forking or a better collaboration and stability approach. I'm even okay with ugly package names if such naming practices would be publicly documented on hackage.haskell.org, announced and generally encouraged.

---

There's also Flora Package Index that introduces Haskell packages with namespaces externally to Hackage:

• https://github.com/flora-pm

[gbaz]

I think the issue of allowing easier forking is orthogonal to namespacing. It would be allowing a mechanism external to hackage whereby a user can say "throughout the entire calculation of a build-plan, substitute AAA-with-any-version-bounds by BBB-with-these-version-bounds"

I think this could be done via an extension to cabal.project files.

[hasufell]

I think this could be done via an extension to cabal.project files.

The drawback here is that cabal.project files are not considered when installing from hackage.

Ultimately, what we want here are not individual technical solutions, imo, but a workflow:

Right now:

1. user starts using a library
2. user gets dissatisfied, but can't contact the maintainer
3. user forks the library on github, decides to not put it up on hackage
   • this forces all downstream users to use it via cabal.project
   • now those downstream users can't put their package up on hackage either
   • people start bundling code, just so they can put it up on hackage
4. even if the user puts the package on hackage, it has poor visibility (no one will know it's a fork)

A better workflow would be:

1. user starts using a library
2. user gets dissatisfied, but can't contact the maintainer
3. user clicks a fork button on hackage
   • new package is created with the existing code under their own <user>/<pkg> namespace and will show up under a forks section of the original package
   • hackage provides a callback for automatically forking on github if the original repo is on github
   • the maintainer gets an email or a hackage-internal notification about the fork (they might be interested, notifications can be disabled)
   • other users can easily see the fork and use it directly from hackage without any additional tooling

I think the problem is exactly that we have too much individual tooling solutions, but barely anything built around "workflows".

[phadej]

There is no guarantee that phadej/foo and hasufell/foo are anywhere close. (We have seen lenz package e.g.).

Needing to fork a package is a social problem, switching the ecosystem to use another library is also partly a social problem.

But there are clearly indicators that we are capable to, e.g. test-framework and tasty (and to some degree criterion and its competitors).

As there is no way to ensure that <user1>/<pkg> and <user2>/<pkg> are compatible (what that even means), I don't see a namespacing to be a solution to those problems.

Namespacing is a solution so you can use std and drop std:: from all your names, so if namespacing would be introduced to Hackage, there is a (tiny) risk that there would be somewhat disjoint parts using the same names. Cause people are people, and there are plenty of us for some to not like each other.

[phadej]

... and namespacing is a very heavy feature technically to introduce today. GHC knows and cares about package names, to begin with.

[hasufell]

There is no guarantee that phadej/foo and hasufell/foo are anywhere close.

There doesn't need to be. Github also doesn't make any attempt in figuring out if the code is anywhere close between forks. It just marks the origin of the package: there was an intersection of code at some point in the past. And hackage UI would even be able to show which version this was.

[gbaz]

It sounds like the main motivation here is just visibility on hackage of forks and I don't thank "namespacing" addresses that either. It only addresses letting you say "foo/pkg" instead of "foo-pkg".

Visibility of forks lets another user mark their package on hackage as a fork of mine, which to active-maintainers could feel even more hostile than a fork today, since its turning my package page into free advertising for someone else's package!

"other users can easily see the fork and use it directly from hackage without any additional tooling" already exists today, it just doesn't link to the forks from the original.

What this doesn't address that a cabal.project solution does is if I want to use pervasively in all my dependencies "bytestring-forked" instead of "bytestring". The cabal project solution would essentially be sugar for having a local overlay and with the "substitute" package locally renamed to the "original".

[hasufell]

It sounds like the main motivation here is just visibility on hackage of forks and I don't thank "namespacing" addresses that either. It only addresses letting you say "foo/pkg" instead of "foo-pkg".

Not quite. user/pkg as a special syntax would allow to e.g. list all packages under a specific namespace and only allow that specific user to upload to that namespace. That's basically the same as github.com/hasufell. No one can create repos there, except me.

This is not true for user-pkg syntax and very likely cannot be reasonably enforced retroactively.

Visibility of forks lets another user mark their package on hackage as a fork of mine, which to active-maintainers could feel even more hostile than a fork today, since its turning my package page into free advertising for someone else's package!

That's quite an odd argument, given that github does the same and is one of the most collaborative platforms.

"other users can easily see the fork and use it directly from hackage without any additional tooling" already exists today

I don't think so. Forks today can have any sort of naming scheme, including completely different names.

As soon as we provide a fork button, there will be a network effect, similar to github. Yes, you can manually fork packages with whatever name you please on github as well, but most people use the fork button, which improves visibility.