Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Moving towards single installer #212

Closed
tomjaguarpaw opened this issue Sep 23, 2022 · 36 comments
Closed

Moving towards single installer #212

tomjaguarpaw opened this issue Sep 23, 2022 · 36 comments

Comments

@tomjaguarpaw
Copy link
Collaborator

Currently the Downloads page has the following instructions:

  1. Install GHC, cabal-install and haskell-language-server via GHCup
  2. To install Stack, follow the Stack installation guide

It has been remarked elsewhere that this situation is undesirable. It would be much simpler and more friendly to have a big "Get Haskell" button. I'd like to move forwards towards that goal.

The first step is to identify what is the difference between the GHCup and Stack installers. Here is my current best understanding:


Stack installer

  • Can install Stack and GHC
  • By default installs Stack (I'm not sure if it installs GHC upfront or if Stack installs GHC only as required by the user's packages)
  • Doesn't install Cabal or HLS
  • GUI installer on Windows, CLI installer otherwise.
  • Installs GHC from official upstream bindists

GHCup installer

  • Can install GHCup, GHC, Cabal, Stack and HLS
  • By default installs GHCup, GHC and Cabal
  • CLI/TUI installer
  • Installs GHC, Cabal and HLS from upstream bindists
  • Does it install Stack from upstream bindists? If not, which bindist is it using?

There are three requirements to "merge" the functionality of these installers (but I think only a subset is worth doing) into a universal installer:

  1. Ensure that the installer installs Stack from upstream bindists otherwise Stack maintainers might get bug reports from a bindist they don't support (perhaps GHCup already does this, I'm not sure)
  2. Ensure that the installer installs GHC, Cabal and Stack by default
  3. Ensure that the installer has CLI and GUI functionality on Windows

Personally I don't see much value in supporting 3. I think we can achieve a satisfactory universal installer by making sure GHCup achieves 1 and 2. Perhaps it already satisfies 1. Satisfying 2 should be technically easy.

I would be interested in technical feedback on this idea, particularly from @hasufell. If it looks like GHCup can achieve 1 and 2 technically then with a bit of work I think we can achieve a consensus that that is the way forward.


References

@hasufell
Copy link
Contributor

Ensure that the installer installs Stack from upstream bindists otherwise Stack maintainers might get bug reports from a bindist they don't support (perhaps GHCup already does this, I'm not sure)

Not possible at the moment since stack maintainers don't build aarch64 macos binaries at the moment, which GHCup developers build on GHC Gitlab CI.

Ensure that the installer installs GHC, Cabal and Stack by default

This will be done once the stack GHC install hook is integrated with GHCup, see https://gitlab.haskell.org/haskell/ghcup-hs/-/merge_requests/285

@tomjaguarpaw
Copy link
Collaborator Author

Not possible at the moment since stack maintainers don't build aarch64 macos binaries at the moment, which GHCup developers build on GHC Gitlab CI.

I see. So does GHCup use official bindists for architectures other than aarch64?

This will be done once the stack GHC install hook is integrated with GHCup

Nice!

@hasufell
Copy link
Contributor

I see. So does GHCup use official bindists for architectures other than aarch64?

If they work well, otherwise no.

I reserve the right to use whatever bindist works best, whether that's official or not.

Every good distribution channel does that.

@tomjaguarpaw
Copy link
Collaborator Author

That's good, because working well is indeed the point. I foresee no problems as long as there's general agreement about what working well means.

@hasufell
Copy link
Contributor

hasufell commented Oct 8, 2022

I foresee no problems as long as there's general agreement about what working well means.

I don't really understand what you're trying to get at.

@tomjaguarpaw
Copy link
Collaborator Author

What would you like me to clarify further?

@hasufell
Copy link
Contributor

hasufell commented Oct 8, 2022

What would you like me to clarify further?

I'm just saying I don't understand what you're implying or suggesting.

GHCup maintainers have their workflow and principles. These will only change if there are good technical arguments for it.

@tomjaguarpaw
Copy link
Collaborator Author

tomjaguarpaw commented Oct 8, 2022

Perhaps I can rewind a bit. One of my points above was

Ensure that the installer installs Stack from upstream bindists otherwise Stack maintainers might get bug reports from a bindist they don't support (perhaps GHCup already does this, I'm not sure)

I think it would be very unfortunate if users came to the www.haskell.org website, followed its instructions to install Stack, but the bindist of Stack thus installed was an unofficial one containing bugs, because then the users would get a poor impression of Stack, and the maintainers of Stack would receive bug reports about things that are not their responsibility. If that occurred it would risk reopening an old wound that is currently in the process of healing, but not quite yet fully healed. That would be a problem.

Currently there is no risk of the above happening, because the instructions on www.haskell.org recommend using the Stack official installation guide, so anything that happens following those instructions is the responsibility of the Stack maintainers.

If www.haskell.org switches to recommending GHCup as the way to obtain Stack, and Stack is (sometimes) provided by unofficial bindists, then there is a risk of the above happening.

You say that "I reserve the right to use whatever bindist works best, whether that's official or not.". Is your opinion of what bindist works best guaranteed not to cause the above-mentioned problems for Stack and its maintainers?

@hasufell
Copy link
Contributor

hasufell commented Oct 8, 2022

I think it would be very unfortunate if users came to the www.haskell.org website, followed its instructions to install Stack, but the bindist of Stack thus installed was an unofficial one containing bugs, because then the users would get a poor impression of Stack, and the maintainers of Stack would receive bug reports about things that are not their responsibility. If that occurred it would risk reopening an old wound that is currently in the process of healing, but not quite yet fully healed. That would be a problem.

Currently ghcup has fixed bugs in upstream bindists and has never introduced new bugs.

So I'm not sure where your concern comes from.

However, it seems you're assuming that we apply code changes downstream. That has never happened either. I'm hesitant to give hard guarantees about this, but it's unlikely to happen.

Further, I'm a contributor to stack:

@mpilgrem (the current active stack maintainer) and me are in frequent communication. So again: I'm not sure where your concerns stem from.


If you want hard guarantees about how ghcup does distribution, I'm afraid I won't give them. But I think I've clearly demonstrated that ghcup maintainers collaborate successfully with all tools and projects that ghcup supports.

@tomjaguarpaw
Copy link
Collaborator Author

Right, there is currently general agreement about what working well means. As long as that continues I foresee no problems, as I said. I reserve the right to change my mind if general agreement (hypothetically) no longer holds. I don't see any particular reason it will change, I'm just saying it's within the realm of possibility.

It seems to me it's you that was concerned about "what I'm trying to get at, imply, or suggest". I hope I have clarified, but if not please ask and I will clarify further to the best of my ability.

@hasufell
Copy link
Contributor

hasufell commented Oct 8, 2022

Sure. GHCup strives to be a good distribution channel. I'm not a novice in that domain and have over a decade of experience there, including major linux distributions.

Given some of my negative experiences I had (which you know about very well), I also want to make clear that I won't let any committee impose how GHCup executes its distribution or what features it has to provide. GHCup serves the community and strives to maintain trust with the community and the tool maintainers (including stack).

So your concerns are absolutely valid, but the discussion about specific distribution policies here is misplaced imo, because it's ultimately a question of trust.

@tomjaguarpaw
Copy link
Collaborator Author

In my personal and professional life I have been a very satisfied user of GHCup for many years. I know it to be well-maintained and reliable and I am grateful to you for this contribution to the Haskell community. I have never used Stack. Nonetheless, in my position as Haskell.org committee member I have a responsibility to represent the interests of a wide variety of different groups, Stack users and leadership amongst them.

The trust-building efforts you have made with the Stack maintainership are very welcome. On that basis, and the basis of everything discussed here, I am happy to recommend to the committee that GHCup become the sole install tool mentioned on www.haskell.org.

I believe, then, the only remaining component is for Stack to be treated the same as Cabal, with regard to when they are installed by default. Please let me know when that work is completed.

@hasufell
Copy link
Contributor

hasufell commented Oct 9, 2022

Nonetheless, in my position as Haskell.org committee member I have a responsibility to represent the interests of a wide variety of different groups, Stack users and leadership amongst them.

Haskell.org committee only represents Haskell.org, not any users. It is not voted by the users.

As GHCup maintainer I don't represent anyone, but my actions have consequences on more than half of the community, can break Github CI (and even block githubs weekly virtual images creation process... and I have occasional communication with Github staff due to that), break private production CIs, cause miscompilations (when switching or fixing already installed bindists), break HLS, break vscode-haskell, break mirrors in China, etc.

So my main concern are the users, not whether GHCup is featured on the download page.

While I appreciate your very valid concerns, I believe I have sufficient overview of the impact of my actions, including the relationship with stack maintainers and I prefer to handle any upcoming issues with them myself, without any "help" from a third party. My experience tells me that is the right way.

But I do believe you definitely should source opinion from current stack maintainer about what installation method they consider primary and what their opinion on the download page is. I don't know their opinion.


As a more general remark:

While GHCup is hosted on haskell.org and greatly relies on its infrastructure (which I'm very grateful for), I also want to make clear that it is not maintained by haskell.org.

In the long run, I hope that I won't be the only one maintaining this, but that more community members will step up. My health is not good these days and my reaction time has slowed down. I don't have the capacity to work on larger features anymore either, for now.

I believe, then, the only remaining component is for Stack to be treated the same as Cabal, with regard to when they are installed by default. Please let me know when that work is completed.

@hasufell
Copy link
Contributor

hasufell commented Oct 9, 2022

@mpilgrem

Possible impact/complications of recommending stack installation via GHCup:

  1. using stack upgrade is not recommended when using GHCup, because stack replaces the current binary unconditionally, which does not work well with GHCup (e.g. it may replace stack-2.7.5 with something newer). There's a post-install message explaining to use ghcup install stack latest instead.
  2. since we now support and install the new GHC installation hook by default for stack that means a large number of users will install GHC through this hook. This will cause bug reports. Since hooks are supported by stack, that's to be expected. I'm watching the stack issue tracker anyway and will be able to give support, when necessary.
  3. As you're aware, GHCup provides some additional stack binaries like Darwin aarch64, which are not "official".

@tomjaguarpaw
Copy link
Collaborator Author

Nonetheless, in my position as Haskell.org committee member I have a responsibility to represent the interests of a wide variety of different groups, Stack users and leadership amongst them.

Haskell.org committee only represents Haskell.org, not any users. It is not voted by the users.

You're right, I should have said "reflect", or "take into account", not "represent".

@mpilgrem
Copy link
Contributor

mpilgrem commented Oct 9, 2022

I am slow to join this discussion. Starting at the beginning:

Stack installer
Can install Stack and GHC
By default installs Stack (I'm not sure if it installs GHC upfront or if Stack installs GHC only as required by the user's packages)
Doesn't install Cabal or HLS
GUI installer on Windows, CLI installer otherwise.
Installs GHC from official upstream bindists

Stack is not an 'installer', as such - but it does, of course, install things.

Stack can upgrade Stack, in various ways - by downloading a bindist or by compiling from source.

Stack can install GHC, if the needed GHC is missing, and it does that by default. Users can choose to use their own 'system' GHC, if they want and it meets what is needed. From Stack 2.9.1, users can use a script to intervene in Stack's mechanism to get the needed GHC. GHCup makes use of that mechanism.

Stack does not install Cabal (the build tool) or HLS. In many respects, Stack and Cabal (the tool) are alternatives. Stack's focus is 'ease of use' (which, like beauty, is in the eye of the beholder) and it seeks to achieve that, in part, by focussing on curated package sets. Stack and Stackage go hand in hand.

The easiest way (outside of GHCup) to install Stack on Windows is indeed to use the windows-x86_64-installer.exe.

The easiest way (outside of GHCup) to install Stack on Unix-like operating systems is to use the sh script at https://get.haskellstack.org/.

There are, however, lots of ways to get Stack, as set out in Stack's online documentation: https://docs.haskellstack.org/en/stable/install_and_upgrade/. The online documentation mentions that GHCup is one way to get Stack.

Stack looks to a 'setup info' dictionary to know where to get versions of GHC for platforms and Linux distributions with different libraries. The default dictionary is at https://github.com/commercialhaskell/stackage-content/blob/master/stack/stack-setup-2.yaml. In recent years, the dictionary has been populated with links to versions of GHC as published by the GHC developers.

  1. ... Stack maintainers might get bug reports from a bindist they don't support

I would put it slightly differently: Stack does already get bug reports from Stack bindists not published at the Stack repository. We deal with them as best we can. Users posting issues in the Stack respository are asked how they obtained Stack. Stack also reports detailed version information (stack --version).

  1. Ensure that the installer has CLI and GUI functionality on Windows

For better or worse, I am a Windows user myself, so this is close to my own heart - although Windows Subsystem for Linux has made switching to using Linux in general (and Ubuntu in particular) very easy for Windows users. From my perspective, as long as it actually works on Windows, I don't much care if it is via a GUI or at the command line in Windows Terminal. Clear and comprehensive documentation that takes into account the existence of Windows users is often the cure for most things.

@mpilgrem
Copy link
Contributor

mpilgrem commented Oct 9, 2022

Continuing:

stack maintainers don't build aarch64 macos binaries at the moment, which GHCup developers build on GHC Gitlab CI.

The Stack repository is on GitHub and makes use of Github Actions to build bindists of Stack. That seems to limit us to the platforms supported by GitHub runners although there is also a self-hosted runner for ARM64/Linux (I think that is provided by FP Complete). I have seen this 9 August 2022 blog post: https://github.blog/changelog/2022-08-09-github-actions-self-hosted-runners-now-support-apple-m1-hardware/, but I do not know what would be required to have a ARM64/macOS self-hosted runner. So, I would describe the current situation of the Stack repository: as very willing, but not yet able, to support Apple silicon.

@mpilgrem
Copy link
Contributor

mpilgrem commented Oct 9, 2022

Continuing:

@mpilgrem (the current active stack maintainer) and me are in frequent communication.

Julian (@hasufell) is correct that he and I are often in touch. I like to think that we are pulling in the same direction - the direction of making reliable Haskell-related tools easy to obtain and easy to use, once obtained.

I am active on the Stack repository, in my spare time. Michael Snoyman was kind enough to give me commit rights recently, and I've been trying to make good use of them. However, I know my limitations. To manage expectations, I describe myself as a janitor not an architect. Like Julian and GHCup, I am accutely conscious that if I make a bad change to Stack, I could affect a lot of people (before it was reversed/corrected). I tread carefully and make use of the Haskell Foundation Slack #stack-collaborators channel if I think I might be proposing something other people might not agree with. Michael also provides me with advice, from time to time.

Also like Julian and GHCup, I hope more people will be active in helping take Stack forward.

@mpilgrem
Copy link
Contributor

mpilgrem commented Oct 9, 2022

Finally:

Possible impact/complications of recommending stack installation via GHCup: ...

  1. I will add something to Stack's online and in-tool documentation. EDIT: Now done, online and in Stack's master branch.
  2. If an issue arrives at the Stack repository that looks like it is GHCup-related, I will pass it on to the GHCup repository.
  3. I am grateful that GHCup fills holes in the platform support for Stack that the Stack repository cannot itself currently fill. I don't view the GHCup builds of Stack as 'unofficial' but as 'complementary'.

@hasufell
Copy link
Contributor

I am grateful that GHCup fills holes in the platform support for Stack that the Stack responsitory cannot itself currently fill. I don't view the GHCup builds of Stack as 'unofficial' but as 'complementary'.

As I indicated on the stack issue tracker, I think the gitlab CI code should be merged into stack repo. But I've had test failures, because the CI environment on gitlab doesn't match the one on Github actions. I had the feeling this might be a time sink, so I didn't pursue fixing the tests further.

@tomjaguarpaw
Copy link
Collaborator Author

Hi @mpilgrem. Thanks for sharing your perspective. It's great to hear about the successful collaboration between Stack and GHCup!

@Martinsos
Copy link
Contributor

Martinsos commented Oct 14, 2022

Hi all, sorry for barging a bit into this discussion, but I was hoping for possibly a succinct explanation as to what is yet left to get this issue resolved? I read the discussion, but got a bit lost in it. Is it just getting https://gitlab.haskell.org/haskell/ghcup-hs/-/merge_requests/285 merged and that is it?

@mpilgrem
Copy link
Contributor

@hasufell, regarding:

As I indicated on the stack issue tracker, I think the gitlab CI code should be merged into stack repo.

There is a lot going on in the Stack issue tracker, and I can miss things. Can you give me a pointer to the particular contribution you had in mind?

@hasufell
Copy link
Contributor

@hasufell, regarding:

As I indicated on the stack issue tracker, I think the gitlab CI code should be merged into stack repo.

There is a lot going on in the Stack issue tracker, and I can miss things. Can you give me a pointer to the particular contribution you had in mind?

commercialhaskell/stack#5709 (comment)

@hasufell
Copy link
Contributor

Hi all, sorry for barging a bit into this discussion, but I was hoping for possibly a succinct explanation as to what is yet left to get this issue resolved? I read the discussion, but got a bit lost in it. Is it just getting https://gitlab.haskell.org/haskell/ghcup-hs/-/merge_requests/285 merged and that is it?

From my side, there is nothing to do.

@tomjaguarpaw
Copy link
Collaborator Author

Hi @Martinsos, to recap from the initial post in the thread, there are three outstanding requirements if we were to "merge" the functionalities of GHCup and the Stack installer1

  1. Ensure that the installer installs Stack from upstream bindists otherwise Stack maintainers might get bug reports from a bindist they don't support (perhaps GHCup already does this, I'm not sure)
  2. Ensure that the installer installs GHC, Cabal and Stack by default
  3. Ensure that the installer has CLI and GUI functionality on Windows

Items 1 and 3 are taken care of. Regarding 1, @hasufell and @mpilgrem have given their assurances that GHCup and Stack maintainers are cooperating, and this issue is taken care of. Regarding 3, I don't see that GHCup not having a GUI on Windows should be a blocker for moving to GHCup as the single installer recommendation.

Item 2 is the only outstanding issue. When GHCup installs Stack by default we will be able to go ahead and recommend it as the single installer. All I know about progress on this issue is what @hasufell said in an earlier comment.

Footnotes

  1. By "the Stack installer" I mean the installer than installs Stack

@hasufell
Copy link
Contributor

Item 2 is the only outstanding issue.

Since it seems you didn't click on the link I provided, I'll say again: this is not outstanding.

Ensure that the installer has CLI and GUI functionality on Windows

If anyone wants a GUI installer for windows, start contributing patches. haskell.org committee is no exception.

https://gitlab.haskell.org/haskell/ghcup-hs/-/issues/198

I'm not going to work on this any time soon.

@tomjaguarpaw
Copy link
Collaborator Author

Item 2 is the only outstanding issue.

Since it seems you didn't click on the link I provided, I'll say again: this is not outstanding.

I would like to request that in future you don't make assumptions about what I have or have not done. I find it very impolite. I did actually click on the link provided. What conclusion am I supposed to draw from that link? The MR says it is merged. Does that mean that Stack is installed by default in the latest release of GHCup? If so, why didn't you say so explicitly? That would have saved a lot of confusion. Saying "this is not outstanding" is not a very clear way of communicating. If GHCup installs Stack by default in the latest released version then please say so explicitly and we can simply resolve this issue!

@tomjaguarpaw
Copy link
Collaborator Author

If anyone wants a GUI installer for windows, start contributing patches. haskell.org committee is no exception.

I haven't requested a GUI installer. Quite the opposite, I've said I don't see much value in it.

@hasufell
Copy link
Contributor

Item 2 is the only outstanding issue.

Since it seems you didn't click on the link I provided, I'll say again: this is not outstanding.

I would like to request that in future you don't make assumptions about what I have or have not done. I find it very impolite. I did actually click on the link provided. What conclusion am I supposed to draw from that link? The MR says it is merged. Does that mean that Stack is installed by default in the latest release of GHCup?

Why would it not be resolved if:

  1. the MR is merged: https://gitlab.haskell.org/haskell/ghcup-hs/-/merge_requests/285
  2. The corresponding ticket is closed: https://gitlab.haskell.org/haskell/ghcup-hs/-/issues/392 (which mentions installing stack by default too) and
  3. It was publicly announced https://discourse.haskell.org/t/ann-stack-2-9-1/5078/5?u=hasufell
  4. And then publicly resolved: https://discourse.haskell.org/t/ann-stack-2-9-1/5078/9?u=hasufell
  5. And then linked to you again in this thread: Moving towards single installer #212 (comment)
  6. And can be easily observed by running the installer

Guess how confused I am after providing all the evidence and then still reading about how this is not resolved.

Do you need any more information to make your decision? I'd like to unsubscribe from this thread, because I believe there is nothing left for me to do. If you have patches or suggestions, head over to https://gitlab.haskell.org/haskell/ghcup-hs/-/issues

@tomjaguarpaw
Copy link
Collaborator Author

tomjaguarpaw commented Oct 15, 2022

Do you need any more information to make your decision?

Yes, I need to know whether GHCup installs Stack by default. I'm not looking for reams of historical evidence and litigation. I would just like you, the GHCup maintainer, to answer this simple question:

  • Does the latest release of GHCup install Stack by default?

I know you think you've already answered this question. I know you think I should be able to deduce the answer from a dump of links and evidence that you have produced. But please just humour me this time and answer the simple question, preferably with "yes" or "no".


Regarding "Why would it not be resolved if". Well,

  1. the MR is merged: https://gitlab.haskell.org/haskell/ghcup-hs/-/merge_requests/285

    Merging an MR doesn't mean the feature is released. Moreover, this MR is only indirectly related to whether Stack is installed by default. I could not conclude from that that the latest version of GHCup installs Stack by default.

  2. The corresponding ticket is closed: https://gitlab.haskell.org/haskell/ghcup-hs/-/issues/392 (which mentions installing stack by default too) and

    This is the first time you're mentioning this ticket. I didn't take it into consideration. Furthermore, closing a ticket does not always mean the issue is fixed in a released version. I could not conclude from that that the latest version of GHCup installs Stack by default.

  3. It was publicly announced https://discourse.haskell.org/t/ann-stack-2-9-1/5078/5?u=hasufell

    You announced a plan to "then update the GHCup bootstrap script to ... always install stack by default". You didn't announce that you'd actually done so. I could not conclude from that that the latest version of GHCup installs Stack by default.

  4. And then publicly resolved: https://discourse.haskell.org/t/ann-stack-2-9-1/5078/9?u=hasufell

    I can't see where that says that the latest release of GHCup installs Stack by default. I could not conclude from that that the latest version of GHCup installs Stack by default.

  5. And then linked to you again in this thread:
    Moving towards single installer #212 (comment)

    I said "I believe, then, the only remaining component is for Stack to be treated the same as Cabal, with regard to when they are installed by default. Please let me know when that work is completed." and instead of saying "It has been completed" you replied with two links, neither of which says 'the latest release of GHCup now installs Stack by default'". I could not conclude from that that the latest version of GHCup installs Stack by default.

  6. And can be easily observed by running the installer

    Yes, I suppose I could have done that.

From my perspective the communication around this issue that I have been receiving is extremely unclear. From your responses to some of my messages here I understand that you feel I have communicated unclearly too. I will try to be clearer in the future.

@tomjaguarpaw
Copy link
Collaborator Author

Oh, and if GHCup changed behaviour to install Stack by default then I would have expected to see that noted in the CHANGELOG, but I can't see any mention of it.

@tomjaguarpaw
Copy link
Collaborator Author

tomjaguarpaw commented Oct 15, 2022

Do you need any more information to make your decision?

Yes, I need to know whether GHCup installs Stack by default.

To save you the trouble of answering I performed a new install of GHCup and can confirm that Stack is installed by GHCup by default.

Great news! Thanks @hasufell, @mpilgrem and to everyone else who contributed to this.

@hasufell
Copy link
Contributor

@tomjaguarpaw apologies... you're right.

I was assuming it was common knowledge that changes to the bootstrap script are released rolling and immediately.

That certainly was an assumption.

Basically, master branch should reflect the state of the bootstrap script at all times.

@tomjaguarpaw
Copy link
Collaborator Author

Apology accepted. Thanks again for all your work on GHCup. The state of installation in the Haskell world is very good because of it!

@Martinsos
Copy link
Contributor

@hasufell thanks for all the effort ensuring the support in GHCup, and thanks @tomjaguarpaw and @mpilgrem for all the effort to overcome communication issues and get this to the finish line. This will now also enable merging of #214 -> looking forward to that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants