Avoid templating GitHub Actions workflow

[andreabedini]

GitHub Actions workflow have sufficient power to express what we need.
We don't need to maintain and additional templating solution on top.

This converts: linux.yml, macos.yml and windows.yml to native GitHub Actions syntax. The result should closely (but perhaps not perfectly) match the original version.

EDIT:

It would be better to check GHC versions with startsWith(matrix.ghc, "8.10") otherwise bumping minor versions is going to be error prone I am in

---

Please include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
• The documentation has been updated, if necessary.

Please also shortly describe how you tested your change. Bonus points for added tests!

[Mikolaj]

@fgaz is just extending our CI to GHC 9 and 9.2 and converting it in the process in #7907, so it may be beneficial to coordinate...

[andreabedini]

Thanks @Mikolaj, I think I was able to include most of the changes of #7907. Happy to coordinate to finish the job.

[fgaz]

Hmm.. why isn't the linux job running?

[andreabedini]

Hmm.. why isn't the linux job running?

/shurg I don't know. I am working to be able to test this out locally (with act).

I will review the failures on Monday.

[fgaz]

Hmm.. why isn't the linux job running?

/shurg I don't know. I am working to be able to test this out locally (with act).

Looks like the workflow is invalid https://github.com/haskell/cabal/actions/runs/1794333751

[andreabedini]

Rebased

[andreabedini]

The workflows for macos and linux at this point are very similar and could be merged. They only differ in how they install cabal-plan (one downloading a binary, one installing from hackage)

[jneira]

The Haskell community has been severely burned by the treason of Travis CI. CI just vanished for hundreds (or thousands?) of Haskell projects when travis-ci.org was shut down. The community is still recovering from this blow. I donated a GHA CI to dozens of Haskell projects. This was possible through @phadej 's heroic migration of Haskell-CI from Travis to GHA.

That was a unsung epic travispocalypse, thank you for your work.
Lot of work was invested in travis and its sunset was really painful. The lessons from that pain should not be forgotten.
However here in Spain we have a proverb: "Gato escaldado, del agua fria huye" (in english it would be "once bitten twice shy" i think)

I think there are two directions in ci lock in (or adoption :-P):

1. Up-down: this is about the top level inputs of workflows. The matrix of ghc versions and os's and the different parameters for each combination of them.

• That was abstracted from ci thanks to GenValidate and that is which we are giving out to github with this pr
• They are essential for the ci logic but
  • The size of data is relatively small and dont vary too much
  • They are not actual code so no ifs, no loops
  • All ci providers have similar ways to encode them so it is relatively easy convert between them

2. Bottom: The actual individual steps of the workflow, full of logic and default values which needs to be programmed

• Imho that logic should live, in that precise order:
  1. haskell code (test suites, benchmarks, haskell scripts, build config)
  2. other language scripts: sh, python, etc
  3. using ci specific features
• cabal uses 1 and 2 (with validate.sh and make etc) and i think we should keep them (but we can discuss that in other issue)

And for the top inputs we always could have it encoded in a ci agnostic data format and inject it in the ci provider, if ci allows to do it (see the linked above https://www.cynkra.com/blog/2020-12-23-dynamic-gha/)

[jneira]

They are not actual code so no ifs, no loops

For completeness, they are not code but it could be computed, deriving it from other sources, to avoid duplication of such sensitive data.
For example we could extract the ghc versions from tested-with like haskell-ci does. But being a relatively small dataset, encoding its generation to avoid oversights and have to do greps should be very cheap to worth, imho.

EDIT: want to note GenValidate did not compute the values but they were duplicated and hardcoded in the haskell code

[jneira]

I have a minor nitpick: the experimental builds are marked as failing, so the entire build is marked that way. Those workflows will not be added to branch protection ruels and prs will be merged anyways but aesthetically it does not look nice
(Btw we have to remember add the succesful workflows to branch protection rules once this is merged and rebase the rest of prs // @Mikolaj)

Otoh, have them red is a compelling reason to work on fixing this so i am not strongly against it, thoughts?
Maybe marking it as "yellow" would be a compromise. not sure how could be done, cancelling them?

In hls we have a final post-jobs which marks the final workflow state, not sure if it could help:

https://github.com/haskell/haskell-language-server/blob/96ea854debd92f9a54e2270b9b9a080c0ce6f3d1/.github/workflows/test.yml#L239-L249

[andreabedini]

I have a minor nitpick: the experimental builds are marked as failing, so the entire build is marked that way.

Only the "branches" are marked as failing, the entire workflow is green if all the required branched pass, see https://github.com/haskell/cabal/actions/runs/1863088120

For example we could extract the ghc versions from tested-with like haskell-ci does.

This is a good idea (especially in relation to the post about dynamic workflows you posted). But, again, my intention is to try to simplify all variations rather than capturing them perfectly.

[jneira]

hmm steps also lives here, sad to see they go away

[jneira]

Many thanks again, really a great work

[jneira]

This pr cant be merged automatically, as it changes required jobs names as commented above we can change branch protection rules to match new identifiers and merge it (to check both pr jobs and updated ruels) and then aaaall prs need to be rebased to pass ci

//cc @Mikolaj

[fgaz]

Looks good! I'll merge tomorrow if nobody objects. Specifically, let's squash-merge since there are a lot of fixup commits. @andreabedini if you don't want them squashed please tell!

[fgaz]

Thanks a lot @andreabedini!

Now I just have to re-select the required jobs... please ping me or @Mikolaj if prs are stuck because of that

[phadej]

I don't see these steps in new CI setup. Is cabal-install dropping support for older GHC.

AFAICS the oldest GHC you try to run tests with is GHC-8.0

Is this correct and intentional, @fgaz, @Mikolaj ?

[jneira]

Afaics it is not intentional and should be restored.

[jneira]

As we are deprecating build cabal with ghc < 8.0, all older version should be checked using a newer build (including 7.8 and 7.10) like < 7.6 before this pr

EDIT: the haskell setup action fall back to apt inlinux so maybe it could be used

[jneira]

the oldest ghc instalable with ghcup is 7.10.3, so we have to continue using apt if we want to keep older ones

[andreabedini]

@phadej It was intentional, following #7534 but now I think I have misunderstood.

We are deprecating building with GHC < 8.0 but we still want to test cabal can use GHC 7.8 and 7.10. Is this correct?

It's a bit of headache (which GHC version we should use to compile the cabal that compiles with what GHC version?) but it can definitely be readded.

Mind adding an issue to track? Please mention me there.

[Mikolaj]

@andreabedini: great job! I might have mislead you re #7534. Let's clean up the fallout later on. Edit: in #8000.

[jneira]

Thanks a lot @andreabedini!

Now I just have to re-select the required jobs... please ping me or @Mikolaj if prs are stuck because of that

The linux and macos validate jobs has the same name and in my experience tha does not play well with branch protection rules, which needs unique jobs names

[andreabedini]

Thanks a lot everyone <3 It's been a jouney for me!