Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Actions backend doesn't respect jobs, resulting in lack of build parallelism #551

Closed
RyanGlScott opened this issue Oct 30, 2021 · 8 comments · Fixed by #552
Closed

Comments

@RyanGlScott
Copy link
Contributor

In the haskell-ci–generated GitHub Actions workflow for transformers-compat, I noticed that despite settings jobs: 2, the dependencies were being built sequentially. On the other hand, this doesn't happen with the Travis backend:

-- Cabal jobs
for_ (cfgJobs >>= cabalJobs) $ \n ->
sh $ "echo 'jobs: " ++ show n ++ "' >> $CABALHOME/config"
-- GHC jobs + ghc-options
for_ (cfgJobs >>= ghcJobs) $ \m -> do
shForJob (Range $ C.orLaterVersion (C.mkVersion [7,8])) $ "GHCJOBS=-j" ++ show m

The GitHub Actions backend should do something similar.

@phadej
Copy link
Collaborator

phadej commented Oct 30, 2021

looks like we didn't fix #421?

@RyanGlScott
Copy link
Contributor Author

This is a slightly different issue. The cabal.haskell-ci file for transformers-compat specifies both of these options:

jobs:                   2
install-dependencies:   False

We explicitly pass -j2 to cabal build when building the dependencies, but not to the subsequent invocation of cabal build. However, transformers-compat is a special case where libraries in its test suite depend on transformers-compat itself, so as a result, one can't build its dependencies beforehand (hence the use of install-dependencies: False). In order to recover some amount of package-level build parallelism, I'm using jobs: 2, which affects the subsequent invocation of cabal build.

@phadej
Copy link
Collaborator

phadej commented Oct 30, 2021

It feels that we want to have granular jobs control for different steps. That is doable, but I won't have time to think about this now. Especially if this is only "bad" for install-dependencies: False setups.

@RyanGlScott
Copy link
Contributor Author

In the meantime, would you accept a PR that makes the behavior of jobs consistent across both the Travis and GitHub Actions backends?

@phadej
Copy link
Collaborator

phadej commented Oct 30, 2021

Btw, please comment on haskell/cabal#6106 so Cabal folks could prioritize the issue. Whole install-dependencies: False is a workaround for it.

In the meantime, would you accept a PR that makes the behavior of jobs consistent across both the Travis and GitHub Actions backends?

No, I won't have time to review it any time soon, I'm sorry.

@RyanGlScott
Copy link
Contributor Author

RyanGlScott commented Oct 30, 2021

No, I won't have time to review it any time soon, I'm sorry.

I'm a little unclear. Are you philosophically opposed to the current design of the jobs flag, or are you just pressed for time? If it's the latter, I'd be happy to wait until you do have time to review it (and I don't expect the patch to be that large).

@phadej
Copy link
Collaborator

phadej commented Oct 30, 2021

just pressed for time?

That.

RyanGlScott added a commit to RyanGlScott/haskell-ci that referenced this issue Oct 30, 2021
@RyanGlScott
Copy link
Contributor Author

Sounds good. When you're ready, I've submitted a fix at #552.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants