Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A middle ground between --tests and --no-tests #236

Closed
RyanGlScott opened this issue Mar 2, 2019 · 5 comments
Closed

A middle ground between --tests and --no-tests #236

RyanGlScott opened this issue Mar 2, 2019 · 5 comments

Comments

@RyanGlScott
Copy link
Contributor

Currently, haskell-ci gives you the ability to prevent building any test suites using the --no-tests flag. This is almost what I need, but not quite. I have a project where I can't run the test suites on Travis due to the tests' reliance on a web browser being run at the same time. That being said, I still want to build (but not run) the test suite on Travis so as to ensure that the code doesn't bitrot. Unfortunately, --no-tests is too heavy-handed, as it prevents the test suites from being built in the first place.

Could there be a flag that permits the test suite to be built but not run, similarly to how benchmarks are currently handled? I'm not sure what the best name for this flag should be, or if it would make sense to make --no-tests further configurable to allow this.

@phadej
Copy link
Collaborator

phadej commented Mar 2, 2019

I was thinking of introducing --run-tests, --no-run-tests --run-tests-job=RANGE configuration, which is

  • intersected with --tests
  • says when to run tests.

@RyanGlScott
Copy link
Contributor Author

Sounds good to me! If these flags do intersect with --tests, it would be good to document somewhere how that works, but otherwise I don't foresee any complications.

@phadej
Copy link
Collaborator

phadej commented Mar 2, 2019

Intersection means hopefully obvious: only built tests can be run, but you can run tests on less jobs. E.g.

haskell-ci --tests-jobs '>= 7.6' --run-tests-jobs '>=8.0' ...

@phadej
Copy link
Collaborator

phadej commented Mar 2, 2019

... and with

haskell-ci --no-tests --run-tests

you won't build (and run) any tests.

phadej added a commit that referenced this issue Mar 6, 2019
- Add global --run-tests/--no-run-tests/--run-tests-jobs to specify
  when we actually run tests (maybe less then what we build!)

- Add steps configuration to constraint sets:
  - tests
  - run-tests
  - benchmarks
  - haddock

  - these are only booleans for now, if you need more control, make
    another constraint set!
phadej added a commit that referenced this issue Mar 7, 2019
@phadej
Copy link
Collaborator

phadej commented Mar 7, 2019

--run-tests is there

@phadej phadej closed this as completed Mar 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants