Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add allow_failures to Travis #993

Merged
merged 5 commits into from
Feb 27, 2019
Merged

Add allow_failures to Travis #993

merged 5 commits into from
Feb 27, 2019

Conversation

sh0ji
Copy link
Contributor

@sh0ji sh0ji commented Feb 26, 2019

Addresses #992.

This should allow jobs to fail when they match certain criteria, allowing us to make some jobs non-blocking.

@nschonni
Copy link
Contributor

wonder if it is possible to make it required for master PRs. That wouldn't fix the linked issue, but would mean that topic branches could keep going, but need to address the issues before merging into the mainline.

@sh0ji
Copy link
Contributor Author

sh0ji commented Feb 26, 2019

From what I can tell, something like the following would allow jobs that aren't part of a pull request (i.e. branch jobs) to fail, but the PR jobs wouldn't be allowed to fail since they don't match envs perfectly.

allow_failures:
  - env:
    - ALLOW_FAILURE=true
    - TRAVIS_PULL_REQUEST=false

I'd rather not implement that, though. The "allowed to fail" jobs still fail and maintainers can still ask contributors to resolve those issues if it is necessary.

And probably more importantly, we don't want regression coverage to be blocking, though. Getting the examples out is more important than ensuring 100% coverage.

@sh0ji
Copy link
Contributor Author

sh0ji commented Feb 26, 2019

I split the linting jobs into their own stage, which I may revert once I see how long everything takes.

@sh0ji
Copy link
Contributor Author

sh0ji commented Feb 26, 2019

Yep, splitting linting/testing jobs doesn't seem worth it. My original thinking was that it might be better to run the cheaper tests first (each linting job takes 40s-2 minutes), but the overall impact doesn't seem worth it, especially since there isn't a build stage.

@mcking65 this should be good to go once everything passes.

@mcking65
Copy link
Contributor

Thank you @sh0ji and @zcorpan; Looks great! Excellent improvement!

@mcking65 mcking65 merged commit 089049b into master Feb 27, 2019
@mcking65 mcking65 deleted the travis-update branch February 27, 2019 08:11
michael-n-cooper pushed a commit that referenced this pull request Feb 27, 2019
Travis CI: Enable deploy to continue when tests we consider informational fail (pull #993)

To address issue #992, In .travis.yml:
* add initial allow_failures flag and apply it to regression test coverage report job.
* only set environment variables where necessary.
* reorganize jobs for readability.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants