Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: improve when we run the full test suite on CI #8233

Open
gforsyth opened this issue Feb 5, 2024 · 1 comment
Open

ci: improve when we run the full test suite on CI #8233

gforsyth opened this issue Feb 5, 2024 · 1 comment
Labels
ci Continuous Integration issues or PRs developer-tools Tools related to ibis development tests Issues or PRs related to tests

Comments

@gforsyth
Copy link
Member

gforsyth commented Feb 5, 2024

Current status

We have a large test suite and don't want to run the entire thing for unrelated changes. Currently, there are two ways that CI gets run:

  1. Run everything
  2. If all the changes in a PR (or push) are within the docs/ folder (or are one of a few config files that lives at the root of the repo), then only run a subset of the test suite (mostly the docs-related bits).

This is a good start, but I think we can probably reduce the testing burden more with a few rules that should give us reasonably good coverage (I think).

New rules

  • Keep the current "docs-only" rule, but also add in ci(codespell): separate codespell config into standalone file #8231 to allow for adding words to the codespell dictionary without triggering a full CI run
  • For changes in ibis/expr/: run everything
  • For changes in ibis/backends/base: run everything
  • For changes in ibis/backends/<backend name>: only run the tests for that backend
    • An exception to this rule is Risingwave, which is mostly the postgres backend and so changes to postgres should also trigger risingwave tests

Then, as a fallback, we add a label akin to our docs-preview label that can trigger a full backend test suite run if the reviewer thinks that something might have a larger blast radius than accounted for by these rules.

@gforsyth gforsyth added tests Issues or PRs related to tests ci Continuous Integration issues or PRs developer-tools Tools related to ibis development labels Feb 5, 2024
@ncclementi
Copy link
Contributor

ncclementi commented Feb 20, 2024

I just found out about path-filters GHA thanks to this blog https://www.flightcontrol.dev/blog/reduced-github-actions-bill-by-63.

I think we could use this to filter by backend folders.

@gforsyth gforsyth removed their assignment Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Continuous Integration issues or PRs developer-tools Tools related to ibis development tests Issues or PRs related to tests
Projects
Status: backlog
Development

No branches or pull requests

2 participants