Cache charmcraft pack container, skip unstable tests except on schedule #18
Conversation
Add time limit to jobs Ported from canonical/mysql-operator#134
tox.ini
Outdated
| commands = | ||
| pytest -v --tb native --ignore={[vars]tst_path}unit --log-cli-level=INFO -s {posargs} | ||
| pytest -v --tb native --log-cli-level=INFO -s {posargs} {[vars]tests_path}/integration/test_s3_charm.py |
There was a problem hiding this comment.
Could we just make it a bit more general? like just do {[vars]tests_path}/integration
I mean, I see the use case of someone adding a integration tests and then not running it
There was a problem hiding this comment.
The idea is that if another test file is added, it should be added to the matrix in ci.yaml. pytest-operator creates a new juju model for each Python test file, so it's more efficient for CI to run each of those models on a separate runner
This is currently the approach we're using in our other repositories
There was a problem hiding this comment.
I believe it makes sense that the [testenv:integration] runs all integrations, other more specific like [testenv:integration-charm] run the specific file.
That's why I was saying that not having the specific file is more consistent. If we develop multiple tests, we can then choose to run them parallel, but we need to create the separated envs
There was a problem hiding this comment.
On mysql-operator CI, I'm currently working in the direction of merging the integration tox environments to a single environment: canonical/mysql-operator#109
If you want a separate [testenv:integration] that runs all integrations, feel free to add it. I don't think such an environment should be called by CI
There was a problem hiding this comment.
I understand the scope of the PR, although in this specific case is not producing much improvements since it has only one file. But I believe it HAS value to already have the structure in place for being efficient should we add more files, hence the PR.
Here I'm just saying that it is consistent with other repo to call [testenv:integration] to run all integration environments, that's why I would just remove the filename.
When one creates other files and wants to speed up things, they will create other envs called [testenv:integratoin-{suffix}] and run specific file on them test_{suffix}.py, and change the ci.yaml accordingly with those names. If they do not do that, they will still run all the tests
Ported from canonical/mysql-k8s-operator#146