Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "no_parallel" pytest's mark #6799

Merged
merged 2 commits into from
Mar 4, 2022
Merged

Add "no_parallel" pytest's mark #6799

merged 2 commits into from
Mar 4, 2022

Conversation

drew2a
Copy link
Contributor

@drew2a drew2a commented Mar 3, 2022

This PR fixes #6709 by introducing the no_parallel pytest's mark.

All tests that have this mark will be excluded from normal "parallel" Jenkins's run. But they will be run sequentially instead (like tunnel tests do).

In case this PR will be approved, I will merge it to the main and then modify corresponding Jenkins' jobs.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Mar 3, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@drew2a drew2a marked this pull request as ready for review March 3, 2022 18:48
@drew2a drew2a requested review from a team and kozlovsky and removed request for a team March 3, 2022 18:48
@drew2a drew2a requested a review from devos50 March 4, 2022 09:10
@drew2a drew2a merged commit 665d164 into main Mar 4, 2022
@drew2a
Copy link
Contributor Author

drew2a commented Mar 4, 2022

The current Jenkins test run is the following:

pytest src/tribler-core -v --tx '12*popen//execmodel=eventlet' -p no:randomly --cov=tribler_core  --cov-report=xml:../output/coverage.xml --junitxml=test_report.xml

# Tunnel tests
pytest src/tribler-core/tribler_core/components/tunnel/tests/test_full_session --tunneltests --log-cli-level=INFO

pytest src/tribler-core -v -p no:randomly --no_parallel

@devos50 @kozlovsky do you know, is it possible to share coverage reports among the different test runs? It would be nice to include coverage from no_parallel and tunneltests to the main coverage report.

@devos50
Copy link
Contributor

devos50 commented Mar 4, 2022

@drew2a yes, I believe cobertura has an option to merge separate code coverage files into one (but they need to have the same base path). We should merge the reports generated by the "no parallel" and parallel test runs. For the tunnel tests, however, I'm not so sure: the tunnel tests are more leaning to a full-fletched application test and the code coverage reported by them might hide lines that are uncovered by our unit tests.

@drew2a
Copy link
Contributor Author

drew2a commented Mar 4, 2022

@devos50 ok, thanks. I'll find a way to merge coverage for parallel and no-parallel test runs.

@kozlovsky
Copy link
Contributor

@drew2a it should be possible to use the --cov-append option.

Then the Jenkins command will look like:

pytest src/tribler-core -v --tx '12*popen//execmodel=eventlet' --cov=tribler_core  --cov-report=xml:../output/coverage.xml --junitxml=test_report.xml
pytest src/tribler-core -v --no_parallel --cov=tribler_core  --cov-report=xml:../output/coverage.xml --cov-append

@drew2a
Copy link
Contributor Author

drew2a commented Mar 4, 2022

Jenkins jobs that run for a PR have been updated (Win, Linux, MacOS).

@devos50 @kozlovsky 🙏

@drew2a drew2a mentioned this pull request Mar 4, 2022
@drew2a drew2a deleted the fix/6709 branch March 15, 2022 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Test test_gigachannel_manager_component randomly fails on Window machine
3 participants