Make parametrized tests xdist-compatible

[abalakh]

Issue by abalakh
Monday Feb 19, 2018 at 12:34 GMT
Originally opened as abalakh/airgun#22

---

(robottelo issue)
Currently test runs with parametrized tests using xdist are failing with msgs like:

collecting 0 items / 3 errors
==================================== ERRORS ====================================
_____________________________ ERROR collecting gw1 _____________________________
Different tests were collected between gw0 and gw1. The difference is:

This is due to random data generated inside parametrize. We should probably use the same hash seed across different workers. More info here: pytest-dev/pytest#1075

[abalakh]

Apart from using the same hash seed we can stop sending random data in test names - instead of test_positive_create[dfh7yt94hf9] introduce names for datafactory values, so tests will always be named like test_positive_create[alphanumeric], test_positive_create[utf-8] etc. That's even better naming and it will also avoid encountering such issue.

[omaciel]

I like your suggestion @abalakh and it does make it easier to see what type of data was passed (the actual value can always be gotten by looking at logs, if needed)

[abalakh]

Addressed in SatelliteQE/robottelo#5904 , you can see updated names in the test results there:

[...]
tests/foreman/ui_airgun/test_activationkey.py::test_positive_create_with_envs[alphanumeric] PASSED [ 22%]
tests/foreman/ui_airgun/test_architecture.py::test_positive_create[alphanumeric] PASSED [ 36%]
tests/foreman/ui_airgun/test_architecture.py::test_positive_delete[alpha] PASSED [ 45%]
tests/foreman/ui_airgun/test_operatingsystem.py::test_positive_create[numeric] PASSED [ 68%]
[...]