Possibility: run tests in tests/integration/cmor/_fixes with one thread only

[valeriupredoi]

I am currently conducting an experiment and am running the tests every hour on the GA machine, but with pytest -n 1 see if my previous find that tests don't segfault when run in single-thread mode. If this proves to be true, I would argue that we should run a number of tests like this - not all of them because for one, not all tests result in segfaults and two, I have noticed only the ones in tests/integration/cmor/_fixes to be segfaulting (well 99.9% of times). @bouweandela or @jvegasbsc do you guys know how to run a subset of the tests in single-thread whereas keeping the others in multi-threading? BTW if you see a ton of Test jobs on the GA machine, that's me experiment 😁

[valeriupredoi]

and I totally forgot that scheduled GA jobs run only on master so am gonna make that a PR and decrease the frequency from hourly to 6-hourly so we don't clog the GA machine too badly and ask to merge into master so we can observe

[stefsmeets]

I looked into this a while ago, and I could not find a way to run a subset of tests in a single thread. There is an open issue for this: pytest-dev/pytest-xdist#385.

Could mark the segfaulting tests using @pytest.mark.sequential, and then do pytest -n1 -m"sequential"

[valeriupredoi]

@stefsmeets cheers! that's a good idea - but decorating them that way means they will run only when they get invoked with the -m "sequential" runtime flag?

[stefsmeets]

Yeah, you would have to do the following I think:

pytest -n1 -m"sequential"
pytest -n4 -m"not sequential"

Maybe one or the other can be set as a default in setup.cfg.

[valeriupredoi]

unfortunately this is not a solution to get us rid of the Segmentation Faults. I've just looked at the 3-hourly tests that have been running with -n 1 and segfaults creep in there too, going to revert the change to the GA workflow (so we don't run that test like pogos every 3 hours for nothing) and close the issue after, the mystery still persists... 😖

[valeriupredoi]

reverted the changes in #1025 if someone is nice to approve it, we'll forget about this. On to figuring out another cause of segfaults