Add parallel test execution to CI with pytest-xdist library

[ClementLaplace]

This PR is related to the parralelisation of the tests as mentioned in this issue

[djhoese]

Very interesting failures. I've restarted the ubuntu 3.12 test just to make sure it isn't macos-specific. As a last resort I think I remember there being a way to tell pytest that some tests can't be run in parallel, but I'm also surprised that these are failing.

[djhoese]

I've been doing some testing of this locally and I think I've discovered one of the main issues although I'm not 100% sure it is the only issue. In satpy/tests/test_writers.py I wrote a ton of stuff that on setup_class and teardown_class will create enhancement YAML files. It currently puts these in the current working directory which messes up other portions of Satpy that are scanning for configuration files and look for YAML files in the current directory or based in the current directory (ex. test_sensor1.yaml, enhancements/test_sensor1.yaml, etc). I'll see if I can clean them up and push them to this branch and see how things behave.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.95%. Comparing base (440925d) to head (53b10be).
Report is 378 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2850      +/-   ##
==========================================
- Coverage   95.96%   95.95%   -0.02%     
==========================================
  Files         366      366              
  Lines       53593    53615      +22     
==========================================
+ Hits        51433    51446      +13     
- Misses       2160     2169       +9     

Flag	Coverage Δ
behaviourtests	4.04% <0.00%> (+<0.01%)	⬆️
unittests	96.05% <100.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[djhoese]

@pytroll/satpy-core and @ClementLaplace any reason not to use -n auto on CI? If they give us the cores why not use them, right?

[coveralls]

Pull Request Test Coverage Report for Build 9976726430

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 57 of 57 (100.0%) changed or added relevant lines in 2 files are covered.
• 57 unchanged lines in 4 files lost coverage.
• Overall coverage decreased (-0.02%) to 96.054%

---

Files with Coverage Reduction	New Missed Lines	%
satpy/tests/utils.py	2	93.16%
satpy/tests/reader_tests/gms/test_gms5_vissr_l1b.py	3	98.67%
satpy/tests/reader_tests/gms/test_gms5_vissr_navigation.py	3	97.18%
satpy/resample.py	49	88.74%

Totals
Change from base Build 9842330086:	-0.02%
Covered Lines:	51675
Relevant Lines:	53798

---

💛 - Coveralls

[djhoese]

I'm not sure I'm happy with my choice of fixing my enhancement tests by using a fixture, but I didn't really like any the solutions I came up with either.

[sfinkens]

Looks good to me, just one question.

[sfinkens]

What is the chdir needed for?

[djhoese]

The tests are testing that configs in the current working directory are discovered and used. However, now that I say that that might not be important to all the tests. I'll try to review the tests and see if this chdir can be limited to only a few tests.

[djhoese]

So there is at least one class of tests that I think it is important that the configs come from the current working directory. Or at least it is nice that they test it. The other tests would be fine if the config path was instead added using the satpy.config interface(s) for adding user custom configs. Either way, this current directory versus satpy.config directory stuff should be handled by other satpy.config-specific tests and is not necessary to test here. The tests look about the same whether satpy.config is used versus chdir, but I think the chdir ends up being a little cleaner but this is more of a personal preference and if I'm being honest some level of laziness that I don't want to change it.

Unless someone feels strongly I think I'd like this merged as-is and if more cleanup is needed in the future I can do that.

[sfinkens]

@pytroll/satpy-core and @ClementLaplace any reason not to use -n auto on CI? If they give us the cores why not use them, right?

I think it's a great idea! Thanks for bringing this up @ClementLaplace

[djhoese]

Ah github actions only gives use 2 cores except on macos it gives us 3 it seems. I guess "auto" is still a good thing to do for now.

[mraspaud]

looks good to me, thanks @ClementLaplace @djhoese !