dsp tests

[gipert]

• added some first tests
• fix/add type hints
• fix docstrings
• make overwrite mode = false in build_raw and build_dsp

[codecov-commenter]

Codecov Report

Merging #310 (7d67619) into main (9ec0884) will increase coverage by 4.53%.
The diff coverage is 65.77%.

@@            Coverage Diff             @@
##             main     #310      +/-   ##
==========================================
+ Coverage   44.88%   49.42%   +4.53%     
==========================================
  Files          74       74              
  Lines        6372     6396      +24     
==========================================
+ Hits         2860     3161     +301     
+ Misses       3512     3235     -277     

Impacted Files	Coverage Δ
src/pygama/dsp/__init__.py	100.00% <ø> (ø)
src/pygama/math/peak_fitting.py	26.61% <ø> (ø)
src/pygama/dsp/processors/fftw.py	20.00% <20.00%> (ø)
src/pygama/dsp/processors/pulse_injector.py	33.33% <33.33%> (ø)
src/pygama/dsp/processors/param_lookup.py	36.36% <36.36%> (ø)
src/pygama/dsp/processors/moving_windows.py	18.75% <45.45%> (ø)
src/pygama/dsp/processors/wiener_filter.py	13.04% <66.66%> (ø)
src/pygama/dsp/processors/pole_zero.py	25.80% <75.00%> (ø)
src/pygama/dsp/processors/multi_t_filter.py	26.47% <80.00%> (ø)
src/pygama/dsp/build_dsp.py	81.81% <100.00%> (+65.32%)	⬆️
... and 28 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ec0884...7d67619. Read the comment docs.

[jasondet]

@iguinn can you review this when you get a chance?

[iguinn]

As someone fairly inexperienced in writing tests for python, I want to make sure that I understand the goals for each test:
test_processing_chain.py: this seems to be essentially have integration tests to make sure that it can run the kinds of processors we want it to run (like running a scipy processor) and unit tests to make sure that the parsing of different cases works
test_build_dsp.py: this seems to be a functional test that's designed to make sure that we can process a real file
test_waveform_browser.py: this seems to be a functional test that draws waveforms

For test_build_dsp, as I commented above, do we want this to use a full-fledged LEGEND DSP config file, or do we want to pare it down to more of a minimal working file? My (possibly flawed) understanding is that we would prefer to use unit testing for the individual processors and design this test to be simpler?

For test_waveform_browser.py, this involves actually drawing waveforms. How does this work for automatic testing? Is it more or less just going to look for if any errors are thrown, or is this more meant for someone to run on their own to visually confirm that waveforms are drawn correctly?

[iguinn]

Are we sure this is how we want to handle paths to json config files for testing? Another option would be to add the files to our setup.cfg (https://setuptools.pypa.io/en/latest/userguide/datafiles.html) and use importtools (https://docs.python.org/3/library/importlib.html#module-importlib.resources) to use them in cases like this. This could also be useful if we wanted to have a default config file that we use for tutorials and such.

[gipert]

I don't think it's useful t to install test data/configs. By design, the test suite is meant to be run by developers on the pygama source (not the installed code!).

For tutorials/docs the story is different. I agree that we should include them as installed data files.

[iguinn]

I think I'm missing something here...What kind of object is lgnd_test_data and how is it getting passed here? I assume this is pointing to the test data repository?

[gipert]

It's defined here as a Pytest fixture. Fixtures are useful to share objects between tests.

[iguinn]

Are we sure we want to use a full-fledged DSP file like this, rather than a more minimal example? It seems like the job of testing the individual processors should go to unit_tests, and for testing build_dsp and processing_chain, we may want a more minimal file

[gipert]

I agree that we want unit tests for the processors folder. My plan (not part of this PR) is to have a test for each single processor. Still, testing something similar to what we have in the L200 data production seems useful to me...

[jasondet]

@iguinn can you check these conflicts when you get a chance?

[gipert]

As someone fairly inexperienced in writing tests for python, I want to make sure that I understand the goals for each test:
test_processing_chain.py: this seems to be essentially have integration tests to make sure that it can run the kinds of processors we want it to run (like running a scipy processor) and unit tests to make sure that the parsing of different cases works
test_build_dsp.py: this seems to be a functional test that's designed to make sure that we can process a real file
test_waveform_browser.py: this seems to be a functional test that draws waveforms

Correct.

For test_build_dsp, as I commented above, do we want this to use a full-fledged LEGEND DSP config file, or do we want to pare it down to more of a minimal working file? My (possibly flawed) understanding is that we would prefer to use unit testing for the individual processors and design this test to be simpler?

Correct, see my comment above.

For test_waveform_browser.py, this involves actually drawing waveforms. How does this work for automatic testing? Is it more or less just going to look for if any errors are thrown, or is this more meant for someone to run on their own to visually confirm that waveforms are drawn correctly?

Yes, It is more or less just looking for if any errors are thrown. Testing image outputs is fairly complicated, obviously, and I'm not sure we'll ever have time to work on this...

[gipert]

@jasondet the Sphinx bug has been fixed: sphinx-doc/sphinx#10701

I'm going to merge this, I'll continue working on tests in future PRs.