Problems with tests

[k-dominik]

#1 has been there for a really long time, and while it will eventually be closed via #6 some principle problems with the tests remain:

• tests take a really long time:

  563.85s call     tests/test_wrapper_selection.py::test_wrapper[SBE-True-1275-expected5]
  315.31s call     tests/test_wrapper_selection.py::test_initial_set
  294.91s call     tests/test_wrapper_selection.py::test_wrapper[SBE-False-1275-expected4]
  219.84s call     tests/test_wrapper_selection.py::test_wrapper[BFS-False-14271-expected0]
  188.01s call     tests/test_wrapper_selection.py::test_mandatory_features
  165.09s call     tests/test_wrapper_selection.py::test_wrapper[BFS-True-14271-expected1]
  139.97s call     tests/test_wrapper_selection.py::test_wrapper[SFS-False-1275-expected2]
  139.90s call     tests/test_wrapper_selection.py::test_wrapper[SFS-True-1275-expected3]
  76.87s call     tests/test_wrapper_selection.py::test_permitted_features
  

• Tests in test_wrapper_selection.py are regression tests, which of course is fine. With the current status of the code (I'd say its unmaintained) the proposed modus operandi in Unit tests fail #1, implemented in Re-enable tests #6, to regenerate the test results, feels shady.
• Right now we don't have the pyfeast package built for Python3 -> those tests are skipped. Question is, whether we want/need those.

[k-dominik]

just a tiny update

test durations with sklearn rf on M1:

116.84s call     tests/test_wrapper_selection.py::test_wrapper[SBE-True-1275-expected5]
82.86s call     tests/test_wrapper_selection.py::test_initial_set
82.46s call     tests/test_wrapper_selection.py::test_wrapper[SBE-False-1275-expected4]
47.49s call     tests/test_wrapper_selection.py::test_wrapper[BFS-True-14271-expected1]
43.48s call     tests/test_wrapper_selection.py::test_wrapper[BFS-False-14271-expected0]
34.38s call     tests/test_wrapper_selection.py::test_mandatory_features
31.51s call     tests/test_wrapper_selection.py::test_wrapper[SFS-True-1275-expected3]
29.59s call     tests/test_wrapper_selection.py::test_wrapper[SFS-False-1275-expected2]
14.28s call     tests/test_wrapper_selection.py::test_permitted_features

test durations with the vigra rf on M1:

68.99s call     tests/test_wrapper_selection.py::test_wrapper[SBE-True-1275-expected5]
46.79s call     tests/test_wrapper_selection.py::test_wrapper[SBE-False-1275-expected4]
28.83s call     tests/test_wrapper_selection.py::test_initial_set
26.95s call     tests/test_wrapper_selection.py::test_mandatory_features
17.65s call     tests/test_wrapper_selection.py::test_wrapper[BFS-False-14271-expected0]
14.58s call     tests/test_wrapper_selection.py::test_wrapper[BFS-True-14271-expected1]
11.69s call     tests/test_wrapper_selection.py::test_wrapper[SFS-True-1275-expected3]
11.05s call     tests/test_wrapper_selection.py::test_wrapper[SFS-False-1275-expected2]
7.03s call     tests/test_wrapper_selection.py::test_permitted_features

Overall, the tests seem to be twice as fast using the vigra random forest.