Distribute tests #669

kohr-h · 2016-10-21T22:29:50Z

Unfortunately I realized too late that the tests are not included, neither in the pip nor the conda packages. The problem is basically that odl/test is not a package and thus not included. We actually explicitly excluded anything containing "test" in setup.py, ahem.

Anyway, it turns out that the python setup.py sdist and bdist_wheel commands can rather easily be convinced to include extra stuff using the package_data parameter AND recursively adding odl/test in MANIFEST.in. However, for python setup.py install, you also have to give the include_package_data=True option (of course, why would I want to install with data when I explicitly give package_data???).

Next issue: conftest.py is not installed, and without it, the extra plugins (--largescale and such) don't work. No test runs since the options in the corresponding files are not recognized. Putting conftest.py one level below didn't work either for a reason I don't remember. Here the solution was to go for the second item here and follow the instructions here to make an entry point for pytest -- which now resides in odl.util.pytest_plugins.py. Here the description was actually very good, and the approach worked immediately.

So now that this is done, I think we need a new release 0.5.1 (sorry about that). I'll put the correct tag into conda/meta.yaml and merge when we agree on the changes.

kohr-h · 2016-10-21T22:31:21Z

Perhaps it's not too bad that Travis CI was down due to connection problems at GitHub (hard to say whose fault), so the new installation doc is not yet deployed. ~~Seems fixed now.~~ No, still problems. Tests fail during apt-get update due to connection timeouts. What the f?

kohr-h · 2016-10-21T22:53:37Z

See Travis CI status

kohr-h · 2016-10-21T23:01:37Z

Another remark: Sudden pytest warnings are not my fault:

pytest-dev/pytest#2005

adler-j · 2016-10-22T11:48:51Z

We need to look into how numby solves this, in the end this really is something we need.

adler-j · 2016-10-22T11:52:27Z

Above comment was me thinking this was a issue ^^.

Anyway nice pull, but does removing conftest.py not screw up running pytest, or does it find it somehow?

adler-j · 2016-10-22T11:53:04Z

setup.py

    package_dir={'odl': 'odl'},
+    package_data={'odl': find_tests()},
+    include_package_data=True,
+    entry_points={'pytest11': ['odl_plugins = odl.util.pytest_plugins']},


Does this make pytest find it? What does 11 mean?

Yes, this is one of the 4 or 5 ways to extend pytest by custom "plugins". The mechanism I used is described here. The name "pytest11" is fixed and magic :-)

adler-j · 2016-10-22T11:54:28Z

How about the pytest.ini file?

adler-j · 2016-10-22T11:55:56Z

doc/source/getting_started/installing.rst

@@ -467,5 +467,5 @@ If that does not help, `make an issue on GitHub <https://github.com/odlgroup/odl
 .. _CVXPY: http://www.cvxpy.org/en/latest/
 .. _odlcuda: https://github.com/odlgroup/odlcuda
 .. _CUDA toolkit: https://developer.nvidia.com/cuda-toolkit
-.. _ASTRA tomography toolbox: https://github.com/astra-toolbox/astra-toolbox
+.. _ASTRA: https://github.com/astra-toolbox/astra-toolbox


This link has to be wrong now?

No, the other one is inline now, I don't see a point in 3 different targets linking to the same page.

adler-j · 2016-10-22T11:57:27Z

setup.py

+        for filename in filenames:
+            basename, suffix = os.path.splitext(filename)
+            if (suffix == '.py' and
+                    (basename.startswith('test') or


It is actually supposed to be _test and test_

See:

http://doc.pytest.org/en/latest/goodpractices.html

Yeah, I had that first, it was correct. Dunno why I changed it.

adler-j · 2016-10-22T11:58:11Z

setup.py

@@ -61,6 +56,23 @@ def run_tests(self):
        errno = pytest.main(self.pytest_args)
        sys.exit(errno)

+
+test_path = os.path.join(root_path, 'odl/test')


os.path.join(root_path, 'odl', test')

Oh, of course.

adler-j · 2016-10-22T12:01:25Z

Looks good except for the minor comments. Are we sure this works? If we we need to make a 5.1 release, perhaps you can look into pypi test (https://wiki.python.org/moin/TestPyPI) before the actual release.

kohr-h · 2016-10-22T18:50:32Z

How about the pytest.ini file?

It can be included as extra package file, however only under odl, not at top repo level (in this case one can specify the file in MANIFEST.in, but it won't be put into the final package since it's counted to metadata).
The price we pay is either a duplicate at top level or that we need to invoke pytest odl in the future, which is also okay IMO.

kohr-h · 2016-10-22T19:46:47Z

Everything is fixed, and I tested all combinations of Python 2/3 and source/wheel/conda packages. Works smoothly.
You can try the packages yourself with

pip install -i https://testpypi.python.org/pypi odl

Good suggestion with the test server.

kohr-h · 2016-10-24T14:16:16Z

Bumpety

adler-j · 2016-10-24T14:17:41Z

Fine by me!

adler-j · 2016-10-24T14:28:05Z

5.1 release comming?

kohr-h · 2016-10-24T14:30:17Z

Done!

adler-j · 2016-10-24T14:36:39Z

Trying install in a VM

adler-j · 2016-10-24T14:39:04Z

Worked perfectly from pip install at least! Roughly 3300 tests, no failures etc. Good work!

kohr-h · 2016-10-24T14:39:53Z

Phew! :-)

Holger Kohr added 3 commits October 22, 2016 00:03

REL: effectively package the tests

e6cbd6b

REL: fix and simplify release script

84eaea5

MAINT: fence fixture definitions in pytest plugin

5d9b302

kohr-h added status: review needed topic: distribution labels Oct 21, 2016

DOC: fix dead links in installing doc

d858f33

adler-j reviewed Oct 22, 2016

View reviewed changes

Holger Kohr added 5 commits October 22, 2016 20:53

REL: include pytest.ini in package

3bab662

TST: rename --documentation pytest option to --doctest-doc

127a73d

TST: add ignores to odl.test() using abs paths in collect_ignore

1254a48

REL: fix small errors test finder in setup.py

09ca27c

BLD: add test suite to meta.yaml test section

38d9bcf

REL: bump version to 0.5.1

9f05097

kohr-h removed the status: review needed label Oct 24, 2016

kohr-h merged commit 9054a0f into master Oct 24, 2016

kohr-h deleted the distribute_tests branch October 24, 2016 14:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Distribute tests #669

Distribute tests #669

kohr-h commented Oct 21, 2016

kohr-h commented Oct 21, 2016 •

edited

Loading

kohr-h commented Oct 21, 2016

kohr-h commented Oct 21, 2016

adler-j commented Oct 22, 2016

adler-j commented Oct 22, 2016

adler-j Oct 22, 2016

kohr-h Oct 22, 2016

adler-j commented Oct 22, 2016

adler-j Oct 22, 2016

kohr-h Oct 22, 2016

adler-j Oct 22, 2016

kohr-h Oct 22, 2016

adler-j Oct 22, 2016

kohr-h Oct 22, 2016

adler-j commented Oct 22, 2016

kohr-h commented Oct 22, 2016

kohr-h commented Oct 22, 2016

kohr-h commented Oct 24, 2016

adler-j commented Oct 24, 2016

adler-j commented Oct 24, 2016

kohr-h commented Oct 24, 2016

adler-j commented Oct 24, 2016

adler-j commented Oct 24, 2016

kohr-h commented Oct 24, 2016

Distribute tests #669

Distribute tests #669

Conversation

kohr-h commented Oct 21, 2016

kohr-h commented Oct 21, 2016 • edited Loading

kohr-h commented Oct 21, 2016

kohr-h commented Oct 21, 2016

adler-j commented Oct 22, 2016

adler-j commented Oct 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adler-j commented Oct 22, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adler-j commented Oct 22, 2016

kohr-h commented Oct 22, 2016

kohr-h commented Oct 22, 2016

kohr-h commented Oct 24, 2016

adler-j commented Oct 24, 2016

adler-j commented Oct 24, 2016

kohr-h commented Oct 24, 2016

adler-j commented Oct 24, 2016

adler-j commented Oct 24, 2016

kohr-h commented Oct 24, 2016

kohr-h commented Oct 21, 2016 •

edited

Loading