Running and bundling tests with namespace packages

[theacodes]

(Context: PR #2223 towards #1978)

PR #2223's tests can't run because nose will place cwd into sys.path when running tests. This means import google.cloud will import ./google/cloud instead of {env}/lib/python{x}.{y}/site-packages/google/cloud. This will clobber all other google namespace packages. There is an long-open issue here, note that Nose has been largely abandoned.

There are a two of ways we could resolve this, and maybe we should do both:

1. Move the tests to their own top-level package as described in the py.test docs.
2. Switch to py.test's test runner, pytest. It's fully compatible with nose/unittest style tests. py.test appropriately deals with namespace/installed packages.

On switching to py.test

1. py.test has slightly different arguments than nose, notably for coverage. This is pretty easy to deal with.
2. Switching to py.test will either mean we have to switch to having the tests in a separate package or that we have to switch from tox to nox. The reason is that py.test will try to import relative to the test modules it discovers, and so will try to import the module-under-test from the current working directory, but will actually give an error because the same module is present in site-packages. This can be resolved by either moving the tests to a separate package or installing the package in "editable" mode with pip install -e. Unfortunately, tox doesn't easily allow you to change the command used to install the package under test. There might be a few hacks (turning off sdist and manually installing the package). Nox however gives you complete control over the creation, here's a simple config that works for gcloud-python.

On moving the tests to a test package

The were originally concerns about having the tests in a separate top-level tests package because we didn't want it to get installed with gcloud-python but we did want to distribute the tests. This should be resolvable by telling find_packages() to skip tests and including tests in manifest.in. This means that wheels and sdists will contain the tests folder for downstream packagers, but pip will not install the tests package into site-packages.

[tseaver]

In https://github.com/GoogleCloudPlatform/google-cloud-python-happybase, I moved the google package into a non-package src directory to resolve this issue. I would prefer that to either of these alternatives.

[dhermes]

@tseaver The fact that the nose project is dead is a signal we should move to py.test. As for moving unit tests, I don't care where they live. I just sent #2227 to move them into a directory called unit_tests so in some sense it'll be more obvious where they live than it currently is.

[theacodes]

@tseaver I'm not super thrilled about src/google and using package_dir, but I'm willing to be fine with it as a temporary solution to get the namespace change unblocked.

[dhermes]

@jonparrott I've already got the test move done and after that the py.test change should be straightforward

[theacodes]

👍 , though I'd love to have @tseaver on board. :)

[tseaver]

The src/ prefix directory is in common use among other namespace-package projects (the zope.* packages all use it).

[dhermes]

@tseaver I'm not sure how src/ fixes the issue?

[theacodes]

@dhermes it side-steps the issue of:

tests can't run because nose will place cwd into sys.path

By putting them under src doing import google.cloud won't look at src/google/cloud.

[dhermes]

I just tried that in #2223 and it made the failures worse (tox -e py27 still breaks with google namesapce and now .tox/py27/bin/nosetests also broken)

[tseaver]

@dhermes Check what I did in google-cloud-python-happybase: the tests are all happy running from inside src, and the namespace package is correct.

[dhermes]

@tseaver Thanks for the pointer. My ambiguous "I just tried that" above meant that I copied what you had done there. Unclear on where the failure is.

[dhermes]

@jonparrott This is "resolved"

[theacodes]

👍