make test-unit failing with ImportError: cannot import name 'ensure_text'

[nchammas]

Describe the bug

I'm trying to run unit tests locally as part of my work on #2120. I'm hitting the error shown below.

Steps To Reproduce

Create an empty test.env and run make test-unit.

Expected behavior

I expect the unit tests to run successfully on the latest checkout of dev/barbara-gittings.

Screenshots and log output

$ make test-unit
Unit test run starting...
Starting dbt_database_1 ... done
.tox create: /usr/app/.tox/.tox
ERROR: invocation failed (exit code 1), logfile: /usr/app/.tox/.tox/log/.tox-0.log
================================= log start =================================
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/usr/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/usr/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/__init__.py", line 3, in <module>
    from .run import cli_run
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/run/__init__.py", line 5, in <module>
    from ..config.cli.parser import VirtualEnvConfigParser
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/config/cli/parser.py", line 6, in <module>
    from ..ini import IniConfig
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/config/ini.py", line 8, in <module>
    from virtualenv.dirs import default_config_dir
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/dirs.py", line 5, in <module>
    from virtualenv.util.lock import ReentrantFileLock
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/util/lock.py", line 11, in <module>
    from virtualenv.util.path import Path
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/util/path/__init__.py", line 5, in <module>
    from ._sync import copy, copytree, ensure_dir, link, symlink
  File "/usr/local/lib/python3.6/dist-packages/virtualenv/util/path/_sync.py", line 7, in <module>
    from six import PY2, PY3, ensure_text
ImportError: cannot import name 'ensure_text'

================================== log end ==================================
ERROR: InvocationError for command /usr/bin/python3 -m virtualenv --no-download --python /usr/bin/python3 .tox (exited with code 1)
        1.64 real         0.37 user         0.22 sys
make: *** [test-unit] Error 2

System information

Which database are you using dbt with?

• postgres

The output of dbt --version:

$ dbt --version
installed version: 0.16.0-b1
   latest version: 0.15.2

Your version of dbt is ahead of the latest release!

The operating system you're using: macOS

The output of python --version: Python 3.7.6

[beckjake]

It looks like something terrible is happening with six and tox when docker-compose tries to build the container now.

[nchammas]

Perhaps we need to add a requirement somewhere to ensure that six >= 1.12.0 ?

https://github.com/benjaminp/six/blob/3a3db7510b33eb22c63ad94bc735a9032949249f/CHANGES#L33-L40

[beckjake]

That's one path. I'm also looking at locking down tox/virtualenv versions in the dockerfile to a low enough version that everything plays well with ubuntu's dist-packages behavior (because at least I know that will work). virtualenv==20.0.0 is implicated here, somehow... see this issue, for example: tox-dev/tox#1516

[nchammas]

Linked from that issue is a PR making a new tox release that pins six as I suggested. So perhaps bumping up to tox == 3.14.4 would also work, though that would be a big jump from the current requirement.

https://github.com/fishtown-analytics/dbt/blob/4e58589afdbb5e74701753e239536e318c5d8db7/dev_requirements.txt#L6

[beckjake]

It's a bit more involved than that, but yeah - I've got a fix going locally that seems adequate.

[beckjake]

@nchammas let me know if the dockerfile/image in #2135 works for you. Sorry for the mess here, this situation sucks :(

[nchammas]

Fixed by #2135. Needed to clear my cached images and rebuild from scratch. make test-unit now works.