Pipenv breaks with pre-existing usage of .venv files

[tgroshon]

Issue description

tl;dr .venv files already exist in some dev setups but when present, pipenv attempts to use that file as a directory and install a virtualenv into it.

Virtualenvwrapper has suggested for awhile to use .venv files in your project directory to hold the name of your virtualenv so that you could automatically activate virtualenvs. Spacemacs (an Emacs editor setup) also does this behavior by default.

Two problems:

1. Pipenv will silently fail to create and activate a virtualenv when a .venv file is present.
2. If a .venv file is added to an existing pipenv project, pipenv breaks because it can't activate the virtualenv at <project_dir>/.venv ignoring the place where it had already created a virtualenv.

Prior Art Links:

• Virtualenvwrapper Tips & Tricks: http://virtualenvwrapper.readthedocs.io/en/latest/tips.html
• Spacemacs Python Layer: https://github.com/syl20bnr/spacemacs/tree/develop/layers/%2Blang/python#automatic-activation-of-local-pyenv-version
• Virtualenv in git directories: https://hmarr.com/2010/jan/19/making-virtualenv-play-nice-with-git/
• Sample .bash_profile using .venv files: justinabrahms/jlilly-bashy-dotfiles@04899f0#commitcomment-1526375

Expected result

At the least, Pipenv should check if .venv is a file or a directory before attempting to automatically use it as a directory and install a virtualenv into it. If it's a file, then behave as if no .venv was found at all.

Actual result

If a .venv file exists in the project directory, Pipenv will attempt to create and use a virtualenv at <project_dir>/.venv and yell at the user a bit because it's not a directory but (a) it won't actually fail and (b) it won't actually save a virtualenv anywhere.

Output:

Creating a virtualenv for this project...
Pipfile: <project_path>\Pipfile
Using <python_path> to create virtualenv...
Running virtualenv with interpreter <python_path>
ERROR: File already exists and is not a directory.
Please provide a different path or delete the file.

Virtualenv location: <project_path>\.venv

Steps to replicate

touch .venv
pipenv install

Also see output of pipenv --venv

$ pipenv --support

REDACTED

Pipenv version: '2018.7.1'

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'nt',
 'platform_machine': 'AMD64',
 'platform_python_implementation': 'CPython',
 'platform_release': '10',
 'platform_system': 'Windows',
 'platform_version': '10.0.17134',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'win32'}

System environment variables:

• ADSK_CLM_WPAD_PROXY_CHECK
• ALIASES
• ALLUSERSPROFILE
• ANSICON
• ANSICON_DEF
• APPDATA
• ARCHITECTURE
• CHOCOLATEYINSTALL
• CHOCOLATEYLASTPATHUPDATE
• CMDER_ROOT
• COMMONPROGRAMFILES
• COMMONPROGRAMFILES(X86)
• COMMONPROGRAMW6432
• COMPUTERNAME
• COMSPEC
• CONEMUANSI
• CONEMUANSILOG
• CONEMUARGS
• CONEMUARGS2
• CONEMUBACKHWND
• CONEMUBASEDIR
• CONEMUBASEDIRSHORT
• CONEMUBUILD
• CONEMUCFGDIR
• CONEMUCONFIG
• CONEMUDIR
• CONEMUDRAWHWND
• CONEMUDRIVE
• CONEMUHOOKS
• CONEMUHWND
• CONEMUPALETTE
• CONEMUPID
• CONEMUSERVERPID
• CONEMUTASK
• CONEMUWORKDIR
• CONEMUWORKDRIVE
• DRIVERDATA
• FPS_BROWSER_APP_PROFILE_STRING
• FPS_BROWSER_USER_PROFILE_STRING
• FSHARPINSTALLDIR
• GIT_INSTALL_ROOT
• GOPATH
• GOROOT
• GTK_BASEPATH
• HOME
• HOMEDRIVE
• HOMEPATH
• LOCALAPPDATA
• LOGONSERVER
• NUMBER_OF_PROCESSORS
• ONEDRIVE
• OS
• PATH
• PATHEXT
• PLINK_PROTOCOL
• PROCESSOR_ARCHITECTURE
• PROCESSOR_IDENTIFIER
• PROCESSOR_LEVEL
• PROCESSOR_REVISION
• PROGRAMDATA
• PROGRAMFILES
• PROGRAMFILES(X86)
• PROGRAMW6432
• PROMPT
• PSMODULEPATH
• PUBLIC
• RUBYOPT
• SESSIONNAME
• SVN_SSH
• SYSTEMDRIVE
• SYSTEMROOT
• TEMP
• TERM
• TMP
• USER-ALIASES
• USERDOMAIN
• USERDOMAIN_ROAMINGPROFILE
• USERNAME
• USERPROFILE
• VERBOSE-OUTPUT
• WINDIR
• WORKON_HOME
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenvûspecific environment variables:

---

Contents of Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[scripts]
server = "python manage.py runserver"
migrate = "python manage.py migrate"
test = "python manage.py test"

[packages]
"boto3" = "==1.7"
dj-database-url = "==0.5.0"
json-logging-py = "*"
"psycopg2" = "==2.7"
pytz = "==2018.4"
requests = "==2.18.4"
Django = "==2.0.6"
django_csp = "==3.4"
"Jinja2" = "==2.10"
djangorestframework = "*"

[dev-packages]
ipython = "*"
pylint = "*"
pylint-django = "*"
pylint-plugin-utils = "*"
autoflake = "*"
importmagic = "*"
epc = "*"

[requires]
python_version = "3.6"

[uranusjr]

I guess one simple way out is to change the exists check to isdir. We can depend on users of this style to not also set PIPENV_VENV_IN_PROJECT since it simply does not make sense for them to work together.

@tgroshon Can you verify that in this configuration, a .venv file would contain the name of the environment you want to activate? I.e. $WORKON_HOME/$(cat .venv) would point me to the virtual environment I want to activate. I’m thinking we might be able to leverage this and make things work even better.

[tgroshon]

@uranusjr Yes I agree with you!

• Attempt to fix by checking isdir and expect users to not also be setting that env var.
• If a .venv directory is present, expect to use it for containing the virtualenv.
• If a .venv file is present, expect for it to contain the name of an environment.

In the future, it may make sense to also allow a .venv file to also contain a path to an environment.

[uranusjr]

I agree with the allow path part :D It should also be relatively simple to do—just detect if the content contains a / or \ (it shouldn’t if it’s an environment name), and treat it as path if it does.

Call out for contributors: This would be a very good feature to work on, since it would also solve a common request about knowing the environment location ahead-of-time, or create the environment in a user-defined location, e.g. #589 #746.