Ansible-lint does not look for ansible binary inside venv

[noonedeadpunk]

Summary

When ansible-lint is installed inside venv, it is expected that ansible-lint will try to find ansible binary inside venv as well, even when venv is not activated. Instead it looks inside $PATH only.
This results in the following error, when you have ansible 2.9 installed by system packages, and ansible 2.10 along with ansible-lint inside the venv:
ERROR FATAL: Ansible CLI (2.9.6) and python module (2.10.7) versions do not match. This indicates a broken execution environment.

Issue Type

• Bug Report

Ansible and Ansible Lint details

$ ~/.virtualenvs/test-lint/bin/ansible-lint --version
ansible-lint 5.0.6 using ansible 2.9.6
FATAL: Ansible CLI (2.9.6) and python module (2.10.7) versions do not match. This indicates a broken execution environment.

$ ~/.virtualenvs/test-lint/bin/ansible --version
ansible 2.10.7
  config file = /home/dmitriy/.ansible.cfg
  configured module search path = ['/home/dmitriy/.ansible/roles/config_template/library', '/home/dmitriy/.ansible/plugins/library', '/home/dmitriy/.ansible/roles/ceph-ansible/library']
  ansible python module location = /home/dmitriy/.virtualenvs/test-lint/lib/python3.8/site-packages/ansible
  executable location = /home/dmitriy/.virtualenvs/test-lint/bin/ansible
  python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]

• ansible installation method: pip
• ansible-lint installation method: pip

OS / ENVIRONMENT

Python 3.8.5
Ubuntu 20.04

STEPS TO REPRODUCE

$ virtualenv ~/.virtualenvs/test-lint/
$ ~/.virtualenvs/test-lint/bin/pip install ansible-lint[community,yamllint]
$ ~/.virtualenvs/test-lint/bin/ansible-lint --version
ansible-lint 5.0.6 using ansible 2.9.6
FATAL: Ansible CLI (2.9.6) and python module (2.10.7) versions do not match. This indicates a broken execution environment.
$ source ~/.virtualenvs/test-lint/bin/activate
(test-lint) $ ~/.virtualenvs/test-lint/bin/ansible-lint --version
ansible-lint 5.0.6 using ansible 2.10.7
(test-lint) $

Desired Behaviour

It is expected to search for ansible binary inside venv, even when venv is not activated

Actual Behaviour

Ansible binary is searched only inside actual $PATH

[stefan-as]

This is also a problem, when there is an Ansible within the virtualenv only and no system Ansible at all:

  File "/usr/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'ansible'

[briantist]

Additionally, ansible-lint itself, when run from the virtual environment seems to be using the version of itself outside.

I ran pip install --upgrade ansible-lint in the venv, got:

Successfully installed ansible-lint-5.1.2

(along with the FATAL error up near the top of the installation, but nothing that actually seemed to fail)

When I run ansible-lint --version within the venv though, it still shows the old one from the system:

ansible-lint 5.0.12 using ansible 2.11.2
FATAL: Ansible CLI (2.11.2) and python module (2.9.22) versions do not match. This indicates a broken execution environment.

[ssbarnea]

Your ansible installation is broken and linter detects that.

[briantist]

@ssbarnea in what way is my ansible installation (which works perfectly in a venv like every other python application) broken, while ansible-lint, which cannot work in a venv, is not broken?

[ssbarnea]

Read https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#upgrading-ansible-with-pip well, and make use of pip in order to see which ansible packages you have installed there, you will find the problem.

[briantist]

I am aware of that issue, but I did not have ansible or ansible-base installed. It did indirectly help me though, in that I deleted and recreated the venv anyway, and things are working now. Please understand that responses like "your stuff is broken." are not very helpful.

tl;dr for anyone else

Try recreating your virtual environment. If you're using virtualenv try adding --no-site-packages when you create it. For venv (what I was using) that's the default, so don't add --system-site-packages.

[markus-as]

@briantist --system-site-packages is the default since virtualenv 20.0.0 ( see pypa/virtualenv#1681 and readthedocs/readthedocs.org#6725 (comment) )

[webknjaz]

Read docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#upgrading-ansible-with-pip well, and make use of pip in order to see which ansible packages you have installed there, you will find the problem.

I think the described issue is different because it hits an executable installed externally and available on the $PATH. I would expect that it'd compare the dist metadata vs what's the importable package reports maybe. Like checking f'{sys.executable} -m pip show ansible' (-core/base) or maybe comparing the importable with what importlib_metadata reports.

[webknjaz]

Reiterating on what was discussed on a call: using software installed a virtualenv does not require one to "activate" it in their shell via . venv/bin/activate. activate is a convenience script that exports a bunch of env vars into interactive shells. It sets $PATH/$PYTHONPATH to point to dirs in the venv layout.
But it is also common to run the executables from venv directly, especially in non-development environments. This is done via venv/bin/script-name. And such invocations do not populate the env vars that the activate script does. But all the pip installed "console_scripts" are tied to their venv in a different manner — they have an absolute interpreter path in their shebangs, like #! /a/full/absolute/path/to/venv/bin/ making the respective /a/full/absolute/path/to/venv/lib/python*/site-packages available automatically. Still, since there's no $PATH set up, bare invocations of binaries like subprocess.check_call('ansible') will end up hitting other (system-wide) paths.
This could be addressed by doing something like subprocess.check_call(f'{sys.executable.rpartition("/")[0]}/ansible') which would hit the Ansible install within the same venv.

[sblask]

This is a weird one! So ansible-lint does not use the ansible from the venv, but at the same time it does not work without ansible being installed in the venv:

$ ansible-lint ansible.yml
ERROR    No module named 'ansible'
FATAL: ansible-lint requires a version of Ansible package >= 2.9, but none was found. Please install a compatible version using the same python interpreter. See https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html#installing-ansible-with-pip

The version should be fine though:

$ ansible --version
ansible [core 2.11.2]

When I install ansible into the venv I get this version:

$ ./bin/ansible --version
ansible [core 2.11.5]

Which leads to:

$ ansible-lint --version
ansible-lint 5.1.3 using ansible 2.11.2
FATAL: Ansible CLI (2.11.2) and python module (2.11.5) versions do not match. This indicates a broken execution environment.

However, there is no problem at all when using ansible-lint as pre-commit hook:

  - repo: https://github.com/ansible-community/ansible-lint
    rev: v5.1.3
    hooks:
      - id: ansible-lint

Even though the installed packages are identical:

ansible-lint-venv $ ./bin/pip freeze
ansible==4.5.0
ansible-core==2.11.5
ansible-lint==5.1.3
bracex==2.1.1
cffi==1.14.6
colorama==0.4.4
commonmark==0.9.1
cryptography==3.4.8
enrich==1.2.6
Jinja2==3.0.1
MarkupSafe==2.0.1
packaging==21.0
pycparser==2.20
Pygments==2.10.0
pyparsing==2.4.7
PyYAML==5.4.1
resolvelib==0.5.4
rich==10.9.0
ruamel.yaml==0.17.16
ruamel.yaml.clib==0.2.6
tenacity==8.0.1
wcmatch==8.2

pre-commit-venv $ ./bin/pip freeze
ansible==4.5.0
ansible-core==2.11.5
ansible-lint @ file:///Users/sebastian/.cache/pre-commit/repopn0zt_fi
bracex==2.1.1
cffi==1.14.6
colorama==0.4.4
commonmark==0.9.1
cryptography==3.4.8
enrich==1.2.6
Jinja2==3.0.1
MarkupSafe==2.0.1
packaging==21.0
pathspec==0.9.0
pycparser==2.20
Pygments==2.10.0
pyparsing==2.4.7
PyYAML==5.4.1
resolvelib==0.5.4
rich==10.9.0
ruamel.yaml==0.17.16
ruamel.yaml.clib==0.2.6
tenacity==8.0.1
wcmatch==8.2
yamllint==1.26.3

[webknjaz]

That's because of the difference between doing imports vs. subprocess invocations.

[sblask]

Yeah, I saw that both are used. Probably not a good idea to mix them, but pre-commit seems to do something that makes it work.

[andreyzhelnin-st]

I'm using this straightforward workaround:
export PATH=~/.virtualenvs/test-lint/bin:$PATH

[rjeffman]

Today I hit this issue. I had ansible-lint installed on the system, but had ansible-core (2.12.1) only installed in the virtual environment (python 3.9.9, venv).

Once I removed the global ansible-lint, and installed it in the virtual environment it started to work again.

[Sispheor]

It would be nice to be able to configure the ansible bin path to use. Because with pipx we can manage multiple version of ansible by adding a suffix. The current path seems to look for "ansible" bin only.

pipx list
venvs are in /home/nico/.local/pipx/venvs
apps are exposed on your $PATH at /home/nico/.local/bin
   package ansible 2.9.14, Python 3.8.5
    - ansible
    - ansible-config
    - ansible-connection
    - ansible-console
    - ansible-doc
    - ansible-galaxy
    - ansible-inventory
    - ansible-playbook
    - ansible-pull
    - ansible-test
    - ansible-vault
   package ansible 4.9.0 (ansible-4), Python 3.8.5
    - ansible-4
    - ansible-config-4
    - ansible-connection-4
    - ansible-console-4
    - ansible-doc-4
    - ansible-galaxy-4
    - ansible-inventory-4
    - ansible-lint
    - ansible-playbook-4
    - ansible-pull-4
    - ansible-test-4
    - ansible-vault-4
   package docker-compose 1.28.4, Python 3.8.5
    - docker-compose
   package pipenv 2020.11.15, Python 3.8.5
    - pipenv
    - pipenv-resolver

I have 2 Ansible installed in 2 different venv

which ansible  
/home/nico/.local/bin/ansible
which ansible-4 
/home/nico/.local/bin/ansible-4

[ssbarnea]

@Sispheor we will not make this configurable, basically because the only case where ansible-lint would really work is when it is installed inside the same env as ansible, it not only calls ansible but also imports ansible modules at runtime. Mixing two ansible installations would a recipe for pain.

You are welcome to test the current patch and report if it works as you expect.

[bendem]

previous dumb comment

I just went through the discussion of this issue and related issues and I'm wondering if it wouldn't be simpler to call ansible executables using the pythonic venv aware way of ${sys.executable} -m ansible[-playbook] <...args>. This would always pick the version of ansible that came as a dependency of ansible-lint, no matter what the venv situation is.

I've checked my assumptions and I was wrong, the correct way to call ansible and always hit the correct version of ansible would be to call the main method (ansible.cli.adhoc.main or ansible.cli.playbook.main), but I'm guessing this doesn't play nicely with parsing output and whatnot.

[ssbarnea]

@bendem Ansible does not have an official supported way to be called using Python, that is why we call it as subprocess. Still, there is work to make this a supported way, once we get confirmation from core team that we can use it, we will do it as it will also introduce serious speed improvements, saving ~2s per call/file.