Error for custom/local module

[memelet]

Issue Type

• Bug report

Ansible and Ansible Lint details

$ ansible --version
ansible 2.6.4
  config file = /Users/bkaplan/si/ops/ansible/ansible.cfg
  configured module search path = [u'/Users/bkaplan/si/ops/ansible/modules']
  ansible python module location = /usr/local/Cellar/ansible/2.6.4/libexec/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15 (default, Sep 18 2018, 20:16:18) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)]

$ ansible-lint --version
ansible-lint 3.4.23

• ansible installation method: one of source, pip, OS package
  brew install ansible
• ansible-lint installation method: one of source, pip, OS package
  brew install ansible-lint

Desired Behaviour

I have a custom module, modules/logrotate.py which is used like

    - name: Configure logrotate
      logrotate:
        name: "{{item.name}}"
        path: "{{item.pattern}}"
        options: "{{logrotate_options}}"
      with_items: []

ansible.cfg contains

library = modules

When I run ansible-lint ansible/playbooks/hosts/configure/agent-tunneller.yml I expect the module logrotate to not cause an error.

Actual Behaviour

What I get instead is

❯ ansible-lint ansible/playbooks/hosts/configure/agent-tunneller.yml                                                 ops/git/master !
Couldn't parse task at ansible/playbooks/hosts/configure/agent-tunneller.yml:258 (no action detected in task. This often indicates a misspelled module name, or incorrect module path.

The error appears to have been in '<unicode string>': line 258, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

(could not open file to display line))
{ u'logrotate': { '__file__': 'ansible/playbooks/hosts/configure/agent-tunneller.yml',
                  '__line__': 260,
                  u'name': u'{{item.name}}',
                  u'options': u'{{logrotate_options}}',
                  u'path': u'{{item.pattern}}'},
  u'name': u'Configure logrotate',
  u'with_items': []}

Where it seems that ansible-lint does not detect custom modules.

[flamein]

It seems to do this for any module it doesn't know about, like gcp_container_cluster which is in core since 2.6

[willthames]

@flamein It should definitely work with any modules that are in the core version of ansible it's run with (otherwise it wouldn't work with any modules) - is somehow ansible-lint using a different version of ansible?

@memelet I must confess I haven't tested with out-of-core modules for a while - I'll flag this as a bug and see what we can do to investigate further

[flamein]

Disregard, dropping the pip install --user installed ansible-lint== 2.5.2 fixes this...

[danihodovic]

@willthames Any updates on this? I'm facing the same problem for local modules.

[ssbarnea]

In fact there are cases where modules are installed by various python dependencies and ansible-lint will also choke on these ones.

Maybe it would be much easier to allow us to whitelist ansible modules, so the linter will assume that they exists and skip checking them.

[ssbarnea]

I think that a way to whitelist missing modules could also be seen as a very useful feature. I am facing lots of errors like this on zuul jobs which can use the zuul_return module. That module comes from zuul itself, which is huge to install, not to mention that it does not even install on all platforms.

The reality is that the linted playbooks would run only under zuul, where the module is available.

Still, we do not want to prevent people from linting locally, where they do not have (or want to install >100mb complex dependency).

@pabelanger ^

[marcjay]

I'm seeing this issue as well (v4.3.4). We rely on the following documented behaviour:

To use a local module only in certain playbooks:

store it in a sub-directory called library in the directory that contains the playbook(s)

https://docs.ansible.com/ansible/latest/dev_guide/developing_locally.html

i.e. our structure is:

playbook.yml
library/some_module.py

and this 'just works' with ansible-playbook, but errors as above with ansible-lint. I can workaround this by setting the env variable: ANSIBLE_LIBRARY=library however we use pre-commit so this is a little tricky (I think we might need to create a wrapper shell script to export this and run ansible-lint)

[ssbarnea]

The linter will not fail if ansible can find the modules, closing as is not an issue but a documented behavior. Please use https://github.com/ansible/ansible-lint/discussions for support.

[marcjay]

I'm sorry but that's not the case here. Ansible finds the module fine, no issues. This is because of the behaviour I quoted. ansible-lint fails, unless you specify that env variable or add library = library to ansible.cfg

This is a valid bug and difference between ansible and ansible-lint. If you want me to put together a minimal test case I can. I did not encounter this on 4.2.0, but did after upgrading to 4.3.4

[marcjay]

@ssbarnea please find a minimal reproduction of the issue below:

$ ansible --version
ansible 2.8.6
  config file = None
  configured module search path = ['/Users/xxx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/xxx/yyy/.venv/lib/python3.7/site-packages/ansible
  executable location = /Users/xxx/yyy/.venv/bin/ansible
  python version = 3.7.8 (v3.7.8:4b47a5b6ba, Jun 27 2020, 04:47:50) [Clang 6.0 (clang-600.0.57)]

$ ansible-lint --version
ansible-lint 4.3.4

$ tree
.
├── library
│   └── baz.py
├── playbook.yml
└── roles
    └── foo
        └── tasks
            └── main.yml

$ cat library/baz.py
#!/usr/bin/python

from ansible.module_utils.basic import *

def main():

    fields = {
        "url": {"type": "str", "required": True},
    }

    module = AnsibleModule(argument_spec=fields)
    response = {"hello": "world"}
    module.exit_json(changed=False, meta=response)


if __name__ == '__main__':
    main()

$ cat playbook.yml
- hosts: localhost
  roles:
    - foo

$ cat roles/foo/tasks/main.yml
- name: bar
  baz:
    url: 'foo'

Running ansible-lint gives the error:

 $ ansible-lint -vvvv
DEBUG    Logging initialized to level 10
DEBUG    Options: Namespace(colored=True, config_file=None, display_relative_path=True, exclude_paths=[], format='rich', listrules=False, listtags=False, parseable=False, parseable_severity=False, playbook=[], quiet=False, rulesdir=[], skip_list=[], tags=[], use_default_rules=False, verbosity=4, warn_list=[])
DEBUG    Loading rules from /Users/xxx/yyy/.venv/lib/python3.7/site-packages/ansiblelint/rules
INFO     Discovering files to lint: git ls-files *.yaml *.yml
INFO     Unknown file type: roles/foo/tasks/main.yml
INFO     Found roles: roles/foo
INFO     Found playbooks: playbook.yml
DEBUG    Examining roles/foo/tasks/main.yml of type tasks
CRITICAL Couldn't parse task at /Users/xxx/yyy/roles/foo/tasks/main.yml:1 (no action detected in task. This often indicates a misspelled module name, or incorrect module path.)
{ 'baz': { '__file__': '/Users/xxx/yyy/roles/foo/tasks/main.yml',
           '__line__': 3,
           'url': 'foo'},
  'name': 'bar'}

It seems to be an issue with the auto-detection - if I pass in playbook.yml, it's fine (which seems different to the original question, so maybe this should be a new issue):

$ ansible-lint playbook.yml -vvvv
DEBUG    Logging initialized to level 10
DEBUG    Options: Namespace(colored=True, config_file=None, display_relative_path=True, exclude_paths=[], format='rich', listrules=False, listtags=False, parseable=False, parseable_severity=False, playbook=['playbook.yml'], quiet=False, rulesdir=[], skip_list=[], tags=[], use_default_rules=False, verbosity=4, warn_list=[])
DEBUG    Loading rules from /Users/xxx/yyy/.venv/lib/python3.7/site-packages/ansiblelint/rules
DEBUG    Examining playbook.yml of type playbook
DEBUG    Examining roles/foo/tasks/main.yml of type tasks

Just to show that ansible has no issue with this:

$ ansible-playbook playbook.yml
[WARNING]: No inventory was parsed, only implicit localhost is available

[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'


PLAY [localhost] ***************************************************************************************************************************************************************************************************************************

TASK [Gathering Facts] *********************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [foo : bar] ***************************************************************************************************************************************************************************************************************************
ok: [localhost]

PLAY RECAP *********************************************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Also to note, this error does not occur in v4.2.0, but exists in v4.3.0+

[jjblack]

@ssbarnea

The linter will not fail if ansible can find the modules, closing as is not an issue but a documented behavior. Please use https://github.com/ansible/ansible-lint/discussions for support.

Did you really close this issue because "it works in ansible so it must work in this"? I'm sorry but this isn't the case. There are 4 users in this thread that have reported that they are seeing this bug. And I am another one verifying this is the case. Please reopen this and at least recognize that this is a bug.

[craigers521]

bump