community.hashi_vault.vault_kv2_get fails if latest version is deleted

[FooBarTrixibell]

SUMMARY

When trying to pull the latest secret in ansible on a playbook which works if the latest version of a secret is still live, if the version has been deleted it will fail with the following -

Invalid or missing path ['SECRET-NAME'] with secret version 'latest'. Check the path or secret version.

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.hashi_vault.vault_kv2_get

ANSIBLE VERSION

ansible [core 2.11.12]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  ansible collection location = /var/lib/awx/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.6.8 (default, Aug 24 2020, 17:57:11) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
  jinja version = 2.10.1
  libyaml = True

COLLECTION VERSION

# /usr/local/lib/python3.6/site-packages/ansible_collections
Collection            Version
--------------------- -------
community.hashi_vault 1.5.0

CONFIGURATION

OS / ENVIRONMENT

Centos 7

STEPS TO REPRODUCE

Create a secret, make a number of versions, delete the latest version.
Create a playbook to pull the latest version of the secret and run it.

---
- hosts: all
  name: Test some vault stuff
  tasks:
    - name: Read the latest version of a kv2 secret from Vault via the remote host with approle auth
      community.hashi_vault.vault_kv2_get:
        path: '{{ inventory_hostname }}'
        url: "{{ansible_hashi_vault_url}}"
        role_id: "{{ansible_hashi_vault_role_id}}"
        secret_id: "{{ansible_hashi_vault_secret_id}}"
        auth_method: "{{ansible_hashi_vault_auth_method}}"
        engine_mount_point: "kv"

      delegate_to: localhost 
      register: response
      # equivalent API path is secret/data/hello
    
    - name: Display the results
      ansible.builtin.debug:
        msg:
          - "Secret: {{ response.secret }}"
          - "Data: {{ response.data }} (contains secret data & metadata in kv2)"
          - "Metadata: {{ response.metadata }}"
          - "Full response: {{ response.raw }}"
          - "Value of key 'password' in the secret: {{ response.secret.<SECRET> }}"

EXPECTED RESULTS

Either -
The data from the deleted version (including deletion_time) passed and a successful status
or
The latest non-deleted version returned (and a successful status)
or
A differect option available for version other than latest, ie. latest-live

ACTUAL RESULTS

The latest version is pulled and the details come across in the JSON output but the status of the run is set to FAILED.

{
  "exception": "Traceback (most recent call last):\n  File \"/tmp/ansible_community.hashi_vault.vault_kv2_get_payload_pkw0u5rb/ansible_community.hashi_vault.vault_kv2_get_payload.zip/ansible_collections/community/hashi_vault/plugins/modules/vault_kv2_get.py\", line 189, in run_module\n  File \"/usr/lib/python3.6/site-packages/hvac/api/secrets_engines/kv_v2.py\", line 85, in read_secret_version\n    params=params,\n  File \"/usr/lib/python3.6/site-packages/hvac/adapters.py\", line 90, in get\n    return self.request('get', url, **kwargs)\n  File \"/usr/lib/python3.6/site-packages/hvac/adapters.py\", line 272, in request\n    utils.raise_for_error(response.status_code, text, errors=errors)\n  File \"/usr/lib/python3.6/site-packages/hvac/utils.py\", line 36, in raise_for_error\n    raise exceptions.InvalidPath(message, errors=errors)\nhvac.exceptions.InvalidPath: {\"request_id\":\"<REDACTED>\",\"lease_id\":\"\",\"renewable\":false,\"lease_duration\":0,\"data\":{\"data\":null,\"metadata\":{\"created_time\":\"2022-10-26T22:06:42.398386929Z\",\"custom_metadata\":null,\"deletion_time\":\"2022-10-27T00:31:25.35393577Z\",\"destroyed\":false,\"version\":4}},\"wrap_info\":null,\"warnings\":null,\"auth\":null}\n",
  "msg": "Invalid or missing path ['<REDACTED>'] with secret version 'latest'. Check the path or secret version.",
  "invocation": {
    "module_args": {
      "path": "<REDACTED>",
      "url": "<REDACTED>",
      "role_id": "<REDACTED>",
      "secret_id": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
      "auth_method": "approle",
      "engine_mount_point": "kv",
      "retry_action": "warn",
      "token_file": ".vault-token",
      "azure_resource": "https://management.azure.com/",
      "proxies": null,
      "ca_cert": true,
      "validate_certs": null,
      "namespace": null,
      "timeout": null,
      "retries": null,
      "mount_point": null,
      "token": null,
      "token_path": null,
      "token_validate": null,
      "username": null,
      "password": null,
      "jwt": null,
      "aws_profile": null,
      "aws_access_key": null,
      "aws_secret_key": null,
      "aws_security_token": null,
      "region": null,
      "aws_iam_server_id": null,
      "azure_tenant_id": null,
      "azure_client_id": null,
      "azure_client_secret": null,
      "cert_auth_private_key": null,
      "cert_auth_public_key": null,
      "version": null
    }
  },
  "warnings": [
  ],
  "_ansible_no_log": false,
  "changed": false,
  "_ansible_delegated_vars": {
    "ansible_host": "localhost",
    "ansible_port": null,
    "ansible_user": "ansible",
    "ansible_connection": "local"
  }
}

[briantist]

Hi @FooBarTrixibell ! Thank you for opening this issue. I started investigating it today, and this is first and foremost an issue in hvac, so I've opened the issue over there:

• Reading a KV v2 secret version that has been deleted raises an InvalidPath exception hvac/hvac#906

Once we decide how to handle it in the hvac library, I will return to this on the collection side and see how we can improve the experience further.

From the options you provided:

1. The data from the deleted version (including deletion_time) passed and a successful status
   or
2. The latest non-deleted version returned (and a successful status)
   or
3. A differect option available for version other than latest, ie. latest-live

I believe we'd end up with 1) being the default behavior once there's support in hvac, and possibly introducing 3) or something like it that would handle iterating down in versions until it found a live one, if one exists.

But we'll see how it plays out.

---

One thing to note is that the next major version (4.0.0) of this collection will probably be released within a week or so, and I'm not sure if this can be addressed before that time.

[briantist]

Just to update this, I do have a PR open in hvac to address it on that side:

• Allow for reading deleted secret versions (kv2) without an exception hvac/hvac#907

Once that gets released, we'll be able to use it within this collection, I just have to give some thought to how we might also accommodate older versions of hvac that don't have the update because it's conditional and requires a specific parameter.