[RFE] Lookup plugin for OCI vault secrets

[ITD27M01]

I would like to suggest adding OCI Vault secrets lookup plugin based on oci_secrets_secret_bundle_facts module.
It is useful to interact with OCI Vault secrets by lookup plugin. The example use case is to store some passwords and sensitive variables like PKI private keys:

---
ansible_password: "{{ lookup('oci_secret', 'SECRED OCID', on_missing='error')}}"
db_password: "{{ lookup('oci_secret', 'SECRED OCID', on_missing='error')}}"

For example there is an AWS lookup plugin for such a service:
https://docs.ansible.com/ansible/latest/collections/amazon/aws/aws_secret_lookup.html

[manojmeda]

@ITD27M01 I understand that the plugin is a little convenient but would like to understand more about the pain points in calling the facts module to get the information.

[ITD27M01]

Hi @manojmeda
I have a couple of cases.

1. The first case is interaction with windows machines. Let's see a routine play:

- name: Run configuration on windows hosts
  hosts: windows
  roles:
    - role
    - some_role

It will fail at start because I must specify the ansible_user and ansible_password variables for winrm connection, of course I can fetch them in the first role:

- name: Run configuration on windows hosts
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - role
    - some_role

But it doesn't help because of gather_facts. And as a workaround I've disabled gather:

- name: Run configuration on windows hosts
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - role
    - some_role

What about my team? I need to leave some conspicuous comment:

- name: Run configuration on windows hosts
  # DO NOT ENABLE FACTS THERE!!! the windows pwd will be retrieved in the first role 
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - role
    - some_role

And what about facts? So, I need to collect them somewhere:

- name: Run configuration on windows hosts
  # DO NOT ENABLE FACTS HERE!!! the windows pwd will be retrieved in the first role 
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - setup # collects facts for windows hosts
    - role
    - some_role

Now this one play works well, but I have to split the playbook into stages because of application logic:

- name: Run configuration on windows hosts
  # DO NOT ENABLE FACTS HERE!!! the windows pwd will be retrieved in the first role 
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - setup # collects facts for windows hosts
    - role
    - some_role

- name: Some preparation play for the other hosts in my Infra
   hosts: some_other_hosts
   roles:
     - some_preparation_role

- name: Run  the second stage of configuration on windows hosts
  hosts: windows
  roles:
    - some_role_second_stage

But... The third play will fail again, because facts are not shared between plays.

- name: Run configuration on windows hosts
  # DO NOT ENABLE FACTS HERE!!! the windows pwd will be retrieved in the first role 
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - setup # collects facts for windows hosts
    - role
    - some_role

- name: Some preparation play for the other hosts in my Infra
   hosts: some_other_hosts
   roles:
     - some_preparation_role

- name: Run  the second stage of configuration on windows hosts
  # DO NOT ENABLE FACTS HERE!!! the windows pwd will be retrieved in the first role 
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - setup # collects facts for windows hosts
    - some_role_second_stage

The workaround of course is the cache and cacheble facts. But the problem is when to invalidate the cache?

2. The second case is a shared secret, the db password, for example.
   In the traditional three tier application I have to include the tasks to retrieve the secret for all the plays:

- name: Configure the DB server
   hosts: db
   roles:
     - fetch_the_db_password_secret_from_vault
     - db_role

- name: Run configuration on windows hosts
  # DO NOT ENABLE FACTS HERE!!! the windows pwd will be retrieved in the first role 
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - setup # collects facts for windows hosts
    - fetch_the_db_password_secret_from_vault
    - role
    - some_role

- name: Configure application servers
   hosts: app
   roles:
     - fetch_the_db_password_secret_from_vault
     - some_preparation_role

- name: Run  the second stage of configuration on windows hosts
  # DO NOT ENABLE FACTS HERE!!! the windows pwd will be retrieved in the first role 
  gather_facts: no
  hosts: windows
  roles:
    - fetch_windows_password_from_vault
    - setup # collects facts for windows hosts
    - some_role_second_stage

- name: Configure web servers
   hosts: web
   roles:
     - fetch_the_db_password_secret_from_vault
     - some_preparation_role

Just compare it with the following:

group_vars/all.yaml:

---
ansible_password: "{{ lookup('oci_secret', 'SECRED OCID') }}"
db_password: "{{ lookup('oci_secret', 'SECRED OCID') }}"

And that's it, I do the job by just two strings instead of crazy playbook above.

I maintain the workaround for this in my OCI ansible collection: https://github.com/ITD27M01/ansible-oci-collection
and have created the plugins for secrets and windows credentials:

https://github.com/ITD27M01/ansible-oci-collection/tree/master/plugins/lookup

[manojmeda]

@ITD27M01 Thanks for providing the detailed description of the use case and pain points. We will review this and let you know for any further questions/updates.

[ITD27M01]

@manojmeda Ok, thank you!

[mtompkins]

This would really be a welcome addition. The overhead for something that is repeatable of this nature would be of great relief.