Unable to create service principal in Azure using ansible collection #500

anikm1987 · 2021-04-19T17:44:39Z

SUMMARY

ISSUE TYPE

Bug Report

COMPONENT NAME

azure.azcollection.azure_rm_adserviceprincipal

ANSIBLE VERSION

ansible 2.9.6
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]

CONFIGURATION

OS / ENVIRONMENT

Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal

STEPS TO REPRODUCE

Configured environment variables -

export AZURE_CLIENT_ID="<client_id>"
export AZURE_SECRET=""
export AZURE_TENANT="tenant_id"

Then run command -
ansible-playbook create-azure-rg-vars.yml --extra-vars "subscription_id=<subscription_id>"

---
- hosts: localhost
  connection: local
  vars:
    azure_app_id: "e0a62513-1d81-480e-a6dc-5c99cdd58d9a"
  tasks:
    - name: Create resource group
      azure_rm_resourcegroup:
        name: ansible-AppResources-DevTest
        location: eastus2
        tags:
          ApplicationName  : "AnsibleTest"
        state: present
        append_tags: no 
      register: rg
      environment:
        AZURE_SUBSCRIPTION_ID: "{{ subscription_id }}"
    - debug:
        var: rg
    - debug:
        var: rg.state.id
    
    - name: Generate password
      set_fact:
        password: "{{ azure_app_id | password_hash('sha512') }}"
        tenant_id: "{{ lookup('env', 'AZURE_TENANT') }}"
        azure_client_id: "{{ lookup('env', 'AZURE_CLIENT_ID') }}"
 
    - name: Create app Service Principal
      azure.azcollection.azure_rm_adserviceprincipal:
        app_id: "{{ azure_app_id }}"
        state: present
        tenant: "{{ tenant_id }}"
      register: azure_output
      environment:
        AZURE_SUBSCRIPTION_ID: "{{ subscription_id }}"

    - name: Set SPN password
      azure.azcollection.azure_rm_adpassword:
        app_id: "{{ azure_app_id }}"
        service_principal_object_id: "{{ azure_output.object_id }}"
        state: present
        value: "{{ password }}"
        environment:
          AZURE_SUBSCRIPTION_ID: "{{ subscription_id }}"

    - debug:
        var: azure_output

EXPECTED RESULTS

ansible-playbook create-azure-rg-vars.yml --extra-vars "subscription_id=<subscription_id>" -vvvvv

This should create the resource group and one service principal with password.

ACTUAL RESULTS


TASK [Create app Service Principal] ************************************************************************************************************************
task path: /mnt/c/pce/2021/github_repos/pce-hosting-portal-awx/samples/create-azure-rg-vars.yml:35
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ubuntu
<127.0.0.1> EXEC /bin/sh -c 'echo ~ubuntu && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1618853901.74323-121896268505388 `" && echo ansible-tmp-1618853901.74323-121896268505388="` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1618853901.74323-121896268505388 `" ) && sleep 0'
Using module_utils file ansible_collections/azure
Using module_utils file ansible_collections/azure/azcollection
Using module_utils file ansible_collections/azure/azcollection/plugins
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/basic.py
Using module_utils file ansible_collections/azure/azcollection/plugins/module_utils
Using module_utils file ansible_collections
Using module_utils file ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common_ext
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/__init__.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/parsing/__init__.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/_text.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/parsing/convert_bool.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/text/__init__.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/file.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/_collections_compat.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/sys_info.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/_json_compat.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/parameters.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/text/formatters.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/text/converters.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/six/__init__.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/_utils.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/pycompat24.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/validation.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/process.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/distro/__init__.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/distro/_distro.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/collections.py
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/common/dict_transformations.py
Using module_utils file ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common
Using module_utils file /usr/lib/python3/dist-packages/ansible/module_utils/ansible_release.py
Using module file /home/ubuntu/.ansible/collections/ansible_collections/azure/azcollection/plugins/modules/azure_rm_adserviceprincipal.py
<127.0.0.1> PUT /home/ubuntu/.ansible/tmp/ansible-local-9546gm2zzpnz/tmp8ptm8ojq TO /home/ubuntu/.ansible/tmp/ansible-tmp-1618853901.74323-121896268505388/AnsiballZ_azure_rm_adserviceprincipal.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1618853901.74323-121896268505388/ /home/ubuntu/.ansible/tmp/ansible-tmp-1618853901.74323-121896268505388/AnsiballZ_azure_rm_adserviceprincipal.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'AZURE_SUBSCRIPTION_ID=1f913984-b1c8-4187-b297-415b71d6d278 /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1618853901.74323-121896268505388/AnsiballZ_azure_rm_adserviceprincipal.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1618853901.74323-121896268505388/ > /dev/null 2>&1 && sleep 0'

The full traceback is:
  File "/tmp/ansible_azure.azcollection.azure_rm_adserviceprincipal_payload_a0bjw075/ansible_azure.azcollection.azure_rm_adserviceprincipal_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_adserviceprincipal.py", line 153, in create_resource
  File "/home/ubuntu/.local/lib/python3.8/site-packages/azure/graphrbac/operations/service_principals_operations.py", line 87, in create
    raise models.GraphErrorException(self._deserialize, response)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "api_profile": "latest",
            "app_id": "e0a62513-1d81-480e-a6dc-5c99cdd58d9a",
            "app_role_assignment_required": null,
            "auth_source": "auto",
            "cert_validation_mode": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "password": null,
            "profile": null,
            "secret": null,
            "state": "present",
            "subscription_id": null,
            "tenant": "<tenant_id>"
        }
    },
    "msg": "Error creating service principle, app id e0a62513-1d81-480e-a6dc-5c99cdd58d9a - Access Token missing or malformed."
}

The text was updated successfully, but these errors were encountered:

paultaiton · 2021-04-20T00:13:36Z

It looks like you're all over the place in terms of providing credentials for authentication. Are you using a service principal's client and secret, a AD user's name and password, or something else? You should pick one of MSI, az-cli login, environmental variables, or explicit parameter in playbook's task definition, and stick with it exclusively for the entirety of the playbook.

anikm1987 · 2021-04-20T09:45:53Z

Hi @paultaiton,

I am using service principal to authenticate with Azure. Environment variables are set using below commands -
export AZURE_CLIENT_ID="<client_id>"
export AZURE_SECRET=""
export AZURE_TENANT="tenant_id"

I have not provided credentials in any other place.

FYI - Resource group is getting created successfully. Having problem with new service principal creation.

paultaiton · 2021-04-20T16:21:44Z

You should also include your subscription id in the environmental variables if that's the method you're using. I assume you've redacted the AZURE_SECRET value for this post, but are still providing it on your system?

anikm1987 · 2021-04-20T17:24:46Z

Hi @paultaiton ,
There are number of subscriptions available (dynamic in nature) so I pass it via extra-vars

ansible-playbook create-azure-rg-vars.yml --extra-vars "subscription_id=<subscription_id>"

Subscription id passed as environment variable for each task using -

environment:
AZURE_SUBSCRIPTION_ID: "{{ subscription_id }}"

Yes AZURE_SECRET value I have redacted for the post.

paultaiton · 2021-04-20T18:41:14Z

@anikm1987
You will need to pass all authentication attributes in at the task level then. You cannot mix partial environmentals and partial module parameters to create a complete set, you must pass a complete set of one and nothing in the other forms. At least I've never seen it working, and based on what I've seen of the code that performs the evaluations, it won't compose the disparate forms.

I personally make use of a variable file (I use host_vars/localhost.yml, but you can use something else ) with vaulted strings for principal_id, principal_secret, tenant_guid, and then pass subscription_guid as an extra-vars. Then I pass --ask-vault-password to my ansible-playbook command.

anikm1987 · 2021-04-20T19:27:19Z

@paultaiton

For testing purpose I set subscription also as environment variable .

export AZURE_CLIENT_ID="<client_id>"
export AZURE_SECRET=""
export AZURE_SUBSCRIPTION_ID="<subscription_id>"
export AZURE_TENANT="tenant_id"

Changed the yaml file like below -

hosts: localhost
connection: local
vars:
azure_app_id: "e0a62513-1d81-480e-a6dc-5c99cdd58d9a"
tasks:
- name: Create resource group
  azure_rm_resourcegroup:
  name: ansible-AppResources-DevTest
  location: eastus2
  tags:
  ApplicationName : "AnsibleTest"
  state: present
  append_tags: no
  register: rg
- debug:
  var: rg
- debug:
  var: rg.state.id
- name: Generate password
  set_fact:
  password: "{{ azure_app_id | password_hash('sha512') }}"
  tenant_id: "{{ lookup('env', 'AZURE_TENANT') }}"
- name: Create app Service Principal
  azure.azcollection.azure_rm_adserviceprincipal:
  app_id: "{{ azure_app_id }}"
  state: present
  tenant: "{{ tenant_id }}"
  auth_source: env
  register: azure_output
- debug:
  var: azure_output

Run the command - ansible-playbook create-azure-rg-vars-v1.yml -vvv

Still getting the same error

TASK [Create app Service Principal] ************************************************************************************************************************
task path: /mnt/c/pce/2021/github_repos/pce-hosting-portal-awx/samples/create-azure-rg-vars-v1.yml:35
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: ubuntu
<127.0.0.1> EXEC /bin/sh -c 'echo ~ubuntu && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "echo /home/ubuntu/.ansible/tmp/ansible-tmp-1618946474.079732-12273314591918" && echo ansible-tmp-1618946474.079732-12273314591918="echo /home/ubuntu/.ansible/tmp/ansible-tmp-1618946474.079732-12273314591918" ) && sleep 0'
Using module file /home/ubuntu/.ansible/collections/ansible_collections/azure/azcollection/plugins/modules/azure_rm_adserviceprincipal.py
<127.0.0.1> PUT /home/ubuntu/.ansible/tmp/ansible-local-19219_oo7mjq1/tmpkl1hn2hn TO /home/ubuntu/.ansible/tmp/ansible-tmp-1618946474.079732-12273314591918/AnsiballZ_azure_rm_adserviceprincipal.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1618946474.079732-12273314591918/ /home/ubuntu/.ansible/tmp/ansible-tmp-1618946474.079732-12273314591918/AnsiballZ_azure_rm_adserviceprincipal.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1618946474.079732-12273314591918/AnsiballZ_azure_rm_adserviceprincipal.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1618946474.079732-12273314591918/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
File "/tmp/ansible_azure.azcollection.azure_rm_adserviceprincipal_payload_x9kp53jm/ansible_azure.azcollection.azure_rm_adserviceprincipal_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_adserviceprincipal.py", line 153, in create_resource
File "/home/ubuntu/.local/lib/python3.8/site-packages/azure/graphrbac/operations/service_principals_operations.py", line 87, in create
raise models.GraphErrorException(self._deserialize, response)
fatal: [localhost]: FAILED! => {
"changed": false,
"invocation": {
"module_args": {
"ad_user": null,
"adfs_authority_url": null,
"api_profile": "latest",
"app_id": "e0a62513-1d81-480e-a6dc-5c99cdd58d9a",
"app_role_assignment_required": null,
"auth_source": "env",
"cert_validation_mode": null,
"client_id": null,
"cloud_environment": "AzureCloud",
"password": null,
"profile": null,
"secret": null,
"state": "present",
"subscription_id": null,
"tenant": "tenant_id"
}
},
"msg": "Error creating service principle, app id e0a62513-1d81-480e-a6dc-5c99cdd58d9a - Access Token missing or malformed."
}

paultaiton · 2021-04-20T19:37:26Z

It looks like this module is repurposing the "tenant" property that is normally only used for authentication purposes to be used as the target of the AD action. When I created the azure_rm_subscription_info module, I had to create a completely different variable for "id" and not reuse the subscription_id parameter to avoid a conflict. My guess is that having "tenant" defined as a module property is making the code think that it should find the details as parameters and not look at the environment, but I'm not sure what's going on since I'm not familiar with this module's code, and am not an expert on the general authentication process.

@Fred-sun and @haiyuazhang are listed as the authors of this module, so hopefully they'll have a better answer for you.

My only other suggestion is to pass all of the parameters to the module explicitely: tenant, subscription_id, client_id, secret.

anikm1987 · 2021-04-20T19:49:53Z

Thanks @paultaiton
I have tried passing all of the parameters to the module explicitely: tenant, subscription_id, client_id, secret. It is throwing the similar error.

Hi @Fred-sun @haiyuazhang ,
I try to delete the service principal using below block (service principal does not exist actually) -

- name: delete ad service principal
  azure.azcollection.azure_rm_adserviceprincipal:
    app_id: "{{ azure_app_id }}"
    tenant: "{{ tenant_id }}"
    state: absent

This block executed successfully without any error but new service principal creation block failed again.

Question reagrding the app_id, the value I mentioned in the yaml var section is just an arbitrary uuid value. Hope that is fine.

anikm1987 · 2021-04-21T19:48:53Z

Hi,

Further investigation shows below log in the error -

File "/home/ubuntu/.local/lib/python3.8/site-packages/azure/graphrbac/models/graph_error_py3.py", line 45, in init\n super(GraphErrorException, self).init(deserialize, response, 'GraphError', *args)\n File "/home/ubuntu/.local/lib/python3.8/site-packages/msrest/exceptions.py", line 184, in init\n response.raise_for_status()\nAttributeError: 'ClientRequest' object has no attribute 'raise_for_status'\n",
"module_stdout": "401\n<Response [401]>\nhttps://graph.windows.net/<tenant_id>/servicePrincipals\n401\n<Response [401]>\n",
"msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
"rc": 1
}

anikm1987 · 2021-04-21T20:07:56Z

I have verified the permission the service principal has and it looks like below -

Do I need to provide any other permission?

haiyuazhang · 2021-05-07T19:03:10Z

@paultaiton @anikm1987
#525 should fix this issue.
I've test locally, works. The SP I used for the test has the role 'cloud application administrator'.

Fred-sun · 2021-06-10T13:31:42Z

#525 has been merged, This Issue can be closed! Thank you very much!

Fred-sun closed this as completed Jun 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to create service principal in Azure using ansible collection #500

Unable to create service principal in Azure using ansible collection #500

anikm1987 commented Apr 19, 2021 •

edited

Loading

paultaiton commented Apr 20, 2021

anikm1987 commented Apr 20, 2021 •

edited

Loading

paultaiton commented Apr 20, 2021

anikm1987 commented Apr 20, 2021

paultaiton commented Apr 20, 2021

anikm1987 commented Apr 20, 2021 •

edited

Loading

paultaiton commented Apr 20, 2021 •

edited

Loading

anikm1987 commented Apr 20, 2021 •

edited

Loading

anikm1987 commented Apr 21, 2021

anikm1987 commented Apr 21, 2021

haiyuazhang commented May 7, 2021

Fred-sun commented Jun 10, 2021

Unable to create service principal in Azure using ansible collection #500

Unable to create service principal in Azure using ansible collection #500

Comments

anikm1987 commented Apr 19, 2021 • edited Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

paultaiton commented Apr 20, 2021

anikm1987 commented Apr 20, 2021 • edited Loading

paultaiton commented Apr 20, 2021

anikm1987 commented Apr 20, 2021

paultaiton commented Apr 20, 2021

anikm1987 commented Apr 20, 2021 • edited Loading

paultaiton commented Apr 20, 2021 • edited Loading

anikm1987 commented Apr 20, 2021 • edited Loading

anikm1987 commented Apr 21, 2021

anikm1987 commented Apr 21, 2021

haiyuazhang commented May 7, 2021

Fred-sun commented Jun 10, 2021

anikm1987 commented Apr 19, 2021 •

edited

Loading

anikm1987 commented Apr 20, 2021 •

edited

Loading

anikm1987 commented Apr 20, 2021 •

edited

Loading

paultaiton commented Apr 20, 2021 •

edited

Loading

anikm1987 commented Apr 20, 2021 •

edited

Loading