Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use azure_rm_aduser_info or azure_rm_adgroup_info: Insufficient privileges to complete the operation. #573

Closed
l3ender opened this issue Jul 3, 2021 · 26 comments
Labels
has_pr PR fixes have been made medium_priority Medium priority

Comments

@l3ender
Copy link
Contributor

l3ender commented Jul 3, 2021

SUMMARY

I am unable to use the azure_rm_adgroup_info or azure_rm_aduser_info modules. They report:

"msg": "failed to get ad group info Insufficient privileges to complete the operation."

I am using a service principal which has Owner access for the subscription and looks like it has sufficient API access for user information:

Screen Shot 2021-07-03 at 2 17 15 PM

ISSUE TYPE
  • Bug Report
COMPONENT NAME

azure_rm_adgroup_info
azure_rm_aduser_info

ANSIBLE VERSION
-> ansible --version
ansible [core 2.11.2]
  config file = /Users/ross/repos/azure-config/ansible.cfg
  configured module search path = ['/Users/ross/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/ross/repos/azure-config/venv/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/ross/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/ross/repos/azure-config/venv/bin/ansible
  python version = 3.9.1 (default, Dec 10 2020, 10:36:35) [Clang 12.0.0 (clang-1200.0.32.27)]
  jinja version = 3.0.1
  libyaml = True
COLLECTION VERSION
-> ansible-galaxy collection list azure.azcollection

# /Users/ross/repos/azure-config/venv/lib/python3.9/site-packages/ansible_collections
Collection         Version
------------------ -------
azure.azcollection 1.7.0

# /Users/ross/.ansible/collections/ansible_collections
Collection         Version
------------------ -------
azure.azcollection 1.7.0
CONFIGURATION
-> ansible-config dump --only-changed
DISPLAY_SKIPPED_HOSTS(/Users/ross/repos/azure-config/ansible.cfg) = False
OS / ENVIRONMENT

Mac OS Big Sur 11.4.

STEPS TO REPRODUCE

I am testing using the following playbook:

---
- name: "Test loading AD group info"
  hosts: localhost
  connection: local
  gather_facts: false

  tasks:
    - name: "Get group info."
      azure.azcollection.azure_rm_adgroup_info:
        tenant: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
        all: true
      register: group_info
EXPECTED RESULTS

I would be able to retrieve results of groups in the tenant.

ACTUAL RESULTS

Module fails with error.

-> ansible-playbook sandbox.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Test loading AD group info] *******************************************************************************************************************************************************

TASK [Get group info.] ******************************************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "failed to get ad group info Insufficient privileges to complete the operation."}

PLAY RECAP ******************************************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Reference: #423.

@l3ender
Copy link
Contributor Author

l3ender commented Jul 3, 2021

@coleneubauer - any ideas?

@l3ender l3ender changed the title Unable to use azure_rm_adgroup_info: Insufficient privileges to complete the operation. Unable to use azure_rm_aduser_info or azure_rm_adgroup_info: Insufficient privileges to complete the operation. Jul 3, 2021
@Fred-sun
Copy link
Collaborator

Fred-sun commented Jul 5, 2021

@coleneubauer - any ideas?

@l3ender Thank you for submitting this issue and we will find out the cause as soon as possible.

@l3ender
Copy link
Contributor Author

l3ender commented Jul 7, 2021

From my research, I've found the Azure Active Directory Graph API permission Directory.Read.All is required to use the modules azure_rm_adgroup_info and azure_rm_aduser_info. Once I added these under the "API Permissions" section of the app registration (service principal), the two modules worked as expected.

This brings up another question, however: when adding the permission, I see the following warning:

This application is using Azure AD Graph API, which is on a deprecation path. Starting June 30th, 2020 we will no longer add any new features to Azure AD Graph API. We strongly recommend that you upgrade your application to use Microsoft Graph API instead of Azure AD Graph API to access Azure Active Directory resources. Learn more

Should a separate issue be opened for this collection using the deprecated APIs?

@Fred-sun
Copy link
Collaborator

Fred-sun commented Jul 8, 2021

@l3ender We are planning to upgrade all the APIs, the old ones will no longer be used, so we can ignore the old ones. Thank you very much!

@Fred-sun Fred-sun added the medium_priority Medium priority label Jul 8, 2021
@l3ender
Copy link
Contributor Author

l3ender commented Jul 8, 2021

We are planning to upgrade all the APIs, the old ones will no longer be used, so we can ignore the old ones.

Thank you! When this is done, please note it in the release notes as it is a breaking change in which a different API permission will need to be granted for account/service principal.

Thanks!

@l3ender l3ender closed this as completed Jul 8, 2021
@l3ender l3ender reopened this Jul 8, 2021
@l3ender
Copy link
Contributor Author

l3ender commented Jul 8, 2021

Apologies for the accidental close! I will keep open as perhaps a documentation PR should be added for these modules.

@Fred-sun
Copy link
Collaborator

@l3ender Can you set "auth_source: cli" in the playbook to retry? The current 'ad' related modules only support CLI Credentials (az login). Thank you very much!

@Fred-sun Fred-sun added the work in In trying to solve, or in working with contributors label Aug 13, 2021
@l3ender
Copy link
Contributor Author

l3ender commented Aug 14, 2021

@Fred-sun I was able to make the modules work with a credentials file. The issue was that the service principal I was using didn't have the necessary permissions (it seems they aren't granted by default). Once I configured the SP with the above API access, our playbook worked as expected with a credentials file.

I kept the issue open because it seems good to add notes in documentation (either for the modules, authentication setup, or both).

Thank you!

@johnpetersjr
Copy link

Sorry for attaching to this old issue - can create a new one if desired -- I use AWX to run all my ansible modules, and use an attached credential type of "Microsoft Azure Resource Manager" that has my Service Principal ID/Secret to authenticate with Azure. From reading comments above, it seems I need to set

auth_source: cli

in my task, which I've never had to do before. This is now causing me grief in the azure_rm_adgroup module task, because it:

Could not retrieve credential from local cache for service principal

Likely because I'm using a remote machine to run this job, which doesn't 'hold' onto my credentials from play to play in the local venv.

Ideas on how to solve this? Without specifying the auth_source on other azcollection modules, I have no issues building Azure Cloud infrastructure. I simply set these env vars and everything works fine:

  vars:
    # Necessary ENV Variables for Azure Resource Manager Credentials
    client_id: "{{ lookup('env', 'AZURE_CLIENT_ID') }}"
    secret: "{{ lookup('env', 'AZURE_SECRET') }}"
    subscription_id: "{{ lookup('env', 'AZURE_SUBSCRIPTION_ID') }}"
    tenant: "{{ lookup('env', 'AZURE_TENANT') }}"

But it seems this particular azure_rm_adgroup module doesn't work that way... Has anyone gotten this module to work from AWX, using a Service Principal Credential?

@l3ender
Copy link
Contributor Author

l3ender commented Apr 12, 2022

@johnpetersjr Sorry for the slow response. I can't speak exactly to using azure_rm_adgroup, but we are using azure_rm_adgroup_info from Ansible Tower without issue. We don't need to specify auth_source in the playbook nor specify any vars with environment lookups. We simply bind the "Microsoft Azure Resource Manager" credential to the job template and it works as expected.

I'm not sure if it matters, but our job template is configured to run on a "localhost" inventory, i.e. the same node as where Tower is installed.

Hope that helps!

@radhikari-arch
Copy link

radhikari-arch commented Apr 13, 2022

Is auth_source. msi supported for these azure.azcollection.azure_rm_adgroup_info and azure.azcollection.azure_rm_adgroup modules? From the dcoumentaiton, it sure says msi for auth-source. I am getting "failed to get ad group info Access Token missing or malformed." error.
cc: @Fred-sun @l3ender @coleneubauer

@radhikari-arch
Copy link

radhikari-arch commented Apr 14, 2022

I gave the global administrator role in Azure AD for the MSI virtual machine. But still getting that error "failed to get ad group info Access Token missing or malformed."

- name: Return a specific group using object_id
  azure.azcollection.azure_rm_adgroup_info:
    object_id: *****
    tenant: xxx-xxx-xx
    auth_source: msi
  environment:
    AZURE_SUBSCRIPTION_ID: xxx-xxx--xxx
    AZURE_TENANT: xxx-xxxx-xxx-

Versions:

ansible 2.9.27
python version = 3.6.8
azure.azcollection = v1.12.0

Error:

The full traceback is:
  File "/tmp/ansible_azure.azcollection.azure_rm_adgroup_info_payload_i6sayeof/ansible_azure.azcollection.azure_rm_adgroup_info_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_adgroup_info.py", line 236, in exec_module
  File "/usr/local/lib/python3.6/site-packages/azure/graphrbac/operations/groups_operations.py", line 472, in get
    raise models.GraphErrorException(self._deserialize, response)
fatal: [localhost]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "ad_user": null,
            "adfs_authority_url": null,
            "all": false,
            "api_profile": "latest",
            "attribute_name": null,
            "attribute_value": null,
            "auth_source": "msi",
            "cert_validation_mode": null,
            "check_membership": null,
            "client_id": null,
            "cloud_environment": "AzureCloud",
            "log_mode": null,
            "log_path": null,
            "object_id": "xxx-xxx-xxx-xxx",
            "odata_filter": null,
            "password": null,
            "profile": null,
            "return_group_members": false,
            "return_member_groups": false,
            "return_owners": false,
            "secret": null,
            "subscription_id": null,
            "tenant": "xxx-xx-xxx-x"
        }
    },
    "msg": "failed to get ad group info Access Token missing or malformed."
}

Howevere, with the same tenant id and subs, using the msi, create resource module works fine:

- name: Create resource group 
  azure_rm_resourcegroup:
    name: "My-testing"
    location: eastus2
    auth_source: msi
  environment:
    AZURE_SUBSCRIPTION_ID:  xxxxxx
    AZURE_TENANT: xxxxxxxx

@radhikari-arch
Copy link

radhikari-arch commented Apr 19, 2022

Well, I found a way to add the Directory.Read.All permissions for the MSI from here:
https://docs.microsoft.com/en-us/azure/app-service/scenario-secure-app-access-microsoft-graph-as-app?tabs=azure-cli#grant-access-to-microsoft-graph
However, even with that still gave the same error: "failed to get ad group info Access Token missing or malformed."
I then created an App Registration and gave the Directory.Read.All access on this as well. For this, I got an error: 'ADGroup' object has no attribute 'mail_nickname'
Looks like API has some limitations! Also @Fred-sun mentioned above, they are planning to upgrade all APIs. Checking to see if there is any updates on this.

@Fred-sun
Copy link
Collaborator

Because all the apis need to be upgraded, it's a big project that we're working on. Thank you very much!

@Fred-sun
Copy link
Collaborator

@l3ender @radhikari-arch It has support mscrosoft gaph in PR #1112, Please review! Thank you very much!

@felixmarch
Copy link

I tried the codes on Fred-sun:Add_msgrpah_support, it works great 👍 👍

---
- hosts: localhost
  gather_facts: false
  tasks:
  - name: Query all user infos
    azure.azcollection.azure_rm_msuser_info:
      all: true
    register: azure_rm_aduser_info
  - debug: var=azure_rm_aduser_info
# ansible-playbook azure_playbook.yml
...
...
PLAY RECAP ***************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

@Fred-sun
Copy link
Collaborator

@felixmarch Thanks for your test! I will push to merge as soon as possible!

@Fred-sun Fred-sun added has_pr PR fixes have been made and removed work in In trying to solve, or in working with contributors labels Mar 26, 2023
@felixmarch
Copy link

Hi @Fred-sun I tried azure_rm_msuser to add new user like this

---
- hosts: localhost
  gather_facts: false
  tasks:
  - name: Add Users
    azure.azcollection.azure_rm_msuser:
      user_principal_name: "[email protected]"
      display_name: "xxx xxx"
      account_enabled: True
      mail_nickname: "xxx xxx"
      password_profile:
        Password: Secret1111!!!!
      mail: "[email protected]"
      state: "present"

During the run, I got error like this:

# ansible-playbook adduser_azure_playbook.yml -vvvvvv
...
...
1406658 1679888825.94367: ^ failed state is now: HOST STATE: block=2, task=3, rescue=0, always=0, run_state=ITERATING_COMPLETE, fail_state=FAILED_TASKS, pending_setup=False, tasks child state? (None), rescue child state? (None), always child state? (None), did rescue? False, did start at task? False
1406658 1679888825.94374: getting the next task for host localhost
1406658 1679888825.94379: host localhost is done iterating, returning
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py", line 102, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py", line 94, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py", line 40, in invoke_module
    runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_msuser', init_globals=None, run_name='__main__', alter_sys=True)
  File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 283, in <module>
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 279, in main
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 174, in __init__
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 473, in __init__
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 243, in exec_module
  File "/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py", line 267, in user_to_dict
KeyError: 'id'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1679888822.5047784-1406714-26876904145436/AnsiballZ_azure_rm_msuser.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible_collections.azure.azcollection.plugins.modules.azure_rm_msuser', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 283, in <module>\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 279, in main\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 174, in __init__\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py\", line 473, in __init__\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 243, in exec_module\n  File \"/tmp/ansible_azure.azcollection.azure_rm_msuser_payload_eaatphjl/ansible_azure.azcollection.azure_rm_msuser_payload.zip/ansible_collections/azure/azcollection/plugins/modules/azure_rm_msuser.py\", line 267, in user_to_dict\nKeyError: 'id'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
1406658 1679888825.94493: no more pending results, returning what we have
1406658 1679888825.94502: results queue empty
...
...

Any steps I miss here? 🤔

@Fred-sun
Copy link
Collaborator

@felixmarch I added a judgment on the creation failure and returned an error message, please try again! Thank you very much!

1 similar comment
@Fred-sun
Copy link
Collaborator

@felixmarch I added a judgment on the creation failure and returned an error message, please try again! Thank you very much!

@felixmarch
Copy link

Cool, able to identify the error detail now. 👍 👍

# ANSIBLE_STDOUT_CALLBACK=yaml ansible-playbook adduser_azure_playbook.yml -vvvvvv
...
...
'Create ad user fail, Msg {''code'': ''Request_BadRequest'', ''message'': "Invalid value specified for property ''mailNickname'' of resource ''User''.", ''details'': [{''code'': ''InvalidValue'', ''message'': "Invalid value specified for property ''mailNickname'' of resource ''User''.", ''target'': ''mailNickname''}], ''innerError'': {''date'': ''2023-03-27T09:32:38'', ''request-id'': ''c503367b-17aa-4b0d-ba66-1d7c94755161'', ''client-request-id'': ''3042f0ca-a323-44a4-8bd8-6f7de2d9c797''}}'
...
...

Further check, it was caused by unsupported characters in mailNickname, similar to what reported in this forum

After removing the offending chars, it works fine then.

Also observed certain attributes like given_name, surname, usage_location were no longer supported.

    "msg": "Unsupported parameters for (azure.azcollection.azure_rm_msuser) module: given_name, surname, usage_location Supported parameters include: account_enabled, ad_user, adfs_authority_url, api_profile, auth_source, cert_validation_mode, ct, display_name, log_mode, log_path, mail, mail_nickname, object_id, password, password_profile, profile, secret, state, subscription_id, tenant, thumbprint, user_principal_name, x509_certificate_path"

It may be good in future to have custom attributes similar to what is provided in the community.windows.win_domain_user module. 😊

@felixmarch
Copy link

Is this "give_name" typos? (not "given_name"?) 🤔

@felixmarch
Copy link

felixmarch commented Mar 27, 2023

I noticed the update logic (the "should_update" flag in azure_rm_aduser.py) was not implemented yet in azure_rm_msuser.py.

So, when I tried to update the "display_name" against existing "user_principal_name", I got error like this:

"msg": "Create ad user fail, Msg {'code': 'Request_BadRequest', 'message': 'Property netId is invalid.', 'details': [{'code': 'GenericError', 'message': 'Property netId is invalid.', 'target': 'netId'}], 'innerError': {'date': '2023-03-27T10:08:01', 'request-id': '6fdf7664-0de8-4809-9a8b-db2a8d9ab683', 'client-request-id': 'd50ac719-3fae-4d7f-b8e7-f92d0489716f'}}"

Hopefully the incoming work will address it 😊

@Fred-sun
Copy link
Collaborator

@felixmarch Thanks for your advice, The spelling errors has update. This module does not support updating for the time being. If there is a good way, it will be improved. Thanks!

@Fred-sun
Copy link
Collaborator

commit as bellow!

2fb6a68

@Fred-sun
Copy link
Collaborator

@l3ender azure-graphrbac has beed deprecated, and migrate to msgraph-sdk. It has support in v2.1.0. I will closed it! If you have any question. Please repoen a new! Thank you very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
has_pr PR fixes have been made medium_priority Medium priority
Projects
None yet
Development

No branches or pull requests

5 participants