Resource IDs containing spaces can't be consumed by other az commands

[mattcuk]

I've been trying to use a command to list all the Logic App IDs which have an Alert set up for them. To do this, I'm running 2 commands; az resource list, and az resource show. However, since az resource list returns IDs that contain spaces, az resource show will then fail.

Command Name
az resource show --ids $(az resource list --resource-type Microsoft.Insights/metricAlerts --query [].id --output tsv) --query properties.scopes

Errors:
az resource: error: argument --ids: invalid ResourceId value: 'failed'

To Reproduce:

Create an alert against any Logic App.. make sure the Alert Name has a space in it, like 'logic-app-test-la failed'.

Run the command above.

Expected Behavior

It should really be able to handle the encoding of the spaces so that az resource show can ingest the result of az resource list without having to resort to a Bash script to work round the problem.

Environment Summary

Linux-4.15.0-1075-azure-x86_64-with-debian-stretch-sid
Python 3.6.5

azure-cli 2.2.0 *

Extensions:
interactive 0.4.4

Additional Context

[yonzhan]

add to S169

[yonzhan]

resource

[zhoxing-ms]

Hi @mattcuk, I tried to use the resource name logic-app-test-la failed, but I didn't reproduce your problem.
Could you please print out the result of the execution of command az resource list --resource-type Microsoft.Insights/metricAlerts --query [].id --output tsv, and show it to me?

[e14mattc]

Sure.. here's a sample of some of the results.. I've removed the subscription ID + changed some of the names a bit..

/subscriptions/eec/resourceGroups/eg-app-rg-009/providers/microsoft.insights/metricalerts/eg-ecomm-azuresphererouting-failures
/subscriptions/eec/resourceGroups/eg-app-rg-009/providers/microsoft.insights/metricalerts/eg-ecomm-azuresphereupdateexcel-failure-alerts
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-extendedrange-init-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-persistent-complete-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-persistent-init-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/eg-persistentlog-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/expiredocs-la failed
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/persistentkeyval-datafactory
/subscriptions/eec/resourceGroups/eg-app-transkeyvalue-rg-019/providers/microsoft.insights/metricalerts/wuk-pf-persistent-imports-la failed
/subscriptions/eec/resourceGroups/eg-int-cmtyautomation-rg-012/providers/microsoft.insights/metricalerts/eg-ecomm-cmty-page-speed-logger-la failed
/subscriptions/eec/resourceGroups/eg-int-cmtyautomation-rg-012/providers/microsoft.insights/metricalerts/eg-ecomm-funcapp-restart-to-msteams-la failed

Hope this helps.

[zhoxing-ms]

@e14mattc Hi, I created some new alerts based on these examples, and the problem still can't be reproduced.
Could you please add --debug to the original command? And then please show me the information returned, thanks~

[e14mattc]

I'm going to email you direct @zhoxing-ms .. since I'm loathed to paste in production output here. Hope that's ok.

[zhoxing-ms]

@e14mattc OK, thanks~ Has the email been sent? I don't seem to have received it

[e14mattc]

@zhoxing-ms yes, I sent it at 10am BST yesterday. I wonder if it got stuck in a spam filter due it having a .txt file attachment? The email address I used was your MS account which you have on your GitHub profile. I'll send another with the output in the email itself, rather than an attachment.

[zhoxing-ms]

@e14mattc The email has been received, but I still can't reproduce your question.
Could you please execute az resource show --ids $(az resource list --resource-type Microsoft.Insights/metricAlerts --query [].id --output tsv) --debug and show me the error message returned?

[e14mattc]

Sure.. an email is winging its way to you.

For anyone reading along, the output I just emailed shows that the alert ID is being split up on the space.. the word 'failed' should be part of the alert ID;

[zhoxing-ms]

I can reproduce the problem currently: there is no problem with this command executing in PowerShell of Windows, it only occurs in the Shell of Linux.
It seems to be a usage problem of $() in the Shell of Linux, the result of $(az resource list XXX) obtained in Shell is separated by the space in ResourceID. This situation is like creating a file with a filename contains spaces, and then executing echo $(ls), the filename containing spaces in the printed result is split into multiple names due to spaces.
So it's not a problem related to CLI, we can't deal with it. @e14mattc Do you have any questions about it?

[e14mattc]

Couldn't the az resource list XXX command encode spaces in any resource IDs? That would solve the problem, wouldn't it?

[zhoxing-ms]

Hi @e14mattc, In fact, this issue is not caused by encoding, but by the subexpression operator $() in Bash handling the result as a string.
For example:

When executing $(az resource list XXX) in PowerShell returns results as an array, so --ids is passed as an array to the CLI and can be parsed normally.

But executing $(az resource list XXX) in Bash returns results as a string, and the elements between arrays in the results are also separated by spaces. CLI parses the string value of --ids to convert the parameter to an array with a space as the separator, so the spaces in the resourceId will affect the parsing of the resourceId.

If you have a good idea, welcome to communicate with us~

[e14mattc]

How many Azure resources allow a space in the ID, I wonder? Alerts are the only one I've come across so far. Feels like they should maybe restrict this to stop the user entering spaces & potentially running across this issue.

If the az resource list XXX command encoded any IDs that contained spaces with a %20 (maybe?) wouldn't this fix the problem & not break any existing Bash/Powershell scripts?

The code below is how I've worked around the issue, but it's not elegant & it's pretty slow. Is there a better way of dealing with this problem?

alerts=$(az resource list --output tsv --resource-type "Microsoft.Insights/metricAlerts" --query [].id)
alerts=${alerts// /SPACEFIX}

for alert in $alerts
do
        az resource show --ids "${alert//SPACEFIX/ }" --query properties.scopes[0] --output tsv
done

[zhoxing-ms]

How many Azure resources allow a space in the ID, I wonder? Alerts are the only one I've come
across so far. Feels like they should maybe restrict this to stop the user entering spaces & potentially running across this issue.

This requires the service team to see why the resource id allows spaces in some resources.

If the az resource list XXX command encoded any IDs that contained spaces with a %20 (maybe?) wouldn't this fix the problem & not break any existing Bash/Powershell scripts?

Yes, this is a solution, and the service team needs to evaluate whether it is breaking change and whether it can be modified~
By the way, if this problem is not easy to solve at the rest API, CLI can consider adding a parameter for customers to encode the space.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @armleads-azure.

[navba-MSFT]

@mattcuk @e14mattc Apologies for the delayed response. Could you please let me know if you are still facing this issue ? Awaiting your reply.

[e14mattc]

Apologies for the delayed response. Could you please let me know if you are still facing this issue ? Awaiting your reply.

Yes @navba-MSFT if I run the command in my OP it still fails with the same error.

[ghost]

Hi, we're sending this friendly reminder because we haven't heard back from you in a while. We need more information about this issue to help address it. Please be sure to give us your input within the next 7 days. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

[mattcuk]

Yes @navba-MSFT if I run the command in my OP it still fails with the same error.

[emilwester-optimizely]

bump @zhoxing-ms & @msftbot

[shanep2300]

I'm bumping this as well. Need to do a FULL sweep of all our resources and az resource list is bringing in the "hidden" resources from the portal, and most of them have spaces.

[thiDucTran]

same issue when trying to use az monitor metrics alert show --ids

az version
{
  "azure-cli": "2.51.0",
  "azure-cli-core": "2.51.0",
  "azure-cli-telemetry": "1.1.0",
  "extensions": {
    "aks-preview": "0.5.122",
    "azure-devops": "0.25.0",
    "bastion": "0.2.3",
    "ssh": "1.1.3"
  }
}

[schreibe72]

I have a similar problem. The spaces in the IDs are also a problem here. I have worked around the problem in Bash as follows:

az resource list -g example-westeurope-dev-rg --query "[?type=='Microsoft.AlertsManagement/prometheusRuleGroups'].id" --output tsv

#OUTPUTS:
#
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/Prometheus Recommended Pod level Alerts - example-aks
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/Prometheus Recommended Cluster level Alerts - example-aks
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/NodeRecordingRulesRuleGroup-Win-example-aks
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/NodeRecordingRulesRuleGroup-example-aks
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/UXRecordingRulesRuleGroup-Win - example-aks
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/NodeAndKubernetesRecordingRulesRuleGroup-Win-example-aks
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/UXRecordingRulesRuleGroup - example-aks
# /subscriptions/000000-0000-0000-0000-00000000/resourceGroups/example-westeurope-dev-rg/providers/Microsoft.AlertsManagement/prometheusRuleGroups/KubernetesRecordingRulesRuleGroup-example-aks

As you can see, there are spaces in the IDs. As shown in the previous posts, this is a problem with the $() operator. Therefore, I tried to work around the problem by explicitly converting the output into an array. This then looks like this. It works for me.

RESOURCES_=$(az resource list -g example-westeurope-dev-rg --query "[?type=='Microsoft.AlertsManagement/prometheusRuleGroups'].id" --output tsv | tr "\n" ",")
IFS=',' read -r -a RESOURCES <<< "$RESOURCES_"
az resource show --ids ${RESOURCES[@]}