Incorrect output of the az network vnet subnet list-available-ips command

[subhasishdc]

I was checking the number of available ip addresses in the subnet where an application gateway is deployed using this:

schowdhu@schowdhu-mn1 ~ % az network vnet subnet list-available-ips --resource-group aag-rg --vnet-name aagVnet -n AAG1Subnet This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus [ "10.0.36.244", "10.0.36.245", "10.0.36.246", "10.0.36.247", "10.0.36.248" ]

Following shows my subnet size and the app gw instances deployed.

[schowdhu@schowdhu-ld3 ~]$ az network vnet subnet show -g aag-rg -n AAG1Subnet --vnet-name aagVnet | jq '.addressPrefix' "10.0.36.240/28"

[schowdhu@schowdhu-ld3 ~]$ az network application-gateway list -g aag-rg | jq '.[] | .name,.sku,.operationalState' "aag01" { "capacity": 8, "name": "Standard_Medium", "tier": "Standard" } "Running"

Based on the app gw documentation (https://docs.microsoft.com/en-us/azure/application-gateway/configuration-infrastructure#size-of-the-subnet), app gw uses one private IP address per instance, plus another private IP address if a private front-end IP is configured. Azure also reserves five IP addresses in each subnet for internal use: the first four and the last IP addresses. Taking the above example for reference, total available ip addresses in the subnet 10.0.36.240/28 is 16. Azure uses 5 ip addresses and app gw has 8 instances. There should be only 3 ip addresses remaining. However, the az network vnet subnet list-available-ips command shown above shows 5 ip addresses available.

https://learn.microsoft.com/en-us/cli/azure/network/vnet/subnet?view=azure-cli-latest#az-network-vnet-subnet-list-available-ips -- This is the example in the documentation. It mentions "List some available ips in the subnet.". Does it mean it shows free ip addresses which are not used yet in the subnet or just it just display a few ip addresses belonging to the subnet irrespective of weather they are being used or not.

Am I missing something here or is there an issue with what the command is showing.

[yonzhan]

@necusjz for awareness

[RakeshMohanMSFT]

#13853 can you take a look at this?

[ghost]

Thank you for your feedback. This has been routed to the support team for assistance.

[subhasishdc]

@RakeshMohanMSFT : Thank you for the link. This leads me to another question. Does the option list-available-ips list free available ip addresses in the subnet or just any IP address ( irrespective of weather its being used or not). If its the first option, then the issue I mentioned above holds true and needs to be investigated. If its the second option, then we need to update the documentation/help for the same.

[navba-MSFT]

@subhasishdc Thanks for reaching out to us and reporting this issue. We are looking into this issue and we will provide an update.

[navba-MSFT]

@subhasishdc The portal shows how many available IP address exist in each subnet.

Did you check if this number in portal matched what you got with CLI command response ?

We can get the list of available Ips from Get-AzVirtualNetworkUsageList PS cmdlet as well, by subtracting the CurrentValue (no of IPs being used) from the Limit (total no of available IPs)

Example:

Get-AzVirtualNetworkUsageList -ResourceGroupName rgB -Name vnetB

Name : Subnet size and usage
Id : /subscriptions/XXXXX-XXXXX-XXXXX-XXX/resourceGroups/rgB/providers/Microsoft.Network/virtualNetworks/vnetB/subnets/default
CurrentValue : 1
Limit : 251
Unit : Count

Name : Subnet size and usage
Id : /subscriptions/XXXXX-XXXXX-XXXXX-XXX/resourceGroups/rgB/providers/Microsoft.Network/virtualNetworks/vnetB/subnets/subnet1
CurrentValue : 0
Limit : 27
Unit : Count

Could you please try and check if the above command helps and returns the required value ?

Also while debugging this further, I see that the REST API invoked by this CLI command is CheckIPAddressAvailability Rest API.

The response (output) returned here is 'Private IP address availability and list of other free addresses if the requested one is not available'. Hope this helps.

[subhasishdc]

@navba-MSFT : Checked the portal. Under available ip's, portal shows availability dependent on dynamic use. What does this mean? It is a V1 SKU application gateway and we do not have auto scaling enabled anywhere.

[navba-MSFT]

@subhasishdc While I get you the answer for the availability dependent on dynamic use message, May I know if you had any update on the second action plan of using Get-AzVirtualNetworkUsageList PS cmdlet mentioned in my above comment ?

[navba-MSFT]

@subhasishdc The availability dependent on dynamic use means the subnet is being used by resource where the number of instances deployed are not fixed. Sending a sample screenshot:

In the screenshot, you can see its a gateway subnet. We can have gateways with multiple instances based on requirement and SKU. So it will be either a VPN gateway or Express Route gateway using this.

Another example could be, if that subnet belongs to app gateway which internally uses VMSS and they keep scaling up and down. So you need to investigate which Azure resource is using that subnet. Hope this answers.

[navba-MSFT]

@subhasishdc I wanted to do quick follow-up to check if you had a chance to look at my above comments. Please let us know if you had any updates on this. Awaiting your reply.

[subhasishdc]

@navba-MSFT : I will get back next week on this.

[navba-MSFT]

@subhasishdc Thanks for your reply. I shall wait for the updates from you.

[subhasishdc]

@navba-MSFT: Checked using the Get-AzVirtualNetworkUsageList PS cmdlet

Name : Subnet size and usage Id : /subscriptions/xyza/resourceGroups/aag-rg/providers/Microsoft.Network/virtualNetworks/aagVnet/subnets/AAG1Subnet CurrentValue : -1 Limit : -1 Unit : Count

Why it displays the negative I do not know.

However, the other subnets within the Vnet show their counts in numbers

[navba-MSFT]

@subhasishdc Thanks for getting back. I am glad that the PS cmdlet is showing the right count in numbers.
If your AAG1Subnet subnet has availability dependent on dynamic use then you will get the CurrentValue : -1 and Limit : -1. This is expected because as I mentioned earlier, if this is subnet is used by a resource where the number of instances deployed are not fixed, then the Available IP address cannot be determine. Hope this answers.

[subhasishdc]

@navba-MSFT : That makes sense, thank you. Also, did you get a chance to figure out what's wrong with the list-available-ips output.

[navba-MSFT]

@subhasishdc Thanks for your reply. Regarding the list-available-ips output, it is mentioned in the documentation itself that it
List some available ips in the subnet. see here.

Similar question was raised here, here and here. The available workaround is to use the Get-AzVirtualNetworkUsageList PS cmdlet or use the below AzCLI command. This will show the limit ( available IP in the response for each subnet).

az rest --url "https://management.azure.com/subscriptions/XXXXXXXXXX/resourceGroups/MYRGName/providers/Microsoft.Network/virtualNetworks/MYVNETName/usages?api-version=2022-01-01"

Hope this helps.

[subhasishdc]

@navba-MSFT : Thank you for the explanation. I get that part. However, the initial query remains the same.

Does list-available-ips shows some of the free and available ip addresses in the subnet. If yes, then the output is incorrect because it shows 5 ip addresses whereas only 3 should be free and available. On the other hand, if it displays some of the ip addresses in the subnet regardless of whether they are used or free, then the name is confusing and adding some explanation in the documentation would help.

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @aznetsuppgithub.

Issue Details

---

I was checking the number of available ip addresses in the subnet where an application gateway is deployed using this:

schowdhu@schowdhu-mn1 ~ % az network vnet subnet list-available-ips --resource-group aag-rg --vnet-name aagVnet -n AAG1Subnet This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus [ "10.0.36.244", "10.0.36.245", "10.0.36.246", "10.0.36.247", "10.0.36.248" ]

Following shows my subnet size and the app gw instances deployed.

[schowdhu@schowdhu-ld3 ~]$ az network vnet subnet show -g aag-rg -n AAG1Subnet --vnet-name aagVnet | jq '.addressPrefix' "10.0.36.240/28"

[schowdhu@schowdhu-ld3 ~]$ az network application-gateway list -g aag-rg | jq '.[] | .name,.sku,.operationalState' "aag01" { "capacity": 8, "name": "Standard_Medium", "tier": "Standard" } "Running"

Based on the app gw documentation (https://docs.microsoft.com/en-us/azure/application-gateway/configuration-infrastructure#size-of-the-subnet), app gw uses one private IP address per instance, plus another private IP address if a private front-end IP is configured. Azure also reserves five IP addresses in each subnet for internal use: the first four and the last IP addresses. Taking the above example for reference, total available ip addresses in the subnet 10.0.36.240/28 is 16. Azure uses 5 ip addresses and app gw has 8 instances. There should be only 3 ip addresses remaining. However, the az network vnet subnet list-available-ips command shown above shows 5 ip addresses available.

https://learn.microsoft.com/en-us/cli/azure/network/vnet/subnet?view=azure-cli-latest#az-network-vnet-subnet-list-available-ips -- This is the example in the documentation. It mentions "List some available ips in the subnet.". Does it mean it shows free ip addresses which are not used yet in the subnet or just it just display a few ip addresses belonging to the subnet irrespective of weather they are being used or not.

Am I missing something here or is there an issue with what the command is showing.

Author:	subhasishdc
Assignees:	navba-MSFT
Labels:	Network, Service Attention, question, customer-reported, Mgmt, needs-team-attention, Auto-Assign, Azure CLI Team
Milestone:	Backlog

[navba-MSFT]

@subhasishdc Thanks for your reply. I am adding the Service team to look into this ask and assist you further.

@aznetsuppgithub Could you please look into this ask and provide an update ? Thanks in advance.

[maciejduk]

I am not sure if this is still an ongoing issue, but since it's open and my results are similar to the OP, I will post here:

I have an empty /24 subnet. The list-available-ips command shows only 5 addresses available:

Portal shows below:

Am I understanding the purpose of this command wrong? I would've imagined list of free IPs would be 251 addresses long (as per portal, and the fact it's /24, minus 5 addresses for Azure services = 251).

[atharva-sarage]

I looked into the code as of 7/27/2023 the value 5 is hardcoded and it doesn't fetch the entire list.
Also this API is intelligent where it checks if the requested IP belongs to any subnet and throws this error if not.

{
  "error": {
    "code": "PrivateIPAddressNotInAnySubnet",
    "message": "Private static IP address 0.0.0.0 does not belong to the range of any subnet in the virtual network /subscriptions/<subId>/resourceGroups/<rg>/providers/Microsoft.Network/virtualNetworks/<vnetName>.",
    "details": []
  }
}