az vnet peering create no longer works cross-tenant

[mvzink]

Describe the bug

Command Name
az network vnet peering create

Errors:

ForbiddenError: (LinkedAuthorizationFailed) The client has permission to perform action 'Microsoft.Network/virtualNetworks/peer/action' on scope '/subscriptions/$SECONDARY_SUBSCRIPTION/resourceGroups/mzink-test/providers/Microsoft.Network/virtualNetworks/mzink-test-vnet/virtualNetworkPeerings/mzink-test-vnet-to-mzink-vnet', however the current tenant '$SECONDARY_TENANT' is not authorized to access linked subscription '$PRIMARY_SUBSCRIPTION'.

To Reproduce:

• Give a user in $SECONDARY_TENANT the Network Contributor role on a Virtual Network ($PRIMARY_VNET_ID) that lives in $PRIMARY_SUBSCRIPTION within $PRIMARY_TENANT.
• Log in to both tenants as the user from the secondary tenant:
  • az login -t $PRIMARY_TENANT
  • az login -t $SECONDARY_TENANT
• az network vnet peering create --allow-vnet-access --allow-forwarded-traffic --allow-gateway-transit -n secondary-to-primary --remote-vnet $PRIMARY_VNET_ID -g $SECONDARY_VNET_RG --vnet-name $SECONDARY_VNET_NAME --subscription $SECONDARY_SUBSCRIPTION

Expected Behavior

The VNet peering should be created. This was the case when using v2.12 of the CLI.

Environment Summary

Linux-5.4.0-58-generic-x86_64-with-debian-buster-sid
Python 3.6.10
Installer: DEB

azure-cli 2.18.0

Additional Context

The same also happens when attempting the reverse direction (primary VNet -> secondary VNet peering), with the error output showing the inverse IDs.

We have also developed on v2.17 of the CLI, which produces this same error, and v2.16, which instead produced a BadRequestError. Between all four versions (including v2.12 which does not error and successfully creates the peering) the only differences I could see when running with --verbose --debug were:

1. the ordering of the JSON body fields
2. the API version (2020-06-01 for v2.12, then 2020-07-01 for v2.16+)

I tried testing all combinations (old API version & old ordering, new API version & old ordering, etc.) on both v2.12 and v2.17 using the az rest command, and got the new LinkedAuthorizationFailed error in all cases.

[yonzhan]

network

[msyyc]

I’m trying to repro the problem. If any useful message, we will let you know.
FYI.

[msyyc]

Please give more detailed steps to reproduce the problem like:
(1)az ...
(2)az ...
(3)az ...
...
(N) az ...

[jsntcy]

@jiasli, any insights on this issue?

[jiasli]

Followed the tutorial Create peering - Azure CLI and I am able to reproduce:

az login

subA="414af076-009b-4282-9a0a-acf75bcb037e"
subB="0b1f6471-1bf0-4dda-aec3-cb9272f09590"

# Switch to subscription A
az account set -s $subA

# Create resource group A
az group create --name myResourceGroupA --location eastus

# Create virtual network A
az network vnet create --name myVnetA --resource-group myResourceGroupA --location eastus --address-prefix 10.0.0.0/16


# Switch to subscription B
az account set -s $subB

# Create resource group B
az group create --name myResourceGroupB --location eastus

# Create virtual network B
az network vnet create --name myVnetB --resource-group myResourceGroupB --location eastus --address-prefix 10.1.0.0/16


# Switch to subscription A
az account set -s $subA

# Peer myVNetA to myVNetB
az network vnet peering create --name myVnetAToMyVnetB --resource-group myResourceGroupA --vnet-name myVnetA --remote-vnet /subscriptions/$subB/resourceGroups/myResourceGroupB/providers/Microsoft.Network/VirtualNetworks/myVnetB --allow-vnet-access --debug

azure.core.pipeline.policies._universal: Request URL: 'https://management.azure.com/subscriptions/414af076-009b-4282-9a0a-acf75bcb037e/resourceGroups/myResourceGroupA/providers/Microsoft.Network/virtualNetworks/myVnetA/virtualNetworkPeerings/myVnetAToMyVnetB?api-version=2020-07-01'
azure.core.pipeline.policies._universal: Request method: 'PUT'
azure.core.pipeline.policies._universal: Request headers:
azure.core.pipeline.policies._universal:     'Content-Type': 'application/json'
azure.core.pipeline.policies._universal:     'Accept': 'application/json'
azure.core.pipeline.policies._universal:     'Content-Length': '485'
azure.core.pipeline.policies._universal:     'x-ms-client-request-id': '8901a934-6471-11eb-9a33-84a93e63aa78'
azure.core.pipeline.policies._universal:     'CommandName': 'network vnet peering create'
azure.core.pipeline.policies._universal:     'ParameterSetName': '--name --resource-group --vnet-name --remote-vnet --allow-vnet-access --debug'
azure.core.pipeline.policies._universal:     'User-Agent': 'AZURECLI/2.18.0 (MSI) azsdk-python-azure-mgmt-network/17.0.0 Python/3.6.8 (Windows-10-10.0.19041-SP0)'
azure.core.pipeline.policies._universal:     'Authorization': '*****'
azure.core.pipeline.policies._universal: Request body:
azure.core.pipeline.policies._universal: {"id": "/subscriptions/414af076-009b-4282-9a0a-acf75bcb037e/resourceGroups/myResourceGroupA/providers/Microsoft.Network/virtualNetworks/myVnetA", "name": "myVnetAToMyVnetB", "properties": {"allowVirtualNetworkAccess": true, "allowForwardedTraffic": false, "allowGatewayTransit": false, "useRemoteGateways": false, "remoteVirtualNetwork": {"id": "/subscriptions/0b1f6471-1bf0-4dda-aec3-cb9272f09590/resourceGroups/myResourceGroupB/providers/Microsoft.Network/VirtualNetworks/myVnetB"}}}

The request header x-ms-authorization-auxiliary is missing due to a limitation of Track 2 SDK (Azure/azure-sdk-for-python#8313), #15750 adopted a workaround to manually insert x-ms-authorization-auxiliary.

    return client.gallery_image_versions.begin_create_or_update(
        ...
        headers={'x-ms-authorization-auxiliary': external_bearer_token}
    )

We need to synchronize with SDK owner to support external_credential.

[jiasli]

Workaround with az rest

Use az rest to directly invoke Virtual Network Peerings - Create Or Update REST API.

x-ms-authorization-auxiliary: Bearer <external_tenant_access_token> header is manually added.

# Switch to subscription A
az account set -s $subA

# Get access token of subscription B
tokenB=$(az account get-access-token --subscription $subB --query accessToken --output tsv)

# Create peering from myVnetA to myVnetB
az rest --method PUT --url "https://management.azure.com/subscriptions/$subA/resourceGroups/myResourceGroupA/providers/Microsoft.Network/virtualNetworks/myVnetA/virtualNetworkPeerings/myVnetAToMyVnetB?api-version=2020-07-01" --headers "x-ms-authorization-auxiliary=Bearer $tokenB" --body "{\"id\": \"/subscriptions/$subA/resourceGroups/myResourceGroupA/providers/Microsoft.Network/virtualNetworks/myVnetA\", \"name\": \"myVnetAToMyVnetB\", \"properties\": {\"allowVirtualNetworkAccess\": true, \"allowForwardedTraffic\": false, \"allowGatewayTransit\": false, \"useRemoteGateways\": false, \"remoteVirtualNetwork\": {\"id\": \"/subscriptions/$subB/resourceGroups/myResourceGroupB/providers/Microsoft.Network/VirtualNetworks/myVnetB\"}}}"

# View the peering state of myVnetA
az network vnet peering list --resource-group myResourceGroupA --vnet-name myVnetA


# Switch to subscription B
az account set -s $subB

# Get access token of subscription A
tokenA=$(az account get-access-token --subscription $subA --query accessToken --output tsv)

# Create peering from myVnetB to myVnetA
az rest --method PUT --url "https://management.azure.com/subscriptions/$subB/resourceGroups/myResourceGroupB/providers/Microsoft.Network/virtualNetworks/myVnetB/virtualNetworkPeerings/myVnetBToMyVnetA?api-version=2020-07-01" --headers "x-ms-authorization-auxiliary=Bearer $tokenA" --body "{\"id\": \"/subscriptions/$subB/resourceGroups/myResourceGroupB/providers/Microsoft.Network/virtualNetworks/myVnetB\", \"name\": \"myVnetBToMyVnetA\", \"properties\": {\"allowVirtualNetworkAccess\": true, \"allowForwardedTraffic\": false, \"allowGatewayTransit\": false, \"useRemoteGateways\": false, \"remoteVirtualNetwork\": {\"id\": \"/subscriptions/$subA/resourceGroups/myResourceGroupA/providers/Microsoft.Network/VirtualNetworks/myVnetA\"}}}"

# View the peering state of myVnetB
az network vnet peering list --resource-group myResourceGroupB --vnet-name myVnetB

[jiasli]

Azure CLI core has supported cross-tenant authentication with Track 2 SDK in #16797.