azurerm_logic_app_standard throws error when working on private network

[benjamin-goldman]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request

Terraform (and AzureRM Provider) Version

• Terraform Core version: v1.0.7
• AzureRM Provider version: v2.81.0

Terraform Configuration Files

terraform {
  required_version = ">= 1.0.7"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">=2.79.0"
    }
  }
}
provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "lafuntest" {
  name     = "lafuntest"
  location = "centralus"
}

resource "azurerm_storage_account" "st0127wbu" {
  name                     = "st0127wbu"
  resource_group_name      = azurerm_resource_group.lafuntest.name
  location                 = azurerm_resource_group.lafuntest.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  network_rules {
    default_action = "Deny"
  }
}

resource "azurerm_app_service_plan" "lafunplan" {
  name                = "lafunplan"
  resource_group_name = azurerm_resource_group.lafuntest.name
  location            = azurerm_resource_group.lafuntest.location
  kind                = "elastic"
  reserved            = true

  sku {
    tier = "Standard"
    size = "WS1"
  }
}

resource "azurerm_logic_app_standard" "fun-logic-app" {
  name                       = "fun-logic-app"
  resource_group_name        = azurerm_resource_group.lafuntest.name
  location                   = azurerm_resource_group.lafuntest.location
  app_service_plan_id        = azurerm_app_service_plan.lafunplan.id
  use_extension_bundle       = true
  storage_account_name       = azurerm_storage_account.st0127wbu.name
  storage_account_access_key = azurerm_storage_account.st0127wbu.primary_access_key
}

Description / Feedback

i would expect that the service would create without this error

Expected Behavior

terraform will complete as expected!

Actual Behavior

azurerm_logic_app_standard.fun-logic-app: Creating...

Error: web.AppsClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="BadRequest" Message="Creation of storage file share failed with: 'The remote server returned an error: (403) Forbidden.'. Please check if the storage account is accessible." Details=[{"Message":"Creation of storage file share failed with: 'The remote server returned an error: (403) Forbidden.'. Please check if the storage account is accessible."},{"Code":"BadRequest"},{"ErrorEntity":{"Code":"BadRequest","ExtendedCode":"99022","Message":"Creation of storage file share failed with: 'The remote server returned an error: (403) Forbidden.'. Please check if the storage account is accessible.","MessageTemplate":"Creation of storage file share failed with: '{0}'. Please check if the storage account is accessible.","Parameters":["The remote server returned an error: (403) Forbidden."]}}]

on test.tf line 44, in resource "azurerm_logic_app_standard" "fun-logic-app":
44: resource "azurerm_logic_app_standard" "fun-logic-app" {

References

this is identical to this issue
#13566

[mleziva]

Seeing the same issue on firewall protected storage accounts when provisioning the file share outside of the logic app resource and setting it using storage_account_share_name = azurerm_storage_share.logic_app.name

However, I am able to use the same fileshare when deploying the logic app using ARM and it works correctly. So a work around is to use deploy the fileshare and then deploy the logic app using azurerm_resource_group_template_deployment

[jcanizalez]

hi @dylanmorley any idea about above error?

[StephanHCB]

Hi!

We're seeing the same error when attempting to deploy a logic_app_standard to an App Service Environment v3 with both the ASE v3 and the storage account in a private network and fully network isolated.

We create all four private endpoint types for the storage account as instructed in the microsoft docs. Note especially the purple box with the link to an ARM template that successfully deploys a logic app in this case.

Unfortunately, doing the same thing with the terraform provider fails at the logic app creation step due to inability to access the storage account:

(the example takes 3+ hours to deploy because of the long creation time of ASE v3 and ASP)

[...]
azurerm_app_service_environment_v3.ase: Still creating... [2h36m11s elapsed]
azurerm_app_service_environment_v3.ase: Still creating... [2h36m21s elapsed]
azurerm_app_service_environment_v3.ase: Creation complete after 2h36m28s [id=/subscriptions/<redacted>/resourceGroups/logicapp-testcase-rg/providers/Microsoft.Web/hostingEnvironments/logicapp-testcase-ase]
azurerm_app_service_plan.asp: Creating...
azurerm_app_service_plan.asp: Still creating... [10s elapsed]
azurerm_app_service_plan.asp: Still creating... [20s elapsed]
azurerm_app_service_plan.asp: Still creating... [30s elapsed]
[...]
azurerm_app_service_plan.asp: Still creating... [25m10s elapsed]
azurerm_app_service_plan.asp: Still creating... [25m20s elapsed]
azurerm_app_service_plan.asp: Still creating... [25m30s elapsed]
azurerm_app_service_plan.asp: Still creating... [25m40s elapsed]
azurerm_app_service_plan.asp: Still creating... [25m50s elapsed]
azurerm_app_service_plan.asp: Still creating... [26m0s elapsed]
azurerm_app_service_plan.asp: Creation complete after 26m4s [id=/subscriptions/<redacted>/resourceGroups/logicapp-testcase-rg/providers/Microsoft.Web/serverfarms/logicapp-testcase-asp]
azurerm_logic_app_standard.logicapp: Creating...
╷
│ Error: web.AppsClient#CreateOrUpdate: 
    Failure sending request: StatusCode=400 -- 
	Original Error: Code="BadRequest" Message="Creation of storage file share failed with: 'The remote server returned an error: (403) Forbidden.'. Please check if the storage account is accessible." 
	  Details=[{"Message":"Creation of storage file share failed with: 'The remote server returned an error: (403) Forbidden.'. Please check if the storage account is accessible."},{"Code":"BadRequest"},
	           {"ErrorEntity":{"Code":"BadRequest","ExtendedCode":"99022","Message":"Creation of storage file share failed with: 'The remote server returned an error: (403) Forbidden.'. Please check if the storage account is accessible.",
			    "MessageTemplate":"Creation of storage file share failed with: '{0}'. Please check if the storage account is accessible.","Parameters":["The remote server returned an error: (403) Forbidden."]}}]
│ 
│   with azurerm_logic_app_standard.logicapp,
│   on main.tf line 147, in resource "azurerm_logic_app_standard" "logicapp":
│  147: resource "azurerm_logic_app_standard" "logicapp" {
│ 
╵

We have also set the logic app parameters like the ARM template does, as described in this tech community post, that is:

"WEBSITE_VNET_ROUTE_ALL"  = "1"
"WEBSITE_CONTENTOVERVNET" = "1"

Side Note: according to Microsoft's response to a ticket, it is not possible to share an ASE v3 between function apps and logic apps, you need to deploy a separate ASE v3 for logic apps to a separate subnet. I am not aware this is mentioned anywhere in the docs.

Here is the complete code used to deploy the example, with some state storage and provider settings left out for privacy reasons, but it should still work as-is:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "2.82.0"
    }
  }
}

provider "azurerm" {
  features {}
}

resource "azurerm_resource_group" "rg" {
  name     = "logicapp-testcase-rg"
  location = "West Europe"
}

# private vnet with subnets

resource "azurerm_virtual_network" "vnet" {
  name                = "logicapp-testcase-vnet"
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  address_space       = [ "10.180.0.0/16" ]
}

resource "azurerm_subnet" "subnet_storage" {
  name                                           = "logicapp-testcase-subnet-storage"
  virtual_network_name                           = azurerm_virtual_network.vnet.name
  resource_group_name                            = azurerm_resource_group.rg.name
  address_prefixes                               = [ "10.180.2.0/24" ]
  enforce_private_link_endpoint_network_policies = true
}

resource "azurerm_subnet" "subnet_ase" {
  name                                           = "logicapp-testcase-subnet-ase"
  virtual_network_name                           = azurerm_virtual_network.vnet.name
  resource_group_name                            = azurerm_resource_group.rg.name
  address_prefixes                               = [ "10.180.3.0/24" ]
  enforce_private_link_endpoint_network_policies = true

  delegation {
    name = "delegation"

    service_delegation {
      name    = "Microsoft.Web/hostingEnvironments"
      actions = ["Microsoft.Network/virtualNetworks/subnets/action"]
    }
  }
}

# checkpoint storage w/ network isolation and private endpoints for all four storage types

resource "azurerm_storage_account" "storage" {
  name                      = "logicapptcs"
  resource_group_name       = azurerm_resource_group.rg.name
  location                  = azurerm_resource_group.rg.location
  account_kind              = "StorageV2"
  account_tier              = "Standard"
  account_replication_type  = "LRS"
  allow_blob_public_access  = false
  enable_https_traffic_only = true
  min_tls_version           = "TLS1_2"

  network_rules {
    default_action             = "Deny"
	# in reality, I have added our proxy IPs to this array so I can look at the storage account in the browser
    ip_rules                   = [ "127.0.0.1" ]
    bypass                     = [ "Logging", "Metrics", "AzureServices" ]
  }  
}

resource "azurerm_private_endpoint" "storage_private_endpoint" {
  for_each            = toset(["blob", "queue", "file", "table"])
  name                = "logicapp-testcase-st-pe-${each.key}"
  location            = azurerm_storage_account.storage.location
  resource_group_name = azurerm_storage_account.storage.resource_group_name
  subnet_id           = azurerm_subnet.subnet_storage.id

  private_service_connection {
    name                           = "logicapp-testcase-st-psc-${each.key}"
    is_manual_connection           = false
    private_connection_resource_id = azurerm_storage_account.storage.id
    subresource_names              = [each.key]
  }
}

# ASE v3 and plan

resource "azurerm_app_service_environment_v3" "ase" {
  name                = "logicapp-testcase-ase"
  resource_group_name = azurerm_resource_group.rg.name
  subnet_id           = azurerm_subnet.subnet_ase.id

  internal_load_balancing_mode = "Web, Publishing"

  cluster_setting {
    name  = "DisableTls1.0"
    value = "1"
  }

  cluster_setting {
    name  = "InternalEncryption"
    value = "true"
  }

  cluster_setting {
    name  = "FrontEndSSLCipherSuiteOrder"
    value = "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"
  }
}

resource "azurerm_app_service_plan" "asp" {
  name                       = "logicapp-testcase-asp"
  resource_group_name        = azurerm_resource_group.rg.name
  location                   = azurerm_resource_group.rg.location
  app_service_environment_id = azurerm_app_service_environment_v3.ase.id
  kind                       = "Windows"
  reserved                   = false

  sku {
    tier     = "IsolatedV2"
    size     = "I1v2"
    capacity = 1
  }
}

# logic app

resource "azurerm_logic_app_standard" "logicapp" {
  name                       = "logicapp-testcase-app"
  location                   = azurerm_resource_group.rg.location
  resource_group_name        = azurerm_resource_group.rg.name
  app_service_plan_id        = azurerm_app_service_plan.asp.id
  storage_account_name       = azurerm_storage_account.storage.name
  storage_account_access_key = azurerm_storage_account.storage.primary_access_key

  site_config {
    vnet_route_all_enabled = "1"
  }

  app_settings = {
    "WEBSITE_VNET_ROUTE_ALL"  = "1"
    "WEBSITE_CONTENTOVERVNET" = "1"
  }
}

Just tried with v2.83.0 this morning, and the error still happens.

[dylanmorley]

I went through similar issues when I was first working on this, there's a comment from me here Azure/Azure-Functions#1361 (comment) and a nice explanation of what's going on under the hood from Jeff Hollan here

How I'm currently doing this - I explicitly create the file share myself, and give that to the application settings. rather than relying on the underlying function app/app service infra to create the file share for me, e.g.

1 - Create storage account, no network restrictions
2 - Generate random share name and create share
3 - Create logic application, pass share name in parameter storage_account_share_name
3a - Settings for vnet_route_all_enabled in site config, and WEBSITE_CONTENTOVERVNET on in app settings
4 - Create swift connection, associate logic app id and subnet
5 - Apply network restrictions

I find with this order, I get a consistent and repeatable create and destroy process

resource "azurerm_storage_account" "storage" {
  name                     = var.storage_account_name
  location                 = "northeurope"
  resource_group_name      = var.resource_group_name
  account_tier             = "Standard"
  account_replication_type = "ZRS"
  tags                     = var.tags
}

resource "random_string" "random_share_name" {
  length           = 15
  special          = false
  lower            = true
  upper            = false
}

resource "azurerm_storage_share" "storage" {
  name                 = random_string.random_share_name.result
  storage_account_name = azurerm_storage_account.storage.name
}

resource "azurerm_logic_app_standard" "logic_application" {
  name                       = var.workflow_name
  location                   = var.azure_location
  resource_group_name        = local.resource_group_name
  app_service_plan_id        = azurerm_app_service_plan.plan.id
  storage_account_name       = azurerm_storage_account.storage.name
  storage_account_access_key = azurerm_storage_account.storage.primary_access_key
  storage_account_share_name = azurerm_storage_share.storage.name
  https_only                 = true
  use_extension_bundle       = false

  app_settings = {
    "FUNCTIONS_V2_COMPATIBILITY_MODE" = "true"
    "APPLICATIONINSIGHTS_CONNECTION_STRING" = local.insights_connection_string
    "APPINSIGHTS_INSTRUMENTATIONKEY" = var.instrumentation_key
    "FUNCTIONS_WORKER_RUNTIME" = "node"
    "WEBSITE_NODE_DEFAULT_VERSION" = "~12"
    "WEBSITE_CONTENTOVERVNET" = var.use_vnet
  }

  site_config {
    vnet_route_all_enabled = true 
  }

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_app_service_virtual_network_swift_connection" "logic_vnet" {
  app_service_id = azurerm_logic_app_standard.logic_application.id
  subnet_id      = var.logic_app_subnet_id
}

resource "azurerm_storage_account_network_rules" "storage_rules" {
  storage_account_id          = azurerm_storage_account.storage.id

  depends_on                  = [
    azurerm_app_service_virtual_network_swift_connection.logic_vnet
  ]

  default_action             = "Deny"
  virtual_network_subnet_ids = var.storage_account_subnet_ids
}

I'll take a look at the linked issue/fix to 13566 and see if that's something we can bring over to this resource

[StephanHCB]

How I'm currently doing this - I explicitly create the file share myself, and give that to the application settings. rather than relying on the underlying function app/app service infra to create the file share for me, e.g.

1 - Create storage account, no network restrictions 2 - Generate random share name and create share 3 - Create logic application, pass share name in parameter storage_account_share_name 3a - Settings for vnet_route_all_enabled in site config, and WEBSITE_CONTENTOVERVNET on in app settings 4 - Create swift connection, associate logic app id and subnet 5 - Apply network restrictions

I find with this order, I get a consistent and repeatable create and destroy process

While this works on a technology level, there's a compliance & security issue. Specifically in our case, that means we would have to use a separate new storage account for every new Logic App - as the old one may contain customer/sensitive data in its workflow checkpoints, and it would thus not be acceptable to lift its network restrictions (even temporarily).

I'll take a look at the linked issue/fix to 13566 and see if that's something we can bring over to this resource

Thank you, very much appreciated.

[jcanizalez]

hey guys I created a PR #13964 to fix this issue

Baically the problem is the app_settings creation order

Currently the app settings defined in app_settings property are updated after the resource is created, so basically the resource is performing the following steps:

1. Create Logic app wiht basic settings (resource defined settings)
2. Calls update resource to update app_settings with user defined settings.

The problem is there are some app_settings must be defined at creation time for example in order to create a logic app using storage account with private endpoints WEBSITE_CONTENTOVERVNET is required at creation time. See microsoft docs for more details.

[github-actions]

This functionality has been released in v2.84.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.