Azurerm_data_factory_linked_service_azure_blob_storage account name stored asterisks

[levgoronshteyn]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

1.0

AzureRM Provider Version

3.0

Affected Resource(s)/Data Source(s)

azurerm_data_factory_linked_service_azure_blob_storage

Terraform Configuration Files

provider "azurerm" {
  # Configuration options
  features {}
}

data "azurerm_resource_group" "test" {
  name     = "test-lg"
 # location = "East US"
}

data "azurerm_client_config" "current" {
}

data "azurerm_storage_account" "example" {
  name                = "levgoronshteynstorage"
  resource_group_name = data.azurerm_resource_group.test.name
}

resource "azurerm_data_factory" "test" {
  name                = "example-lev"
  location            = data.azurerm_resource_group.test.location
  resource_group_name = data.azurerm_resource_group.test.name
}

resource "azurerm_key_vault" "test" {
  name                = "example-lev"
  location            = data.azurerm_resource_group.test.location
  resource_group_name = data.azurerm_resource_group.test.name
  tenant_id           = data.azurerm_client_config.current.tenant_id
  sku_name            = "standard"
}

resource "azurerm_data_factory_linked_service_key_vault" "test" {
  name            = "linkkv-lev"
  data_factory_id = azurerm_data_factory.test.id
  key_vault_id    = azurerm_key_vault.test.id
}

resource "azurerm_data_factory_linked_service_azure_blob_storage" "test" {
  name            = "example-lev"
  data_factory_id = azurerm_data_factory.test.id
 service_principal_id = "f38c0ec8-c8d0-4df8-b29b-69057f6236984239"
 service_principal_key = "6zc5e2d7-5f12-5669-9ea5-8108093a60ce"
 tenant_id = "0d4q3e88-8caf-41ca-b4da-e3b33b6c52ec"
 connection_string = "DefaultEndpointsProtocol=https;AccountName=levgoronshteynstorage;AccountKey=NJaByjNTH3HZcRyROBeDFPjKoPIUMViI0p7r+79tRLBVSQNmmspGBdwI4ns2qGyp1HtvrZC+6hYf+AStVaZ8eg==;EndpointSuffix=core.windows.net"

ALL sensitive information has been changed!

Debug Output/Panic Output

No panic output but what occurs is when you login to Azure portal and navigate to Azure Data Factory to see what the blob storage account name is, you are not able to see anything but asterisks(********). If you perform the same steps manually via Azure Portal, you can see the blob storage account name.

Same issue accounts even if you use AZ CLI

Expected Behaviour

When a user uses Azure Portal to add a linked service for blob storage, the portal breaks up the connection string into two properties below:

{
  "connectionString": "<value>",
  "encryptedCredential": "<value>"
}

Actual Behaviour

When using AzureRM provider, the connection string is stored as secure string and the storage account name is hidden with asterisks.

"connectionString": {
      "type": "SecureString",
      "value": "**********"
    },
    "credential": null,
    "description": null,
    "encryptedCredential": "ew0KICAiVmVyc2lvbiI6ICIyMDE3LTExLTMwIiwNCiAgIlByb3RlY3Rpb25Nb2RlIjogIktleSIsDQogICJTZWNyZXRDb250ZW50VHlwZSI6ICJQbGFpbnRleHQiLA0KICAiQ3JlZGVudGlhbElkIjogIkRBVEFGQUNUT1JZQEREMjREMjZGLTFEQkYtNDVERi04NEM1LTIzRUI4MDIzMUU1QV9hZGU1YTkzYS00MWZlLTRjZGYtOTA1NS1lNzg5YzcwMzM2NDciDQp9",

Steps to Reproduce

Use the Terraform Configuration File provided

change the resource group and storage account

apply the terraform file.

Important Factoids

No response

References

The AZ CLI tool has the same behaviour as terraform:

az datafactory linked-service create --factory-name "example-lev" --properties "{\"type\":\"AzureBlobStorage\",\"typeProperties\":{\"connectionString\":

{\"type\":\"SecureString\",\"value\":\"DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx+xxx+6hYf+AStVaZ8eg==;EndpointSuffix=core.windows.net\"}}}" --name "exampleLinkedService" --resource-group "test-lg"
{
  "etag": "4f008de4-0000-0100-0000-63b477df0000",
  "id": "/subscriptions/xxx/resourceGroups/test-lg/providers/Microsoft.DataFactory/factories/example-lev/linkedservices/exampleLinkedService",
  "name": "exampleLinkedService",
  "properties": {
    "accountKey": null,
    "accountKind": null,
    "additionalProperties": null,
    "annotations": null,
    "azureCloudType": null,
    "connectVia": null,
    "connectionString": {
      "type": "SecureString",
      "value": "**********"
    },
    "credential": null,
    "description": null,
    "encryptedCredential": "xxxxxxxxxxxxxxxxxxxxxxxx",
    "parameters": null,
    "sasToken": null,
    "sasUri": null,
    "serviceEndpoint": null,
    "servicePrincipalId": null,
    "servicePrincipalKey": null,
    "tenant": null,
    "type": "AzureBlobStorage"
  },
  "resourceGroup": "test-lg",
  "type": "Microsoft.DataFactory/factories/linkedservices"

[ms-henglu]

Hi @levgoronshteyn ,

Thank you for taking time to open this issue.

Is it duplicated with #14358?

[levgoronshteyn]

Hi @ms-henglu,

It is the same issue as #14358 but since 14358 was never solved i have opened a new issue.

If using Terraform or even AZ CLI you cannot see the storage account name.

[ms-henglu]

Hi @levgoronshteyn ,

Thanks for the clarification. I have a workaround for this, https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/data_factory_linked_custom_service is a generic data factory linked service resource, you can use it to pass any payload, I think it might solve the problem.

[levgoronshteyn]

thank you for getting back to me @ms-henglu, i will test the workaround but at the same time please check the code and see if it can be fixed without using this workaround? As stated also that this can be reproduced via AZ CLI and not just on Terraform.

Even if we use AZ API Provider it is asking to send the connection string as SecureString:

https://learn.microsoft.com/en-us/azure/templates/microsoft.datafactory/factories/linkedservices?pivots=deployment-language-terraform:

ype = "AzureBlobStorage" typeProperties = { accountKey = { store = { parameters = {} referenceName = "string" type = "LinkedServiceReference" } type = "string" } accountKind = "string" credential = { referenceName = "string" type = "CredentialReference" } encryptedCredential = "string" sasToken = { store = { parameters = {} referenceName = "string" type = "LinkedServiceReference" } type = "string" } serviceEndpoint = "string" servicePrincipalKey = { type = "string" // For remaining properties, see SecretBase objects } }

If you then see the SecretBase object, it will refer you to the SecureString section.

also for clarity, as an example, the connection string is below:
DefaultEndpointsProtocol=https;AccountName=StorageAccount;AccountKey=FLoCyjNTG3XXXrYR0beDFPjK0PIUMViIop7r+99tRLBVSQNmmspGBdwI4ns2qGyp1HtvrZC+6hYf+AStVaZ8eg==;EndpointSuffix=core.windows.net

If you do this from Azure Portal UI, Azure will break up the string into two properties:

1. Will be the connectionstring = DefaultEndpointsProtocol=https;AccountName=StorageAccount;EndpointSuffix=core.windows.net and the
2. Will be the encryptedCredential=AccountKey=FLoCyjNTG3XXXrYR0beDFPjK0PIUMViIop7r+99tRLBVSQNmmspGBdwI4ns2qGyp1HtvrZC+6hYf+AStVaZ8eg== (but this get encrypted)

If you do the same via Terraform or AZ CLI, the connection string is not broken out after it is received by the portal.

[levgoronshteyn]

@ms-henglu I have tested the work around provided and it worked as well. I sent the outcome and details to the client until the reported issue is resolved.

I also see that there has bee a PR created and tested to verify that the resource can be used as designed without encrypting non-sensitive data. Do you know when the fix will be in and i can let the client know that they can test?

[tombuildsstuff]

@ms-henglu this is an Azure Portal issue which needs to be fixed there, since Terraform is matching the behaviour of the Azure CLI etc?

[katbyte]

I opened an issue on the rest API specs as this appears to be something that needs to be corrected in the portal as both providers + the CLI are sending the connection string in the same way.

[levgoronshteyn]

@ms-henglu any update on this?

[levgoronshteyn]

@ms-henglu the client also tested your recommended workaround but they cannot use it as the resource is not an approved one to be used in their org.

[github-actions]

This functionality has been released in v3.44.0 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.