Skip to content

Commit

Permalink
Version 14.4.0
Browse files Browse the repository at this point in the history 
 - *ADDED:* Added new `ManagedIdentity` authentication type to eliminate the need for a UserName and Password to authenticate to Azure SQL databases that have Azure AD authentication enabled and identity assigned
 - *ADDED:* Ability to use Managed Identity for Service Bus, Event Hub and Blob storage connections with most services (see [managed_identity.md](/docs/managed_identity.md) for details and limitations)
 - *ADDED:* New `--monitor` argument for `sbm batch run` to get running count of datbase activity  (commits, error, in queue)
 - *ADDED:* New `--stream` argument for `sbm batch run` (used in conjunction with `--monitor`) to also stream specific database completion messages as the occur
 - *UPDATED:* Monitoring of remaining queue messages only when Service Bus is used, but no Event Hub connection is provided
 - *UPDATED:* Reorganized Unit Test settings file creation scripts to group by execution compute type
  • Loading branch information
@mmckechney
mmckechney committed Jul 7, 2022
1 parent d778ad4 commit 752fea7
Show file tree
Hide file tree
Showing 37 changed files with 557 additions and 403 deletions.
5 changes: 3 additions & 2 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
# SQL Build Manager Change Log

### Version 14.4.0
- *ADDED:* Added new `ManagedIdentity` authentication type to eliminate the need for a UserName and Password to authenticate to Azure SQL databases that have Azure AD authenication enabled and identity assigned
- *ADDED:* Ability to use Managed Identity for Service Bus and EventHub connections (see [README.md](README.md) for details)
- *ADDED:* Added new `ManagedIdentity` authentication type to eliminate the need for a UserName and Password to authenticate to Azure SQL databases that have Azure AD authentication enabled and identity assigned
- *ADDED:* Ability to use Managed Identity for Service Bus, Event Hub and Blob storage connections with most services (see [managed_identity.md](/docs/managed_identity.md) for details and limitations)
- *ADDED:* New `--monitor` argument for `sbm batch run` to get running count of datbase activity (commits, error, in queue)
- *ADDED:* New `--stream` argument for `sbm batch run` (used in conjunction with `--monitor`) to also stream specific database completion messages as the occur
- *UPDATED:* Monitoring of remaining queue messages only when Service Bus is used, but no Event Hub connection is provided
- *UPDATED:* Reorganized Unit Test settings file creation scripts to group by execution compute type

### Version 14.3.0
- *ADDED:* Managed Identity and Key Vault support for Container Apps
Expand Down
13 changes: 3 additions & 10 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,11 @@ SQL Build Manager is a multi-faceted tool to allow you to manage the life-cycle

![.NET Core Build](https://github.com/mmckechney/SqlBuildManager/workflows/.NET%20Core%20Build/badge.svg)

### _Be sure to review the [change log](CHANGELOG.md) for the latest updates, enhancements and bug fixes_
#### _Be sure to review the [change log](CHANGELOG.md) for the latest updates, enhancements and bug fixes_

### **Key feature enhancement with Version 14.4+: Expanded use of Azure Managed Identity to connect to resources**
### **Key feature enhancement with Version 14.4+: Expanded use of Azure User Assigned Managed Identity**

With this update, it significantly reduces the the need to save and manage secrets and connection strings

- _Database Connections:_ [Kubernetes](docs/kubernetes.md), [Azure Container Instance](docs/aci.md) and [Batch](docs/massively_parallel.md) methods now allow for the use of Azure Managed Identity to authenticate to SQL Databases (with new `--authtype ManagedIdentity`).

- _Service Bus:_ For any deployment type other than [Azure Container Apps](docs/containerapp.md), use the name of the Service Bus namespace as the value for `--servicebustopicconnection` in place of the full connection string (see below for explanation)
- _Event Hub:_ For any deployment type, use the Event Hub Namespace and Event Hub name as values for `--eventhubconnection` in the pipe delimited format `<Event Hub Namespace>|<Event Hub Name>`

_NOTE:_ [Azure Container Apps](docs/containerapp.md) does not currently allow for Managed Identity authentication to Azure SQL database. Also, the Service Bus connection string is required for the KEDA scaler so you can not use just the namespace. You _can_ use the `<Event Hub Namespace>|<Event Hub Name>` convention for Event Hubs however!
With this update, it significantly reduces the the need to save and manage secrets and connection strings. For full details on leveraging Managed Identity to connect to the other Azure resources such as SQL Database, Blob storage, Service Bus, Event Hub, Key Vault and Azure Container registry, see the [Managed Identity documentation here](/docs/managed_identity.md).


### **Important changes in Version 14+:**
Expand Down
4 changes: 2 additions & 2 deletions devops/azure-docker-pipeline.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ steps:
command: 'buildAndPush'
Dockerfile: './src/Dockerfile'
tags: |
'latest-$(Build.SourceBranchName)'
'latest-vNext'
latest-$(Build.SourceBranchName)
latest-vNext
- task: Docker@2
displayName: Logout of DockerHub
Expand Down
89 changes: 89 additions & 0 deletions docs/managed_identity.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Using User Assigned Managed Identity

Each of the remote execution options allows for varying use of Azure Managed Identities for authentication to supporting resources. This document shows:
- Which [execution option is capable of leveraging the identity](#managed-identity-support-between-resources-and-execution-type) to connect to which resource
- [How the identity gets associated](#managed-identity-to-compute-assignment) with the execution compute type
- The [configuration and/or arguments needed](#runtime-configuration-to-leverage-managed-identity) to instruct the use of the identity
- The [Azure RBAC roles requred](#azure-rbac-role-assignment-requirements) for the identity


## Managed Identity support between resources and execution type

| | Azure Batch | Kubernetes (AKS) | Container Apps | Container Instance (ACI) |
| ------------------------ | :-----------: | :----------------------------------------------------------------------------------------------------------------------------:| :---------------: | :-----------------------: |
| Azure Key Vault | Yes | Yes, with [Key Vault Identity Provider](https://docs.microsoft.com/en-us/azure/aks/csi-secrets-store-driver) | Yes | Yes |
| SQL databases | Yes | Yes | No, see [note](#sql-database) | Yes |
| Blob Storage | No, see [note](#blob-storage)| Yes | Yes | Yes |
| Service Bus | Yes | Yes | No, see [note](#service-bus) | Yes |
| Event Hub | Yes | Yes | Yes | Yes |
| Azure Container Registry | N/A | Yes, with [`--attach-acr`](https://docs.microsoft.com/en-us/azure/aks/cluster-container-registry-integration?tabs=azure-cli)| No | No |
| | | | | |

## Managed Identity to Compute Assignment

Examples of each of these can be generated for you by running [create_azure_resources.ps1](../scripts/templates/create_azure_resources.ps1) which will create samples Azure resources (including a user assigned Managed Identity) as well as sample settings files with various options that are used in the test caes. You can also review the test methods in [SqlBuildManager.Console.ExternalTest](../src/SqlBuildManager.Console.ExternalTest/) to see working examples of various compute options and settings.

### Azure Batch

The identity is assigned at the creation of the Azure Batch account. For an example, see [azuredeploy_batch.bicep](../scripts/templates/Batch/azuredeploy_batch.bicep)

### Kubernetes

The identity first must be associated with the AKS VM Scale set. You can find an example of this assignment near the bottom of [this srcipt](../scripts/templates//kubernetes//create_aks_cluster.ps1)


The identity is assigned to the app at runtime in two steps. First `AzureIdentity` and `AzureIdentityBinding` resources are created in the cluster (see [podIdentityAndBinding_template.yaml](../scripts//templates/kubernetes/podIdentityAndBinding_template.yaml)). Then when the SQL Build Manger job is deployed, an `aadpodidbinding` label is added to the job spec (see [sample_job.yaml](../scripts/templates/kubernetes/sample_job.yaml)). This will tell Kubernetes to assign the identity to the job.

### Container Apps

The Container Apps workload that is deployed via `sbm containerapp deploy` leverages ARM templates to deploy the workload. This ARM template contains the Managed Identity assignment (see [containerapp_identity_arm_template.json](../src/SqlBuildManager.Console/ContainerApp/containerapp_identity_arm_template.json). The identity information needs to be passed via the Identity options in command line or be saved in to a settings file via `sbm containerapp savesettings`

### Container Instance (ACI)

ACI also uses an ARM template to deploy the workload when running `sbm aci deploy`. This ARM template contains the Managed Identity assignment(see [aci_arm_template.json](../src/SqlBuildManager.Console/Aci/aci_arm_template.json)). The identity information needs to be passed via the Identity options in command line or be saved in to a settings file via `sbm aci savesettings`

## Runtime Configuration to leverage Managed Identity

### Azure Key Vault

To instruct the app to pull secrets from Azure Key Vault, you need to provide the Key Vault name in the `--keyvault` argument of the appropriate `savesettings`, `prep` and/or `deploy`. Alternatively, you can simply save it to the settings file .json with the `savesetting` command and leverage the settings file for all subsequent commands

### SQL Database

By default, the app uses username/password database authentication. To enable Managed Identity authentication, you will first need to add the [Manged Identity as a user to the database](https://docs.microsoft.com/en-us/azure/azure-sql/database/authentication-azure-ad-user-assigned-managed-identity?view=azuresql#managing-a-managed-identity-for-a-server-or-instance). Once this has been done, you can direct the app to use the identity to authenticate with the `--authtype ManagedIdentity` flag. (As always, this can be saved in a settings file with `savesettings` for easier execution and reuse).

**NOTE:** Azre Container Apps does not currently allow for Managed Identity authentication to Azure SQL Databases


### Blob Storage

To use Managed Identity to access blob storage, simply don't provide a value for `--storageaccountkey`. If this is not provided, the app will default to connecting with the identity.

**NOTE:** Azure Batch requires the storage account key to manage storage and create SAS token URLs. The key must be provided as a settings file value, command line argument or be saved in Azure Key Vault.

### Service Bus

To use Managed Identity to connect to Azure Service Bus, use the Service Bus namespace as the value (\<name>.servicebus.windows.net) for `--servicebustopicconnection`.

**NOTE:** Azure Container Apps uses a [KEDA](https://keda.sh/) [service bus scaler](https://keda.sh/docs/2.7/scalers/azure-service-bus/) to manage scaling. This does not currently allow for Managed Identity authentication, so a full Service Bus connection string is still needed.

### Event Hub

To use Managed Identity to connect to Azure Event Hub, use the Event Hub namespace and Event Hub name as a pipe delimited value (<name>.servicebus.windows.net) for `--eventhubconnection`. For example "\<ehnamespace>.servicebus.windows.net|\<eh name>"

### Azure Container Registry

Only Kubernetes is able to natively connect to the container registry without an admin username and password. This is assigned at creation or update of the cluster using the [`--assign-acr`](https://docs.microsoft.com/en-us/azure/aks/cluster-container-registry-integration?tabs=azure-cli) flag

## Azure RBAC Role Assignment Requirements


The Managed Identity assigned to the runtime compute will need the following Azure RBAC roles assigned to the resource group or the specific services. _NOTE:_ You can also use an identity to authenticate for the `prep` and `enqueue` steps. The identity of that user or machine will just also need these roles.
- `Storage Blob Data Contributor` - to read the build package and save log files
- `Key Vault Secrets User` - to pull secrets from the Key Vault
- `Azure Service Bus Data Owner` - to read messages and delete compelted Service Bus Topic Subscription
- `Azure Event Hubs Data Receiver` - to read events from Event Hub
- `Azure Event Hubs Data Sender` - to send events to Event Hub
- `AcrPull` - to pull images from Azure Container Registry

You can see an example of the assignments in [`set_managedidentity_rbac.ps1`](../scripts/templates/ManagedIdentity/set_managedidentity_rbac.ps1)
24 changes: 12 additions & 12 deletions scripts/templates/Batch/create_batch_settingsfiles.ps1
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,23 +38,23 @@ $serviceBusConnectionString = az servicebus topic authorization-rule keys list -
$identity = az identity show --resource-group $resourceGroupName --name $identityName | ConvertFrom-Json -AsHashtable
$subscriptionId = az account show -o tsv --query id

$settingsJsonWindows = Join-Path $path "settingsfile-windows.json"
$settingsJsonWindowsMi = Join-Path $path "settingsfile-windows-mi.json"
$settingsJsonWindows = Join-Path $path "settingsfile-batch-windows.json"
$settingsJsonWindowsMi = Join-Path $path "settingsfile-batch-windows-mi.json"

$settingsJsonLinux = Join-Path $path "settingsfile-linux.json"
$settingsJsonLinuxMi = Join-Path $path "settingsfile-linux-mi.json"
$settingsJsonLinux = Join-Path $path "settingsfile-batch-linux.json"
$settingsJsonLinuxMi = Join-Path $path "settingsfile-batch-linux-mi.json"

$settingsJsonWindowsQueue = Join-Path $path "settingsfile-windows-queue.json"
$settingsJsonWindowsQueueMi = Join-Path $path "settingsfile-windows-queue-mi.json"
$settingsJsonWindowsQueue = Join-Path $path "settingsfile-batch-windows-queue.json"
$settingsJsonWindowsQueueMi = Join-Path $path "settingsfile-batch-windows-queue-mi.json"

$settingsJsonLinuxQueue = Join-Path $path "settingsfile-linux-queue.json"
$settingsJsonLinuxQueueMi = Join-Path $path "settingsfile-linux-queue-mi.json"
$settingsJsonLinuxQueue = Join-Path $path "settingsfile-batch-linux-queue.json"
$settingsJsonLinuxQueueMi = Join-Path $path "settingsfile-batch-linux-queue-mi.json"

$settingsJsonWindowsQueueKv = Join-Path $path "settingsfile-windows-queue-keyvault.json"
$settingsJsonWindowsQueueKvMi = Join-Path $path "settingsfile-windows-queue-keyvault-mi.json"
$settingsJsonWindowsQueueKv = Join-Path $path "settingsfile-batch-windows-queue-keyvault.json"
$settingsJsonWindowsQueueKvMi = Join-Path $path "settingsfile-batch-windows-queue-keyvault-mi.json"

$settingsJsonLinuxQueueKv = Join-Path $path "settingsfile-linux-queue-keyvault.json"
$settingsJsonLinuxQueueKvMi = Join-Path $path "settingsfile-linux-queue-keyvault-mi.json"
$settingsJsonLinuxQueueKv = Join-Path $path "settingsfile-batch-linux-queue-keyvault.json"
$settingsJsonLinuxQueueKvMi = Join-Path $path "settingsfile-batch-linux-queue-keyvault-mi.json"

$keyFile = Join-Path $path "settingsfilekey.txt"
if($false -eq (Test-Path $keyFile))
Expand Down
15 changes: 9 additions & 6 deletions scripts/templates/ContainerApp/create_containerapp_settingsfile.ps1
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ param
[string] $identityClientId,
[string] $imageTag,
[bool] $withContainerRegistry,
## TODO: Enable Managed Identity. For now, ManagedIdentity for SQL Auth is not available on Container Apps, but SB and EH are OK.
## TODO: Enable Managed Identity. For now, ManagedIdentity for SQL Auth is not available on Container Apps. SB KEDA also requires a connection string. But EH can fully use MI
[ValidateSet("Password", "ManagedIdentity", "Both")]
[string] $authType = "Both"
)
Expand Down Expand Up @@ -96,28 +96,31 @@ if($false -eq (Test-Path $keyFile))

foreach($auth in $authTypes)
{
$params = @("containerapp", "savesettings")

if($auth -eq "ManagedIdentity" )
{
$settingsContainerApp =$settingsContainerApp + "-mi"
$settingsContainerApp = $settingsContainerApp + "-mi"
$sbAndEhArgs = @("--eventhubconnection","$($eventHubNamespaceName)|$($eventHubName)")
}
else
{
$sbAndEhArgs = @("--eventhubconnection",$eventHubConnectionString)
$params +=("--storageaccountkey",$storageAcctKey)
}
$sbAndEhArgs += @("--servicebustopicconnection",$serviceBusConnectionString)
$tmpPath = "$($settingsContainerApp).json"
Write-Host "Saving settings file to $tmpPath" -ForegroundColor DarkGreen

$params = @("containerapp", "savesettings")

$params +=("--environmentname",$containerAppEnvironmentName)
$params +=("--location",$location)
$params +=("--resourcegroup", $resourceGroupName)
$params +=("--imagetag",$imageTag)
$params +=("--settingsfile",$tmpPath)
$params +=("--settingsfilekey",$keyFile)
$params +=("--storageaccountname",$storageAccountName)
$params +=("--storageaccountkey",$storageAcctKey)


$params +=("--defaultscripttimeout",500)
$params +=("--subscriptionid",$subscriptionId)
Expand Down Expand Up @@ -150,10 +153,10 @@ foreach($auth in $authTypes)
$params += ("--clientid", $identityClientId)
}
# Container apps don't take this yet!
#$params += ("--authtype", $auth)
#$params += ("--authtype", "Password")


Write-Host $params $sbAndEhArgs -ForegroundColor DarkYellow
#Write-Host $params $sbAndEhArgs -ForegroundColor DarkYellow
Start-Process $sbmExe -ArgumentList ($params + $sbAndEhArgs)
}

7 changes: 5 additions & 2 deletions scripts/templates/aci/create_aci_settingsfile.ps1
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ if($false -eq (Test-Path $keyFile))
$sbAndEhArgs =
foreach($auth in $authTypes)
{
$params = @("aci", "savesettings")

if($auth -eq "ManagedIdentity" )
{
$settingsAci =$baseFileName + "-mi.json"
Expand All @@ -91,10 +93,11 @@ foreach($auth in $authTypes)
$settingsAci = $baseFileName + ".json"
$sbAndEhArgs = @("-sb", """$serviceBusConnectionString""")
$sbAndEhArgs += ("-eh","""$eventHubConnectionString""")
$params += ("--storageaccountkey",$storageAcctKey)
}
Write-Host "Saving settings file to $settingsAci" -ForegroundColor DarkGreen

$params = @("aci", "savesettings")

$params += ("--settingsfile", "$($settingsAci)")
$params += ("--aciname", $aciName)
$params += ("--identityname", $identityName)
Expand All @@ -103,7 +106,7 @@ foreach($auth in $authTypes)
$params += ("--acirg", $resourceGroupName)
$params += ("-kv", $keyVaultName)
$params += ("--storageaccountname", $storageAccountName)
$params += ("--storageaccountkey",$storageAcctKey)

$params += ("--defaultscripttimeout", "500")
$params += ("--subscriptionid",$subscriptionId)
$params += ("--force")
Expand Down
8 changes: 4 additions & 4 deletions scripts/templates/kubernetes/create_aks_job_yaml.ps1
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ Write-Host "Saving AKS Job YAML files to :'$outputPath'" -ForegroundColor DarkGr
# Copy sample K8s YAML files for test configs
#############################################
$scriptDir = Split-Path $script:MyInvocation.MyCommand.Path
Copy-Item (Join-Path $scriptDir sample_job.yaml) (Join-Path $outputPath basic_job.yaml)
Copy-Item (Join-Path $scriptDir sample_job_keyvault.yaml) (Join-Path $outputPath basic_job_keyvault.yaml)
Copy-Item (Join-Path $scriptDir sample_job.yaml) (Join-Path $outputPath k8s-basic_job.yaml)
Copy-Item (Join-Path $scriptDir sample_job_keyvault.yaml) (Join-Path $outputPath k8s-basic_job_keyvault.yaml)

if([string]::IsNullOrWhiteSpace($acrName) -eq $false)
{
$acrLoginServer = az acr show --resource-group $resourceGroupName --name $acrName -o tsv --query loginServer
Write-Host "Using ACR login server name:'$acrLoginServer'" -ForegroundColor DarkGreen

((Get-Content -Path (Join-Path $scriptDir sample_job.yaml)) -replace "blueskydevus", $acrLoginServer) | Out-File (Join-Path $outputPath acr_basic_job.yaml)
((Get-Content -Path (Join-Path $scriptDir sample_job_keyvault.yaml)) -replace "blueskydevus", $acrLoginServer) | Out-File (Join-Path $outputPath acr_basic_job_keyvault.yaml)
((Get-Content -Path (Join-Path $scriptDir sample_job.yaml)) -replace "blueskydevus", $acrLoginServer) | Out-File (Join-Path $outputPath k8s-acr_basic_job.yaml)
((Get-Content -Path (Join-Path $scriptDir sample_job_keyvault.yaml)) -replace "blueskydevus", $acrLoginServer) | Out-File (Join-Path $outputPath k8s-acr_basic_job_keyvault.yaml)
}
Loading

0 comments on commit 752fea7

Please sign in to comment.