Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azurerm_machine_learning_workspace - Add support for serverless_compute #25660

Merged
merged 10 commits into from
May 29, 2024
+193 −0
Merged

azurerm_machine_learning_workspace - Add support for serverless_compute #25660

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
66324eb
add support for serverlesscompute
xuzhang3 Apr 18, 2024
9378f15
merge upstream
xuzhang3 Apr 18, 2024
e2fa392
add support for serverlesscompute
xuzhang3 Apr 19, 2024
df90c80
fix test case
xuzhang3 May 8, 2024
8efd38c
fix test case
xuzhang3 May 9, 2024
d92a211
fix test case
xuzhang3 May 9, 2024
fd9afde
fix test case
xuzhang3 May 9, 2024
58578c3
change default value
xuzhang3 May 9, 2024
3b93028
1. rename custom_subnet_id to custom_subnet_id
xuzhang3 May 28, 2024
110fbcb
formant
xuzhang3 May 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions internal/services/machinelearning/machine_learning_workspace_resource.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,26 @@ func resourceMachineLearningWorkspace() *pluginsdk.Resource {
Default: false,
},

"serverless_compute": {
Type: pluginsdk.TypeList,
Optional: true,
MaxItems: 1,
Elem: &pluginsdk.Resource{
Schema: map[string]*pluginsdk.Schema{
"subnet_id": {
Type: pluginsdk.TypeString,
Optional: true,
ValidateFunc: commonids.ValidateSubnetID,
},
"public_ip_enabled": {
Type: pluginsdk.TypeBool,
Optional: true,
Default: false,
},
},
},
},

"discovery_url": {
Type: pluginsdk.TypeString,
Computed: true,
Expand Down Expand Up @@ -315,6 +335,22 @@ func resourceMachineLearningWorkspaceCreateOrUpdate(d *pluginsdk.ResourceData, m
},
}

serverlessCompute := expandMachineLearningWorkspaceServerlessCompute(d.Get("serverless_compute").([]interface{}))
if serverlessCompute != nil {
if *serverlessCompute.ServerlessComputeNoPublicIP && serverlessCompute.ServerlessComputeCustomSubnet == nil && !networkAccessBehindVnetEnabled {
return fmt.Errorf("`public_ip_enabled` must be set to `true` if `subnet_id` is not set and `public_network_access_enabled` is `false`")
}

if serverlessCompute.ServerlessComputeCustomSubnet == nil {
oldVal, newVal := d.GetChange("serverless_compute.0.public_ip_enabled")
if oldVal.(bool) && !newVal.(bool) {
return fmt.Errorf(" Not supported to update `public_ip_enabled` from `true` to `false` when `subnet_id` is null or empty")
}
}
Comment on lines +344 to +349
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic looks like it belongs in a CustomizeDiff

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will add a CustomizeDiff for this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried several ways to update this property, customers can still update it but it takes several steps. We can omit `CustomizeDiff``, cx can update the settings according to the error message.

  1. Set the serverless_compute.subnet_id
  2. They may need to add the a private link for this machine learning workspace.
  3. Update the serverless_compute.public_ip_enabled from true to false

Error message cx may get:

Error: creating/updating Workspace (Subscription: "<sub id>"
Resource Group Name: "<rg name>"
Workspace Name: "xz3workspace16"): unexpected status 400 (400 Bad Request) with error: ValidationError: Found errors while validating Subnet Id <subnet id>. Subnet Diagnostic Result = WorkspaceVnet:ErrorCode=WorkspaceVnetConflictErrorMessage=The workspace does not have public access and does not have a VNET. Pleas
e either add a VNET to the workspace or enable public access. For more information, please refer to https://docs.microsoft.com/en-us/azure/machine-learning/how-to-configure-private-link?tabs=python#add-a-private-endpoint-to-a-workspace.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation @xuzhang3.

Is this something we can do in the resource by calling the CreateOrUpdate method twice in the same apply if public_ip_enabled changes from true -> false?

e.g. First CreateOrUpdate call sets subnet_id
Second CreateOrUpdate call updated public_ip_enabled from true -> false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, the operation needs to be split into several steps

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that because of

  1. They may need to add the a private link for this machine learning workspace.

Is this an external thing that they need to configure? What is meant by may are there situations where they don't need to?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, I create a new MLW with following config and I want to set the serverless_compute.public_ip_enabled to false, I need to update the public_network_access_enabled and serverless_compute.subnet_id and I need add a private endpoint for this MLW then I can update the serverless_compute.public_ip_enabled, this is one of the scenario. Other scenario depends on the MLW configurations.

resource "azurerm_machine_learning_workspace" "example" {
  name                    = "mymlwexample"
  location                = azurerm_resource_group.example.location
  resource_group_name     = azurerm_resource_group.example.name
  application_insights_id = azurerm_application_insights.example.id
  key_vault_id            = azurerm_key_vault.example.id
  storage_account_id      = azurerm_storage_account.example.id

#  public_network_access_enabled = true

  image_build_compute_name = "terraformComputeUpdate11"

  serverless_compute {
#    subnet_id         = azurerm_subnet.test.id
    public_ip_enabled = true
  }

  identity {
    type = "SystemAssigned"
  }
}

}

workspace.Properties.ServerlessComputeSettings = serverlessCompute

if networkAccessBehindVnetEnabled {
workspace.Properties.PublicNetworkAccess = pointer.To(workspaces.PublicNetworkAccessEnabled)
}
Expand Down Expand Up @@ -414,6 +450,7 @@ func resourceMachineLearningWorkspaceRead(d *pluginsdk.ResourceData, meta interf
d.Set("v1_legacy_mode_enabled", props.V1LegacyMode)
d.Set("workspace_id", props.WorkspaceId)
d.Set("managed_network", flattenMachineLearningWorkspaceManagedNetwork(props.ManagedNetwork))
d.Set("serverless_compute", flattenMachineLearningWorkspaceServerlessCompute(props.ServerlessComputeSettings))

kvId, err := commonids.ParseKeyVaultIDInsensitively(*props.KeyVault)
if err != nil {
Expand Down Expand Up @@ -670,3 +707,39 @@ func flattenMachineLearningWorkspaceManagedNetwork(i *workspaces.ManagedNetworkS

return &[]interface{}{out}
}

func expandMachineLearningWorkspaceServerlessCompute(i []interface{}) *workspaces.ServerlessComputeSettings {
if len(i) == 0 || i[0] == nil {
return nil
}

v := i[0].(map[string]interface{})

serverlessCompute := workspaces.ServerlessComputeSettings{
ServerlessComputeNoPublicIP: pointer.To(!v["public_ip_enabled"].(bool)),
}

if subnetId, ok := v["subnet_id"].(string); ok && subnetId != "" {
serverlessCompute.ServerlessComputeCustomSubnet = pointer.To(subnetId)
}

return &serverlessCompute
}

func flattenMachineLearningWorkspaceServerlessCompute(i *workspaces.ServerlessComputeSettings) *[]interface{} {
if i == nil {
return &[]interface{}{}
}

out := map[string]interface{}{}

if i.ServerlessComputeCustomSubnet != nil {
out["subnet_id"] = *i.ServerlessComputeCustomSubnet
}

if i.ServerlessComputeNoPublicIP != nil {
out["public_ip_enabled"] = !*i.ServerlessComputeNoPublicIP
}

return &[]interface{}{out}
}
108 changes: 108 additions & 0 deletions internal/services/machinelearning/machine_learning_workspace_resource_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,41 @@ func TestAccMachineLearningWorkspace_purgeSoftDelete(t *testing.T) {
})
}

func TestAccMachineLearningWorkspace_serverlessCompute(t *testing.T) {
data := acceptance.BuildTestData(t, "azurerm_machine_learning_workspace", "test")
r := WorkspaceResource{}

data.ResourceTest(t, r, []acceptance.TestStep{
{
Config: r.serverlessCompute(data),
Check: acceptance.ComposeTestCheckFunc(
check.That(data.ResourceName).ExistsInAzure(r),
check.That(data.ResourceName).Key("serverless_compute.#").HasValue("1"),
check.That(data.ResourceName).Key("serverless_compute.0.subnet_id").Exists(),
check.That(data.ResourceName).Key("serverless_compute.0.public_ip_enabled").HasValue("false"),
),
},
data.ImportStep(),
})
}

func TestAccMachineLearningWorkspace_serverlessCompute_withoutSubnet(t *testing.T) {
data := acceptance.BuildTestData(t, "azurerm_machine_learning_workspace", "test")
r := WorkspaceResource{}

data.ResourceTest(t, r, []acceptance.TestStep{
{
Config: r.serverlessComputeWithoutSubnet(data),
Check: acceptance.ComposeTestCheckFunc(
check.That(data.ResourceName).ExistsInAzure(r),
check.That(data.ResourceName).Key("serverless_compute.#").HasValue("1"),
check.That(data.ResourceName).Key("serverless_compute.0.public_ip_enabled").HasValue("true"),
),
},
data.ImportStep(),
})
}

func (r WorkspaceResource) Exists(ctx context.Context, client *clients.Client, state *pluginsdk.InstanceState) (*bool, error) {
workspacesClient := client.MachineLearning.Workspaces
id, err := workspaces.ParseWorkspaceID(state.ID)
Expand Down Expand Up @@ -1166,3 +1201,76 @@ resource "azurerm_machine_learning_workspace" "test" {
}
`, template, data.RandomInteger)
}

func (r WorkspaceResource) serverlessCompute(data acceptance.TestData) string {
template := r.template(data)
return fmt.Sprintf(`
%s

resource "azurerm_virtual_network" "test" {
name = "acctestvirtnet%[2]d"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
}
resource "azurerm_subnet" "test" {
name = "internal"
resource_group_name = azurerm_resource_group.test.name
virtual_network_name = azurerm_virtual_network.test.name
address_prefixes = ["10.0.2.0/24"]
}

resource "azurerm_machine_learning_workspace" "test" {
name = "acctest-MLW-%[2]d"
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
application_insights_id = azurerm_application_insights.test.id
key_vault_id = azurerm_key_vault.test.id
storage_account_id = azurerm_storage_account.test.id

serverless_compute {
subnet_id = azurerm_subnet.test.id
}
Comment on lines +1231 to +1233
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also add a test for the case where we don't supply subnet_id


identity {
type = "SystemAssigned"
}
}
`, template, data.RandomInteger)
}

func (r WorkspaceResource) serverlessComputeWithoutSubnet(data acceptance.TestData) string {
template := r.template(data)
return fmt.Sprintf(`
%s

resource "azurerm_virtual_network" "test" {
name = "acctestvirtnet%[2]d"
address_space = ["10.0.0.0/16"]
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
}
resource "azurerm_subnet" "test" {
name = "internal"
resource_group_name = azurerm_resource_group.test.name
virtual_network_name = azurerm_virtual_network.test.name
address_prefixes = ["10.0.2.0/24"]
}

resource "azurerm_machine_learning_workspace" "test" {
name = "acctest-MLW-%[2]d"
location = azurerm_resource_group.test.location
resource_group_name = azurerm_resource_group.test.name
application_insights_id = azurerm_application_insights.test.id
key_vault_id = azurerm_key_vault.test.id
storage_account_id = azurerm_storage_account.test.id

serverless_compute {
public_ip_enabled = true
}
identity {
type = "SystemAssigned"
}
}
`, template, data.RandomInteger)
}
12 changes: 12 additions & 0 deletions website/docs/r/machine_learning_workspace.html.markdown
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -390,6 +390,8 @@ The following arguments are supported:

* `sku_name` - (Optional) SKU/edition of the Machine Learning Workspace, possible values are `Free`, `Basic`, `Standard` and `Premium`. Defaults to `Basic`.

* `serverless_compute` - (Optional) A `serverless_compute` block as defined below.

* `tags` - (Optional) A mapping of tags to assign to the resource.

---
Expand Down Expand Up @@ -422,6 +424,16 @@ An `managed_network` block supports the following:

---

A `serverless_compute` block supports the following:

* `subnet_id` - (Optional) The ID of an existing Virtual Network Subnet in which the serverless compute nodes should be deployed to.

* `public_ip_enabled` - (Optional) Should serverless compute nodes deployed in a custom Virtual Network have public IP addresses enabled for a workspace with private endpoint? Defaults to `false`.

~> **Note:** `public_ip_enabled` cannot be updated from `true` to `false` when `subnet_id` is not set. `public_ip_enabled` must be set to `true` if `subnet_id` is not set and when `public_network_access_enabled` is `false`.

---

An `feature_store` block supports the following:

* `computer_spark_runtime_version` - (Optional) The version of Spark runtime.
Expand Down
Loading