New resource `azurerm_azure_ai_services` #26008

xuzhang3 · 2024-05-17T09:18:39Z

Community Note

Please vote on this PR by adding a 👍 reaction to the original PR to help the community and maintainers prioritize for review
Please do not leave "+1" or "me too" comments, they generate extra noise for PR followers and do not help prioritize for review

Description

PR Checklist

I have followed the guidelines in our Contributing Documentation.
I have checked to ensure there aren't other open Pull Requests for the same update/change.
I have checked if my changes close any open issues. If so please include appropriate closing keywords below.
I have updated/added Documentation as required written in a helpful and kind way to assist users that may be unfamiliar with the resource / data source.
I have used a meaningful PR title to help maintainers and other users understand this change and help prevent duplicate work.
For example: “resource_name_here - description of change e.g. adding property new_property_name_here”

Changes to existing Resource / Data Source

I have added an explanation of what my changes do and why I'd like you to include them (This may be covered by linking to an issue above, but may benefit from additional explanation).
I have written new tests for my resource or datasource changes & updated any relevent documentation.
I have successfully run tests with my changes locally. If not, please provide details on testing challenges that prevented you running the tests.
(For changes that include a state migration only). I have manually tested the migration path between relevant versions of the provider.

Testing

My submission includes Test coverage as described in the Contribution Guide and the tests pass. (if this is not possible for any reason, please include details of why you did or could not add test coverage)

Change Log

Below please provide what should go into the changelog (if anything) conforming to the Changelog Format documented here.

azurerm_resource - support for the thing1 property [GH-00000]

This is a (please select all that apply):

Bug Fix
New Feature (ie adding a service, resource, or data source)
Enhancement
Breaking Change

Related Issue(s)

Fixes #0000

Note

If this PR changes meaningfully during the course of review please update the title and description as required.

xuzhang3 · 2024-05-17T09:23:31Z

=== RUN   TestAccCognitiveAIServicesAccount_basic
=== PAUSE TestAccCognitiveAIServicesAccount_basic
=== RUN   TestAccCognitiveAIServicesAccount_requiresImport
=== PAUSE TestAccCognitiveAIServicesAccount_requiresImport
=== RUN   TestAccCognitiveAIServicesAccount_complete
=== PAUSE TestAccCognitiveAIServicesAccount_complete
=== RUN   TestAccCognitiveAIServicesAccount_update
=== PAUSE TestAccCognitiveAIServicesAccount_update
=== RUN   TestAccCognitiveAIServicesAccount_networkACLsVirtualNetworkRules
=== PAUSE TestAccCognitiveAIServicesAccount_networkACLsVirtualNetworkRules
=== RUN   TestAccCognitiveAIServicesAccount_networkACLs
=== PAUSE TestAccCognitiveAIServicesAccount_networkACLs
=== RUN   TestAccCognitiveAIServicesAccount_identity
=== PAUSE TestAccCognitiveAIServicesAccount_identity
=== RUN   TestAccCognitiveAIServicesAccount_customerManagedKey
=== PAUSE TestAccCognitiveAIServicesAccount_customerManagedKey
=== CONT  TestAccCognitiveAIServicesAccount_basic
=== CONT  TestAccCognitiveAIServicesAccount_networkACLsVirtualNetworkRules
=== CONT  TestAccCognitiveAIServicesAccount_identity
=== CONT  TestAccCognitiveAIServicesAccount_requiresImport
=== CONT  TestAccCognitiveAIServicesAccount_networkACLs
=== CONT  TestAccCognitiveAIServicesAccount_update
--- PASS: TestAccCognitiveAIServicesAccount_requiresImport (288.59s)
--- PASS: TestAccCognitiveAIServicesAccount_networkACLs (419.91s)
--- PASS: TestAccCognitiveAIServicesAccount_identity (505.50s)
--- PASS: TestAccCognitiveAIServicesAccount_networkACLsVirtualNetworkRules (509.13s)
--- PASS: TestAccCognitiveAIServicesAccount_customerManagedKey (1402.96s)
--- PASS: TestAccCognitiveAIServicesAccount_update (1459.11s)
--- PASS: TestAccCognitiveAIServicesAccount_complete (1530.32s)
PASS
ok      github.com/hashicorp/terraform-provider-azurerm/internal/services/cognitive     1546.807s

tombuildsstuff

hey @xuzhang3

I've taken a look through and left some comments inline, but there's a few questions in there - once those are resolved/clarified we can take another look through 👍

Thanks!

tombuildsstuff · 2024-05-22T16:38:25Z

website/docs/guides/features-block.html.markdown

@@ -114,6 +118,8 @@ The `features` block supports the following:

 * `cognitive_account` - (Optional) A `cognitive_account` block as defined below.

+* `cognitive_account_ai_services` - (Optional) A `cognitive_account_ai_services` block as defined below.


can we reuse the cognitive_account block for this purpose?

Suggested change

* `cognitive_account_ai_services` - (Optional) A `cognitive_account_ai_services` block as defined below.

tombuildsstuff · 2024-05-22T16:38:31Z

website/docs/guides/features-block.html.markdown

+    cognitive_account_ai_services {
+      purge_soft_delete_on_destroy = true
+    }


can we reuse the cognitive_account block for this purpose?

Suggested change

cognitive_account_ai_services {

purge_soft_delete_on_destroy = true

}

@xuzhang3 could we please do as tom suggests here and resue the cognitive account block and make ai_services a sub block of it?

updated, unused configs deleted.

tombuildsstuff · 2024-05-22T16:39:49Z

internal/features/defaults.go

+		CognitiveAccountAIServices: CognitiveAccountAIServicesFeatures{
+			PurgeSoftDeleteOnDestroy: true,
+		},


we can reuse the CognitiveAccount block/toggle here

tombuildsstuff · 2024-05-22T16:39:56Z

internal/features/user_flags.go

+	AppConfiguration           AppConfigurationFeatures
+	ApplicationInsights        ApplicationInsightFeatures
+	CognitiveAccount           CognitiveAccountFeatures
+	CognitiveAccountAIServices CognitiveAccountAIServicesFeatures


Suggested change

CognitiveAccountAIServices CognitiveAccountAIServicesFeatures

tombuildsstuff · 2024-05-22T16:40:04Z

internal/provider/features.go

+		"cognitive_account_ai_services": {
+			Type:     pluginsdk.TypeList,
+			Optional: true,
+			MaxItems: 1,
+			Elem: &pluginsdk.Resource{
+				Schema: map[string]*pluginsdk.Schema{
+					"purge_soft_delete_on_destroy": {
+						Type:     pluginsdk.TypeBool,
+						Optional: true,
+						Default:  true,
+					},
+				},
+			},
+		},


can we reuse the cognitive_account block for this purpose?

Suggested change

"cognitive_account_ai_services": {

Type: pluginsdk.TypeList,

Optional: true,

MaxItems: 1,

Elem: &pluginsdk.Resource{

Schema: map[string]*pluginsdk.Schema{

"purge_soft_delete_on_destroy": {

Type: pluginsdk.TypeBool,

Optional: true,

Default: true,

},

},

},

},

tombuildsstuff · 2024-05-22T16:42:59Z

internal/services/cognitive/cognitive_account_ai_services_resource.go

+		"public_network_access_enabled": {
+			Type:     pluginsdk.TypeBool,
+			Optional: true,
+			Default:  true,
+		},


can we expose this as public_network_access with the constant values here?

rename to public_network_access ? Sorry, I didn't get the point.

We're moving away from using public_network_access_enabled, since the Azure API is gradually supporting three values for this field (Disabled, Enabled and SecuredByPerimeter) - as such new resources should expose this as a string field (we'll be making this a commonschema field in hashicorp/go-azure-helpers#238) - and existing resources will be updated in time.

As such, can we make this a string field, public_network_access, with the constant values being the possible values - rather than a boolean?

public_network_access_enabled has been renamed to public_network_access

tombuildsstuff · 2024-05-22T16:43:36Z

internal/services/cognitive/cognitive_account_ai_services_resource.go

+			// also lock on the Virtual Network ID's since modifications in the networking stack are exclusive
+			virtualNetworkNames := make([]string, 0)
+			for _, v := range subnetIds {
+				subnetId, err := commonids.ParseSubnetIDInsensitively(v)


These'll be validated from the config, so can be parsed directly:

Suggested change

subnetId, err := commonids.ParseSubnetIDInsensitively(v)

subnetId, err := commonids.ParseSubnetID(v)

tombuildsstuff · 2024-05-22T16:44:19Z

internal/services/cognitive/cognitive_account_ai_services_resource.go

+				return err
+			}
+
+			id := cognitiveservicesaccounts.NewAccountID(subscriptionId, model.ResourceGroupName, model.Name)


we can parse the ID from metadata.ID

internal/services/cognitive/cognitive_account_ai_services_resource.go

tombuildsstuff · 2024-05-22T16:45:50Z

internal/services/cognitive/cognitive_account_ai_services_resource.go

+func aiServicesAccountStateRefreshFunc(ctx context.Context, client *cognitiveservicesaccounts.CognitiveServicesAccountsClient, id cognitiveservicesaccounts.AccountId) pluginsdk.StateRefreshFunc {
+	return func() (interface{}, string, error) {
+		res, err := client.AccountsGet(ctx, id)
+		if err != nil {
+			return nil, "", fmt.Errorf("polling for %s: %+v", id, err)
+		}
+
+		if res.Model != nil && res.Model.Properties != nil && res.Model.Properties.ProvisioningState != nil {
+			return res, string(*res.Model.Properties.ProvisioningState), nil
+		}
+		return nil, "", fmt.Errorf("unable to read provisioning state")
+	}
+}


hashicorp/go-azure-sdk should do this for you - is this an LRO/is there a bug in the API Definitions here?

this is copied from Cognitive Account and I just remove the unnecessary properties and codes

Sure, but presumably when doing so you've looked to see if these can be removed? As such, what's the API actually doing here, is the Swagger wrong and this is an LRO - or is there also an API bug going on here?

thanks, will remove the refresh part. There's only one API for cognitive account, keep everything the same is my first choice.

@xuzhang3 with respect, you still haven’t answered my question: what’s the API actually doing here? Can you show the HTTP Request/Response of what’s being sent/coming back over the wire, so that we can understand if there’s another issue here?

@tombuildsstuff the SDK can help handle the request, so I removed the state refresh and replaced by future.Poller.PollUntilDone

2. remove refresh codes 3. add lock for vnet resources

xuzhang3 · 2024-05-29T10:01:46Z

@tombuildsstuff all required changes has been updated

katbyte · 2024-06-04T02:53:57Z

internal/services/cognitive/cognitive_account_ai_services_resource.go

+					"key_vault_key_id": {
+						Type:         pluginsdk.TypeString,
+						Required:     true,
+						ValidateFunc: keyVaultValidate.NestedItemIdWithOptionalVersion,
+					},


@xuzhang3 could we add in a mhsm_key_vault_key property like we now have in storage_account resource so we can support them properly?

katbyte · 2024-06-04T02:54:28Z

internal/services/cognitive/cognitive_account_ai_services_resource.go

+
+		"identity": commonschema.SystemAssignedUserAssignedIdentityOptional(),
+
+		"local_auth_enabled": {


@xuzhang3 could wemplease update this as tom asked?

xuzhang3 · 2024-08-02T08:46:18Z

@stephybun updated and all test passed in my local env

stephybun

Thanks @xuzhang3. In addition to some of the unresolved comments and new comments left-inline, can you also rename all of the files from azure_ai_services_*.go to ai_services_*.go to reflect the resource name properly?

Once all of the comments are resolved and the file renames done I think this should be good to go.

stephybun · 2024-08-09T09:37:46Z

internal/services/cognitive/azure_ai_services_resource.go

+				Kind:     utils.String("AIServices"),
+				Location: utils.String(azure.NormalizeLocation(model.Location)),
+				Sku: &cognitiveservicesaccounts.Sku{


Hasn't been addressed

stephybun · 2024-08-09T09:37:57Z

internal/services/cognitive/azure_ai_services_resource.go

+				return fmt.Errorf("waiting for creating of %s: %+v", id, err)
+			}
+
+			customMangedKey, err := expandAzureAIServicesCustomerManagedKey(model.CustomerManagedKey)


Hasn't been addressed

stephybun · 2024-08-13T05:56:00Z

internal/services/cognitive/azure_ai_services_resource.go

+			if customMangedKey != nil {
+				props.Properties.Encryption = customMangedKey
+				futureUpdate, err := client.AccountsUpdate(ctx, id, props)
+				if err != nil {
+					return fmt.Errorf("updating %s: %+v", id, err)
+				}
+
+				if err := futureUpdate.Poller.PollUntilDone(ctx); err != nil {
+					return fmt.Errorf("waiting for updating of %s: %+v", id, err)
+				}
+			}


Why does this take so long for a HSM encryption and not for a regular KV, sounds like an API bug. Can you please:

Raise this with the service team

Leave a comment above this with the explanation that it takes longer with HSM encryption and that the update is more stable.

stephybun · 2024-08-13T05:56:45Z

internal/services/cognitive/azure_ai_services_resource.go

+			future, err := client.AccountsUpdate(ctx, *id, *props)
+			if err != nil {
+				return fmt.Errorf("updating %s: %+v", id, err)
+			}
+
+			if err := future.Poller.PollUntilDone(ctx); err != nil {
+				return fmt.Errorf("waiting for updating of %s: %+v", id, err)
+			}
+
+			return nil


Suggested change

future, err := client.AccountsUpdate(ctx, *id, *props)

if err != nil {

return fmt.Errorf("updating %s: %+v", id, err)

}

if err := future.Poller.PollUntilDone(ctx); err != nil {

return fmt.Errorf("waiting for updating of %s: %+v", id, err)

}

return nil

if err := client.AccountsUpdateThenPoll(ctx, *id, *props); err != nil {

return fmt.Errorf("updating %s: %+v", id, err)

}

return nil

stephybun · 2024-08-14T07:22:48Z

internal/services/cognitive/azure_ai_services_resource.go

+				futureUpdate, err := client.AccountsUpdate(ctx, id, props)
+				if err != nil {
+					return fmt.Errorf("updating %s: %+v", id, err)
+				}
+
+				if err := futureUpdate.Poller.PollUntilDone(ctx); err != nil {
+					return fmt.Errorf("waiting for updating of %s: %+v", id, err)
+				}


Suggested change

futureUpdate, err := client.AccountsUpdate(ctx, id, props)

if err != nil {

return fmt.Errorf("updating %s: %+v", id, err)

}

if err := futureUpdate.Poller.PollUntilDone(ctx); err != nil {

return fmt.Errorf("waiting for updating of %s: %+v", id, err)

}

if err := client.AccountsUpdate(ctx, id, props); err != nil {

return fmt.Errorf("updating %s: %+v", id, err)

}

stephybun · 2024-08-14T12:39:22Z

website/docs/r/azure_ai_services.html.markdown

Could you rename this file as well to reflect the resource name please?

Done, file name has been renamed.

stephybun

Thanks @xuzhang3 LGTM 👍

github-actions · 2024-09-16T02:15:12Z

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

xuzhang3 added 2 commits May 16, 2024 14:59

add support for Cognitive account AI services

a53fbd2

rename to azurerm_cognitive_account_ai_services

70ed5af

github-actions bot added documentation service/cognitive-services size/XL labels May 17, 2024

merge upstream update

32fc6e4

github-actions bot added the waiting-response label May 17, 2024

foramt code

e9fd983

github-actions bot added waiting-response and removed waiting-response labels May 17, 2024

fix feature unit test

20c701f

github-actions bot removed the waiting-response label May 17, 2024

tombuildsstuff suggested changes May 22, 2024

View reviewed changes

github-actions bot added the waiting-response label May 22, 2024

1. reuseCognitiveAccount.PurgeSoftDeleteOnDestroy

7e3bc7b

2. remove refresh codes 3. add lock for vnet resources

github-actions bot removed the waiting-response label May 24, 2024

update create/update request poller

d624256

github-actions bot added the waiting-response label May 24, 2024

rename public_network_access_enabled to public_network_access

388a5fe

github-actions bot removed the waiting-response label May 27, 2024

This comment was marked as duplicate.

Sign in to view

tombuildsstuff added the waiting-response label May 27, 2024

github-actions bot added waiting-response and removed waiting-response labels May 29, 2024

katbyte requested changes Jun 4, 2024

View reviewed changes

xuzhang3 added 2 commits June 4, 2024 15:20

rename local_auth_enabled to local_authentication_enabled

34db8a6

add support for KV-hsm encrption

07eddfa

github-actions bot removed the waiting-response label Jun 14, 2024

rename azure_ai_services to ai_services

4d87ee3

github-actions bot added waiting-response and removed waiting-response labels Aug 2, 2024

remove unused code

22bbff7

github-actions bot added waiting-response and removed waiting-response labels Aug 2, 2024

format import

c17aca7

github-actions bot removed the waiting-response label Aug 2, 2024

xuzhang3 closed this Aug 14, 2024

xuzhang3 deleted the f/cognitive_aiservice branch August 14, 2024 02:44

xuzhang3 restored the f/cognitive_aiservice branch August 14, 2024 02:52

xuzhang3 reopened this Aug 14, 2024

stephybun mentioned this pull request Aug 14, 2024

azurerm_cognitive_account - add AIServices as a new option for Kind #27032

Closed

13 tasks

stephybun requested changes Aug 14, 2024

View reviewed changes

github-actions bot added the waiting-response label Aug 14, 2024

stephybun removed the blocked label Aug 14, 2024

optimize create/update

9ee7a6d

github-actions bot removed the waiting-response label Aug 14, 2024

stephybun requested changes Aug 14, 2024

View reviewed changes

github-actions bot added the waiting-response label Aug 14, 2024

rename azure_ai_services to ai_services

f93dbc6

github-actions bot removed the waiting-response label Aug 15, 2024

stephybun added the new-resource label Aug 15, 2024

stephybun approved these changes Aug 15, 2024

View reviewed changes

stephybun merged commit 65334cd into hashicorp:main Aug 15, 2024
35 checks passed

github-actions bot added this to the v3.116.0 milestone Aug 15, 2024

stephybun added a commit that referenced this pull request Aug 16, 2024

Update for #27055 #26008 #26991 #26922

65601d5

github-actions bot locked as resolved and limited conversation to collaborators Sep 16, 2024

		@@ -114,6 +118,8 @@ The `features` block supports the following:

		* `cognitive_account` - (Optional) A `cognitive_account` block as defined below.

		* `cognitive_account_ai_services` - (Optional) A `cognitive_account_ai_services` block as defined below.

	cognitive_account_ai_services {
	purge_soft_delete_on_destroy = true
	}

	subnetId, err := commonids.ParseSubnetIDInsensitively(v)
	subnetId, err := commonids.ParseSubnetID(v)


		"identity": commonschema.SystemAssignedUserAssignedIdentityOptional(),

		"local_auth_enabled": {

New resource azurerm_azure_ai_services #26008

New resource azurerm_azure_ai_services #26008

Conversation

xuzhang3 commented May 17, 2024

Community Note

Description

PR Checklist

Changes to existing Resource / Data Source

Testing

Change Log

Related Issue(s)

xuzhang3 commented May 17, 2024

tombuildsstuff left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xuzhang3 May 27, 2024 • edited Loading

Choose a reason for hiding this comment

This comment was marked as duplicate.

xuzhang3 commented May 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xuzhang3 commented Aug 2, 2024

stephybun left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephybun left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 16, 2024

New resource `azurerm_azure_ai_services` #26008

New resource `azurerm_azure_ai_services` #26008

xuzhang3 May 27, 2024 •

edited

Loading