azurerm_netapp_volume - support for smb_access_based_enumeration and smb_non_browsable

[Orexii]

This PR provides the following:

Azure NetApp Files Volume's Non-browsable shares implementation
Azure NetApp Files Volume's Access-based enumeration implementation
Documentation for both of the above features

Please note that the failing test cases below are external to the changes presented in this PR. The failures presently exist on the main branch as well. The failing test cases fall into two error categories:

1. Pandora related polling issue, tracked in 3791. This affects tests:
   TestAccNetAppVolume_volEncryptionCmkUserAssigned
   TestAccNetAppVolume_updateExportPolicyRule
   TestAccNetAppVolume_update
2. Seemingly api change related issue causing changes in the snapshot_directory_visible property of a volume to be ignored, and consequently casing the test to fail:
   TestAccNetAppVolume_crossRegionReplication

user [ ~/go/src/github.com/terraform-providers/terraform-provider-azurerm ]$ make acctests SERVICE='netapp' TESTARGS=' -parallel 5 -run=TestAccNetAppVolume_' TESTTIMEOUT='1200m'
==> Checking that code complies with gofmt requirements...
==> Checking that Custom Timeouts are used...
==> Checking that acceptance test packages are used...
TF_ACC=1 go test -v ./internal/services/netapp -parallel 5 -run=TestAccNetAppVolume_ -timeout 1200m -ldflags="-X=github.com/hashicorp/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccNetAppVolume_basic
=== PAUSE TestAccNetAppVolume_basic
=== RUN   TestAccNetAppVolume_availabilityZone
=== PAUSE TestAccNetAppVolume_availabilityZone
=== RUN   TestAccNetAppVolume_nfsv41
=== PAUSE TestAccNetAppVolume_nfsv41
=== RUN   TestAccNetAppVolume_standardNetworkFeature
=== PAUSE TestAccNetAppVolume_standardNetworkFeature
=== RUN   TestAccNetAppVolume_snapshotPolicy
=== PAUSE TestAccNetAppVolume_snapshotPolicy
=== RUN   TestAccNetAppVolume_crossRegionReplication
=== PAUSE TestAccNetAppVolume_crossRegionReplication
=== RUN   TestAccNetAppVolume_nfsv3FromSnapshot
=== PAUSE TestAccNetAppVolume_nfsv3FromSnapshot
=== RUN   TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleFalse
=== PAUSE TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleFalse
=== RUN   TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleTrue
=== PAUSE TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleTrue
=== RUN   TestAccNetAppVolume_requiresImport
=== PAUSE TestAccNetAppVolume_requiresImport
=== RUN   TestAccNetAppVolume_complete
=== PAUSE TestAccNetAppVolume_complete
=== RUN   TestAccNetAppVolume_update
=== PAUSE TestAccNetAppVolume_update
=== RUN   TestAccNetAppVolume_updateSubnet
=== PAUSE TestAccNetAppVolume_updateSubnet
=== RUN   TestAccNetAppVolume_updateExportPolicyRule
=== PAUSE TestAccNetAppVolume_updateExportPolicyRule
=== RUN   TestAccNetAppVolume_volEncryptionCmkUserAssigned
=== PAUSE TestAccNetAppVolume_volEncryptionCmkUserAssigned
=== RUN   TestAccNetAppVolume_volEncryptionCmkSystemAssigned
=== PAUSE TestAccNetAppVolume_volEncryptionCmkSystemAssigned
=== CONT  TestAccNetAppVolume_basic
=== CONT  TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleTrue
=== CONT  TestAccNetAppVolume_snapshotPolicy
=== CONT  TestAccNetAppVolume_standardNetworkFeature
=== CONT  TestAccNetAppVolume_nfsv41
--- PASS: TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleTrue (1974.94s)
=== CONT  TestAccNetAppVolume_availabilityZone
--- PASS: TestAccNetAppVolume_standardNetworkFeature (2034.46s)
=== CONT  TestAccNetAppVolume_nfsv3FromSnapshot
--- PASS: TestAccNetAppVolume_snapshotPolicy (2102.35s)
=== CONT  TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleFalse
--- PASS: TestAccNetAppVolume_nfsv41 (2180.75s)
=== CONT  TestAccNetAppVolume_updateSubnet
--- PASS: TestAccNetAppVolume_basic (2233.64s)
=== CONT  TestAccNetAppVolume_volEncryptionCmkSystemAssigned
--- PASS: TestAccNetAppVolume_availabilityZone (1684.80s)
=== CONT  TestAccNetAppVolume_volEncryptionCmkUserAssigned
    testcase.go:113: Step 1/2 error: Error running apply: exit status 1

        Error: updating Net App Account (Subscription: "<sub id>"
        Resource Group Name: "acctestRG-netapp-240215113651003320"
        Net App Account Name: "acctest-NetAppAccount-240215113651003320"): polling after AccountsUpdate: `result.Status` was nil/empty - `op.Status` was "Patching" / `op.Properties.ProvisioningState` was ""

          with azurerm_netapp_account_encryption.test,
          on terraform_plugin_test.tf line 119, in resource "azurerm_netapp_account_encryption" "test":
         119: resource "azurerm_netapp_account_encryption" "test" {

        updating Net App Account (Subscription:
        "<sub id>"
        Resource Group Name: "acctestRG-netapp-240215113651003320"
        Net App Account Name: "acctest-NetAppAccount-240215113651003320"): polling
        after AccountsUpdate: `result.Status` was nil/empty - `op.Status` was
        "Patching" / `op.Properties.ProvisioningState` was ""
--- FAIL: TestAccNetAppVolume_volEncryptionCmkUserAssigned (253.63s)
=== CONT  TestAccNetAppVolume_updateExportPolicyRule
--- PASS: TestAccNetAppVolume_nfsv3SnapshotDirectoryVisibleFalse (2021.81s)
=== CONT  TestAccNetAppVolume_complete
--- PASS: TestAccNetAppVolume_volEncryptionCmkSystemAssigned (1982.90s)
=== CONT  TestAccNetAppVolume_update
=== NAME  TestAccNetAppVolume_updateExportPolicyRule
    testcase.go:113: Step 3/4 error: Error running apply: exit status 1

        Error: updating Volume (Subscription: "<sub id>"
        Resource Group Name: "acctestRG-netapp-240215113651003253"
        Net App Account Name: "acctest-NetAppAccount-240215113651003253"
        Capacity Pool Name: "acctest-NetAppPool-240215113651003253"
        Volume Name: "acctest-NetAppVolume-240215113651003253"): polling after Update: `result.Status` was nil/empty - `op.Status` was "Patching" / `op.Properties.ProvisioningState` was ""

          with azurerm_netapp_volume.test,
          on terraform_plugin_test.tf line 93, in resource "azurerm_netapp_volume" "test":
          93: resource "azurerm_netapp_volume" "test" {

    testing_new.go:90: Error running post-test destroy, there may be dangling resources: exit status 1

        Error: deleting replicate Volume (Subscription: "<sub id>"
        Resource Group Name: "acctestRG-netapp-240215113651003253"
        Net App Account Name: "acctest-NetAppAccount-240215113651003253"
        Capacity Pool Name: "acctest-NetAppPool-240215113651003253"
        Volume Name: "acctest-NetAppVolume-240215113651003253"): unexpected status 409 with error: RequestConflict: Cannot modify resource with id '/subscriptions/<sub id>/resourceGroups/acctestRG-netapp-240215113651003253/providers/Microsoft.NetApp/netAppAccounts/acctest-NetAppAccount-240215113651003253/capacityPools/acctest-NetAppPool-240215113651003253/volumes/acctest-NetAppVolume-240215113651003253' because the resource entity provisioning state is not terminal. Please wait for the provisioning state to become terminal and then retry the request.

--- FAIL: TestAccNetAppVolume_updateExportPolicyRule (495.65s)
=== CONT  TestAccNetAppVolume_requiresImport
--- PASS: TestAccNetAppVolume_nfsv3FromSnapshot (2865.77s)
=== CONT  TestAccNetAppVolume_crossRegionReplication
=== NAME  TestAccNetAppVolume_update
    testcase.go:113: Step 3/4 error: Error running apply: exit status 1

        Error: updating Volume (Subscription: "<sub id>"
        Resource Group Name: "acctestRG-netapp-240215113651009721"
        Net App Account Name: "acctest-NetAppAccount-240215113651009721"
        Capacity Pool Name: "acctest-NetAppPool-240215113651009721"
        Volume Name: "acctest-NetAppVolume-240215113651009721"): polling after Update: `result.Status` was nil/empty - `op.Status` was "Patching" / `op.Properties.ProvisioningState` was ""

          with azurerm_netapp_volume.test,
          on terraform_plugin_test.tf line 105, in resource "azurerm_netapp_volume" "test":
         105: resource "azurerm_netapp_volume" "test" {

    testing_new.go:90: Error running post-test destroy, there may be dangling resources: exit status 1

        Error: deleting replicate Volume (Subscription: "<sub id>"
        Resource Group Name: "acctestRG-netapp-240215113651009721"
        Net App Account Name: "acctest-NetAppAccount-240215113651009721"
        Capacity Pool Name: "acctest-NetAppPool-240215113651009721"
        Volume Name: "acctest-NetAppVolume-240215113651009721"): unexpected status 409 with error: RequestConflict: Cannot modify resource with id '/subscriptions/<sub id>/resourceGroups/acctestRG-netapp-240215113651009721/providers/Microsoft.NetApp/netAppAccounts/acctest-NetAppAccount-240215113651009721/capacityPools/acctest-NetAppPool-240215113651009721/volumes/acctest-NetAppVolume-240215113651009721' because the resource entity provisioning state is not terminal. Please wait for the provisioning state to become terminal and then retry the request.

--- FAIL: TestAccNetAppVolume_update (1266.22s)
--- PASS: TestAccNetAppVolume_updateSubnet (3849.00s)
--- PASS: TestAccNetAppVolume_complete (2269.47s)
--- PASS: TestAccNetAppVolume_requiresImport (2048.56s)
=== NAME  TestAccNetAppVolume_crossRegionReplication
    testcase.go:113: Step 1/2 error: After applying this test step, the plan was not empty.
        stdout:


        Terraform used the selected providers to generate the following execution
        plan. Resource actions are indicated with the following symbols:
          ~ update in-place

        Terraform will perform the following actions:

          # azurerm_netapp_volume.test_secondary will be updated in-place
          ~ resource "azurerm_netapp_volume" "test_secondary" {
                id                                   = "/subscriptions/<sub id>/resourceGroups/acctestRG-netapp-240215113651001833/providers/Microsoft.NetApp/netAppAccounts/acctest-NetAppAccount-secondary-240215113651001833/capacityPools/acctest-NetAppPool-secondary-240215113651001833/volumes/acctest-NetAppVolume-secondary-240215113651001833"
                name                                 = "acctest-NetAppVolume-secondary-240215113651001833"
              ~ snapshot_directory_visible           = true -> false
                tags                                 = {
                    "CreatedOnDate"    = "2022-07-08T23:50:21Z"
                    "SkipASMAzSecPack" = "true"
                }
                # (17 unchanged attributes hidden)

                # (2 unchanged blocks hidden)
            }

        Plan: 0 to add, 1 to change, 0 to destroy.
--- FAIL: TestAccNetAppVolume_crossRegionReplication (2582.24s)
FAIL
FAIL    github.com/hashicorp/terraform-provider-azurerm/internal/services/netapp        7482.485s
FAIL
make: *** [GNUmakefile:99: acctests] Error 1
user [ ~/go/src/github.com/terraform-providers/terraform-provider-azurerm ]$

[paulomarquesc]

Thanks for the latest changes.

[katbyte]

Thanks or the pr @Orexii - having a hard time running these tests due to quotas:

 Net App Account Name: "acctest-NetAppAccount-240220011629641600"
        Capacity Pool Name: "acctest-NetAppPool-240220011629641600"): performing PoolsCreateOrUpdate: unexpected status 409 with error: SizeQuotaExceeded: Creating a new capacityPool of size 8 TiB would exceed the regional quota of 25 TiB. Current usage is 24 TiB. To learn more about Azure NetApp Files capacity constraints, see Resource limits for Azure NetApp Files https://docs.microsoft.com/azure/azure-netapp-files/azure-netapp-files-resource-limits

is it possible to provision resources with a smaller quota?

[Orexii]

Thanks or the pr @Orexii - having a hard time running these tests due to quotas:

 Net App Account Name: "acctest-NetAppAccount-240220011629641600"
        Capacity Pool Name: "acctest-NetAppPool-240220011629641600"): performing PoolsCreateOrUpdate: unexpected status 409 with error: SizeQuotaExceeded: Creating a new capacityPool of size 8 TiB would exceed the regional quota of 25 TiB. Current usage is 24 TiB. To learn more about Azure NetApp Files capacity constraints, see Resource limits for Azure NetApp Files https://docs.microsoft.com/azure/azure-netapp-files/azure-netapp-files-resource-limits

is it possible to provision resources with a smaller quota?

@katbyte the error message seems to indicate that you already have capacity pool resources consuming quota for that subscription in the region the test is running. This may be due to resources you have already created (or someone else with access to the subscription), or that you are running tests in parallel and therefore consuming more than the quota for that region. Unfortunately changing the test to use a smaller capacity pool is not possible, however you have two options:

1. Request more quota for that region in your subscription. See the link provided.
2. Remove already existing capacity pools in the subscription and run tests sequentially and not in parallel, e.g.:
   make acctests SERVICE='netapp' TESTARGS=' -parallel 1 -run=TestAccNetAppVolume_' TESTTIMEOUT='1200m'

[katbyte]

@Orexii - i tried reaching out to increase quota and am waiting for the capacity team to get back to us

[katbyte]

Have some test failures, also could you please add these properties to a test?

------- Stdout: -------
=== RUN   TestAccNetAppVolumeGroupSapHanaDataSource_basic
=== PAUSE TestAccNetAppVolumeGroupSapHanaDataSource_basic
=== CONT  TestAccNetAppVolumeGroupSapHanaDataSource_basic
    testing_new.go:90: Error running post-test destroy, there may be dangling resources: exit status 1
        
        Error: deleting `volume`: deleting Volume (Subscription: "*******"
        Resource Group Name: "acctestRG-netapp-240221201535290485"
        Net App Account Name: "acctest-NetAppAccount-240221201535290485"
        Capacity Pool Name: "acctest-NetAppPool-240221201535290485"
        Volume Name: "acctest-NetAppVolume-5-240221201535290485"): performing Delete: unexpected status 409 with error: RequestConflict: Cannot modify resource with id '/subscriptions/*******/resourceGroups/acctestRG-netapp-240221201535290485/providers/Microsoft.NetApp/netAppAccounts/acctest-NetAppAccount-240221201535290485/capacityPools/acctest-NetAppPool-240221201535290485/volumes/acctest-NetAppVolume-5-240221201535290485' because the resource entity provisioning state is not terminal. Please wait for the provisioning state to become terminal and then retry the request.
        
        deleting `volume`: deleting Volume (Subscription:
        "*******"
        Resource Group Name: "acctestRG-netapp-240221201535290485"
        Net App Account Name: "acctest-NetAppAccount-240221201535290485"
        Capacity Pool Name: "acctest-NetAppPool-240221201535290485"
        Volume Name: "acctest-NetAppVolume-5-240221201535290485"): performing Delete:
        unexpected status 409 with error: RequestConflict: Cannot modify resource
        with id
        '/subscriptions/*******/resourceGroups/acctestRG-netapp-240221201535290485/providers/Microsoft.NetApp/netAppAccounts/acctest-NetAppAccount-240221201535290485/capacityPools/acctest-NetAppPool-240221201535290485/volumes/acctest-NetAppVolume-5-240221201535290485'
        because the resource entity provisioning state is not terminal. Please wait
        for the provisioning state to become terminal and then retry the request.
--- FAIL: TestAccNetAppVolumeGroupSapHanaDataSource_basic (1073.12s)
FAIL

[Orexii]

Have some test failures, also could you please add these properties to a test?

------- Stdout: -------
=== RUN   TestAccNetAppVolumeGroupSapHanaDataSource_basic
=== PAUSE TestAccNetAppVolumeGroupSapHanaDataSource_basic
=== CONT  TestAccNetAppVolumeGroupSapHanaDataSource_basic
    testing_new.go:90: Error running post-test destroy, there may be dangling resources: exit status 1
        
        Error: deleting `volume`: deleting Volume (Subscription: "*******"
        Resource Group Name: "acctestRG-netapp-240221201535290485"
        Net App Account Name: "acctest-NetAppAccount-240221201535290485"
        Capacity Pool Name: "acctest-NetAppPool-240221201535290485"
        Volume Name: "acctest-NetAppVolume-5-240221201535290485"): performing Delete: unexpected status 409 with error: RequestConflict: Cannot modify resource with id '/subscriptions/*******/resourceGroups/acctestRG-netapp-240221201535290485/providers/Microsoft.NetApp/netAppAccounts/acctest-NetAppAccount-240221201535290485/capacityPools/acctest-NetAppPool-240221201535290485/volumes/acctest-NetAppVolume-5-240221201535290485' because the resource entity provisioning state is not terminal. Please wait for the provisioning state to become terminal and then retry the request.
        
        deleting `volume`: deleting Volume (Subscription:
        "*******"
        Resource Group Name: "acctestRG-netapp-240221201535290485"
        Net App Account Name: "acctest-NetAppAccount-240221201535290485"
        Capacity Pool Name: "acctest-NetAppPool-240221201535290485"
        Volume Name: "acctest-NetAppVolume-5-240221201535290485"): performing Delete:
        unexpected status 409 with error: RequestConflict: Cannot modify resource
        with id
        '/subscriptions/*******/resourceGroups/acctestRG-netapp-240221201535290485/providers/Microsoft.NetApp/netAppAccounts/acctest-NetAppAccount-240221201535290485/capacityPools/acctest-NetAppPool-240221201535290485/volumes/acctest-NetAppVolume-5-240221201535290485'
        because the resource entity provisioning state is not terminal. Please wait
        for the provisioning state to become terminal and then retry the request.
--- FAIL: TestAccNetAppVolumeGroupSapHanaDataSource_basic (1073.12s)
FAIL

@katbyte could you please verify once more that the TestAccNetAppVolumeGroupSapHanaDataSource_basic test does indeed fail on your end? Could you possibly also verify with a colleague that failures are not local to solely your dev environment? The test does not fail on my end, and I've ran it several times:

malmorga@ZEPHYRUS-M16:~/go/src/github.com/terraform-providers/terraform-provider-azurerm$ git branch
  main
* malmorga/add_smbNonBrowsable_and_smbAccessBasedEnumeration
malmorga@ZEPHYRUS-M16:~/go/src/github.com/terraform-providers/terraform-provider-azurerm$ go version
go version go1.21.6 linux/amd64
malmorga@ZEPHYRUS-M16:~/go/src/github.com/terraform-providers/terraform-provider-azurerm$ terraform version
Terraform v1.7.4
on linux_amd64
malmorga@ZEPHYRUS-M16:~/go/src/github.com/terraform-providers/terraform-provider-azurerm$ make build
==> Checking that code complies with gofmt requirements...
==> Checking that Custom Timeouts are used...
==> Checking that acceptance test packages are used...
go generate ./internal/services/...
go generate ./internal/provider/
go install
malmorga@ZEPHYRUS-M16:~/go/src/github.com/terraform-providers/terraform-provider-azurerm$ go build
malmorga@ZEPHYRUS-M16:~/go/src/github.com/terraform-providers/terraform-provider-azurerm$ make acctests SERVICE='netapp' TESTARGS=' -parallel 5 -run=TestAccNetAppVolumeGroupSapHanaDataSource_basic' TESTTIMEOUT='1200m'
==> Checking that code complies with gofmt requirements...
==> Checking that Custom Timeouts are used...
==> Checking that acceptance test packages are used...
TF_ACC=1 go test -v ./internal/services/netapp -parallel 5 -run=TestAccNetAppVolumeGroupSapHanaDataSource_basic -timeout 1200m -ldflags="-X=github.com/hashicorp/terraform-provider-azurerm/version.ProviderVersion=acc"
=== RUN   TestAccNetAppVolumeGroupSapHanaDataSource_basic
=== PAUSE TestAccNetAppVolumeGroupSapHanaDataSource_basic
=== CONT  TestAccNetAppVolumeGroupSapHanaDataSource_basic
--- PASS: TestAccNetAppVolumeGroupSapHanaDataSource_basic (1637.41s)
PASS
ok      github.com/hashicorp/terraform-provider-azurerm/internal/services/netapp        1637.420s
malmorga@ZEPHYRUS-M16:~/go/src/github.com/terraform-providers/terraform-provider-azurerm$

[Orexii]

@katbyte Historically there are no SMB-based volume tests in this repo due to requirement of a domain controller needing to be deployed and configured in addition to other complexities required to test SMB-based volume features. It is unfeasible at the moment to mitigate this historical lack of SMB-based volume testing in this repo within this PR. An internal ticket can be made to address this repo's SMB-based volume's testing deficiency at a later time

[katbyte]

@Orexii - this is running in our test subscription which runs nightly tests, so it has to work in this environment.

[Orexii]

@Orexii - this is running in our test subscription which runs nightly tests, so it has to work in this environment.

@katbyte can you please verify that the same test, TestAccNetAppVolumeGroupSapHanaDataSource_basic, succeeds in your main branch, but fails in the malmorga/add_smbNonBrowsable_and_smbAccessBasedEnumeration branch? Your output from the failing test does not seem to clearly indicate that changes in this PR are the cause of the test's failures, especially considering that the failures are not occurring outside of your test environment. The changes in this PR should not affect volume groups at all, which leads me to believe that the failures you're experiencing may be transient or already exists if you were to run the same test in your main branch within the test environment.

If I had access to your test sub/environment, this is what I would do:

1. Run the test TestAccNetAppVolumeGroupSapHanaDataSource_basic again on the malmorga/add_smbNonBrowsable_and_smbAccessBasedEnumeration branch to ensure the failure is not transient.
2. If the failure persists, run the same test TestAccNetAppVolumeGroupSapHanaDataSource_basic on the main branch in the exact same environment to ensure that the failures are as a result of changes in the malmorga/add_smbNonBrowsable_and_smbAccessBasedEnumeration branch.

If you could please attempt the above steps to ensure that the error you're receiving is pertinent to this PR, I would greatly appreciate it.

Also, if you could please respond to my earlier comment regarding this repo's lack of SMB-based volume testing, that would be spectacular as well. Thanks

[katbyte]

LGTM 🎃

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.