[CosmosDB] Unable to set any MongoDB preview feature/capabilities

[katbyte]

Bug Report

• import path of package in question, e.g. .../services/compute/mgmt/2018-06-01/compute
github.com/Azure/azure-sdk-for-go/services/cosmos-db/mgmt/2015-04-08/documentdb

• SDK version e.g. master, latest, 18.1.0
20.1.0

• output of go version
go version go1.10.3 darwin/amd64

• What happened?
  When attempting to set one of: EnableAggregationPipeline MongoDBv3.4 mongoEnableDocLevelTTL via the SDK nothing seems to happen, there no activity and no matter how long I wait they never get set.

• What did you expect or want to happen?
  The capabilities being set on the DB.

• How can we reproduce it?
  run the acceptance tests in either of these PRs: MongoDBv3.4 & mongoEnableDocLevelTTL, EnableAggregationPipeline

• Anything we should know about your environment.

[vladbarosan]

@katbyte can you try setting the capability through a Patch after creating the account. @arsalana-microsoft, @dmakwana can you help us with what is the proper way of enabling this feature?

Trying it from the CLI it doesnt seem to be possible to enable this feature when creating the account

[katbyte]

@vladbarosan,

I have tried updating it after creating the account, no luck there either

[katbyte]

@marstr, would you be able to follow up on this?

[marstr]

Absolutely.

In general it looks like you have a lot of threads open that are tagged as being server-side issues. Let me work with @mozehgir to make sure that these are actually getting seen by the relevant teams.

For this one specifically, I'll try to build out a stand-alone min-repro that we can distribute.

[tombuildsstuff]

@marstr 👋 did you hear back from the Service Team on this issue?

[katbyte]

👋 @jhendrixMSFT, any word on this issue?

[jhendrixMSFT]

Sorry for the delay, I'm taking a look.

[jhendrixMSFT]

I was able to get this working by subsequently calling the Patch operation here. Here's sample code.

future, _ := client.Patch(context.Background(), "rg", "acc", documentdb.DatabaseAccountPatchParameters{
	DatabaseAccountPatchProperties: &documentdb.DatabaseAccountPatchProperties{
		Capabilities: &[]documentdb.Capability{
			documentdb.Capability{
				Name: to.StringPtr("EnableAggregationPipeline"),
			},
		},
	},
})

future.WaitForCompletionRef(context.Background(), client.Client)

I will follow up with the service team about why it doesn't work during creation. Would you be able to use this workaround in the meantime?

[katbyte]

That worked! Thank you very much @jhendrixMSFT

[shurd]

Hi @katbyte are you still seeing this issue? I can take a look to determine if it is a client problem if it persists. Adding capabilities when provisioning an account works on other SDKs, so the server correctly handles capabilities.

[katbyte]

hi @shurd

Checked our tests and while patching it doesn't fail, it does seem like it doesn't stick and subsequent reads fail:

------- Stdout: -------
=== RUN   TestAccAzureRMCosmosDBAccount_capabilityUpdate
=== PAUSE TestAccAzureRMCosmosDBAccount_capabilityUpdate
=== CONT  TestAccAzureRMCosmosDBAccount_capabilityUpdate
--- FAIL: TestAccAzureRMCosmosDBAccount_capabilityUpdate (1426.25s)
    testing.go:568: Step 1 error: After applying this step and refreshing, the plan was not empty:
        
        DIFF:
        
        UPDATE: azurerm_cosmosdb_account.test
          capabilities.#:                               "2" => "1"
          capabilities.0.name:                          "MongoDBv3.4" => "MongoDBv3.4"
          capabilities.1.name:                          "mongoEnableDocLevelTTL" => ""
          connection_strings.#:                         "4" => "4"

[shurd]

@katbyte I just validated that I could set those capabilities through our API, which the Go SDK uses, so they should both be valid. Could you provide a test account name that saw this failure? I can look at our logs to see if there is any data on what is happening.

You should also be able to set capabilities at create-time, so if you are still interested in setting capabilities when you create an account, I can check and see if the delay is something in our service (versus the SDK not polling correctly or otherwise). If you repro the long-running create and give me an account name, I can check those logs as well.

[shurd]

I just tried to replicate the original error, and the account create completed with all 3 capabilities in 8 minutes. Subsequent GETs on the account show that they are still on the account. I copied the Azure sample for Cosmos DB and made slight modifications to make it have those capabilities.

Creating Client

func getDatabaseAccountClient() documentdb.DatabaseAccountsClient {
    dbAccountClient := documentdb.NewDatabaseAccountsClient(SubscriptionID())
    auth, _ := GetResourceManagementAuthorizer()
    dbAccountClient.Authorizer = auth
    dbAccountClient.AddToUserAgent(UserAgent())
    return dbAccountClient
}

Provisioning Account

func CreateDatabaseAccount(ctx context.Context, accountName string) (dba documentdb.DatabaseAccount, err error) {
    dbAccountClient := getDatabaseAccountClient()
    future, err := dbAccountClient.CreateOrUpdate(
        ctx,
        BaseGroupName(),
        accountName,
        documentdb.DatabaseAccountCreateUpdateParameters{
            Location: to.StringPtr(DefaultLocation()),
            Kind:     documentdb.MongoDB,
            DatabaseAccountCreateUpdateProperties: &documentdb.DatabaseAccountCreateUpdateProperties{
                DatabaseAccountOfferType: to.StringPtr("Standard"),
                Locations: &[]documentdb.Location{
                    {
                        FailoverPriority: to.Int32Ptr(0),
                        LocationName:     to.StringPtr(DefaultLocation()),
                    },
                },
                Capabilities: &[]documentdb.Capability{
                    {
                        Name: to.StringPtr("EnableAggregationPipeline"),
                    },
                    {
                        Name: to.StringPtr("MongoDBv3.4"),
                    },
                    {
                        Name: to.StringPtr("mongoEnableDocLevelTTL"),
                    },
                },
            },
        })
    if err != nil {
        return dba, fmt.Errorf("cannot create database account: %v", err)
    }

    err = future.WaitForCompletionRef(ctx, dbAccountClient.Client)
    if err != nil {
        return dba, fmt.Errorf("cannot get the database account create or update future response: %v", err)
    }

    return future.Result(dbAccountClient)
}

[shurd]

I am going to close this issue now since there is a working example. If you have a subscription Id and account name that you saw the issue with, you can reopen it with those details and I can take a look at the logs.