Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic resource name generation function #12651

Closed
WhitWaldo opened this issue Dec 7, 2023 · 11 comments
Closed

Automatic resource name generation function #12651

WhitWaldo opened this issue Dec 7, 2023 · 11 comments
Assignees
Labels
enhancement New feature or request Needs: Triage 🔍

Comments

@WhitWaldo
Copy link

WhitWaldo commented Dec 7, 2023

Is your feature request related to a problem? Please describe.
In a recent release, linting rules were added that look for resource name constraint violations. This doesn't yet appear to support constant folding, so it's been of limited use for my own purposes.

But there are a lot of different naming constraints for resources and there's no single standard across them all. Even with prefix values, one can frequently run into validation issues that crop up only at deployment (though hopefully less frequently with the linting rules).

Describe the solution you'd like
I'd like to request a Bicep function, but not necessarily one that just calls an ARM function, but one that produces an interpolated string with existing functions as necessary at build time. I'd like to request a function that can generate resource names with a few optional parameters:

  • Prefix
  • Suffix
  • Seed (enabling idempotent names)

If I pass illegal characters into any of these parameters (e.g. a prefix includes disallowed uppercase characters or exceeds the maximum string length), I'd like for the produced name to honor them as best as possible (force casing, take a substring of the provided value), but conform to the rules and always produce a valid output - I would consider it out of scope to ensure that the output isn't already taken as that might be better handled with a try/retry functionality.

Thank you for the consideration!

@anthony-c-martin
Copy link
Member

anthony-c-martin commented Dec 8, 2023

@WhitWaldo - is this something that could be accomplished by publishing a registry module with the experimental imports and user-defined functions?

E.g. in the registry module:

@export()
func generateResourceName(prefix string, suffix string, seed string) string => '${prefix}-${suffix}-${seed}' // your implementation here

Using the function:

import { generateResourceName } from 'br:...' // your registry reference here

resource foo '...' = {
  name: generateResourceName('foo', 'bar', 'mySeed')
}

Given that everyone has different naming conventions, I think it would be difficult to justify adding this as a built-in function, but I believe the above should allow you to roll your own.

@jeskew
Copy link
Contributor

jeskew commented Dec 11, 2023

This doesn't yet appear to support constant folding, so it's been of limited use for my own purposes.

There should be fairly robust constant folding within the type system, though some values can't be known until deploy time (like param values or the return type of functions like resourceGroup()). Is there something you've run into that you think bicep should be able to calculate at compile time but isn't?

@WhitWaldo
Copy link
Author

@anthony-c-martin
I'll explore that as an option. The crux of what I was looking for though is something that's able to take advantage of the infrastructure apparently now in Bicep to know about the naming constraints on each of the resources. Perhaps instead of name generation there could be a per-resource validity test?

Resource name errors are fatal for deployments and are only validated when the resource is actually being deployed, so something failing towards the tail end of a 2-hour deployment wouldn't be caught until this has run for a while. Having a check just at build time (e.g. not necessarily translated into an ARM function, but just a hint during the build) could be a real time-saver here.

@jeskew
I was basing the constant folding status on the linked issue (#444) and its open status and the fact that the linter rules seem to be incessantly calling out my resource names as too short or too long despite a concatenation of (mostly) constants pulled in via parameters. I haven't seen an issue at build time, but at least a dev time, it hasn't seemed like it's pre-calculating the value length as it reasonably could locally.

@jeskew
Copy link
Contributor

jeskew commented Dec 12, 2023

I was basing the constant folding status on the linked issue (#444) and its open status and the fact that the linter rules seem to be incessantly calling out my resource names as too short or too long despite a concatenation of (mostly) constants pulled in via parameters.

#444 is really about how Bicep emits compiled JSON. Today, a Bicep expression like 2 + 2 will be compiled to add(2, 2) even though the compiler could emit 4 instead. This work is largely already implemented in the type analysis engine as of #10102, though: the type of the expressions 2 + 2 and 4 are identical, and similar analysis is done for string operations. For example, in:

@minLength(1)
param foo string

@minLength(2)
param bar string

var baz = '${foo}${bar}'
var quux = uniqueString(baz)
var pop = first(quux)

the type engine will know that baz will always be at least three characters, quux will be exactly 13 characters, and pop will be exactly one character. If the arguments to a function can be precisely calculated at compile time, the return type of the function should be as well, and if constraints are known about the arguments passed in, the compiler should be using constraints on the inputs to determine what constraints can be known about the return value. There are some cases where this analysis falls short -- e.g., replace can be applied deterministically to literal values but drops refinements on non-literal values as discussed in #12271 -- but if there are particular expressions that you think should be analyzable at compile time/when you're authoring a template, please raise an issue and maybe it can be addressed.

@stephaniezyen stephaniezyen added Needs: Author Feedback Awaiting feedback from the author of the issue and removed Needs: Triage 🔍 labels Dec 13, 2023
@alex-frankel
Copy link
Collaborator

take advantage of the infrastructure apparently now in Bicep to know about the naming constraints on each of the resources

@WhitWaldo - can you clarify this statement? There is currently an open issue (#1679) tracking this ask, but other than that I'm not aware of any work to address this.

@jeskew
Copy link
Contributor

jeskew commented Dec 18, 2023

@alex-frankel That's my fault for not closing the issue out. We shipped "refinement types" (compile-time min/maxLength & min/maxValue checking) in #9870 and started deriving resource name refinements from RP's swagger models as of Bicep 0.21.

@WhitWaldo
Copy link
Author

The work in #9870 is what I was thinking of as well - reading it, it's possible that there is still insufficient information to do a more comprehensive validation (e.g. if it's only got length information available so far, it's going to make it tough to do a more comprehensive regex validation.

And perhaps this is also just eventually a next-stage of the work in 9870 in that when that information can be more autonomously gleaned, the rules can get more rigorous (though there isn't an regex validation function I'm aware of - closest ask I see is yours from #8409), but ideally it means that passing a dash or capital letter through a constant, variable and/or parameter whether directly or via string concatenation gets caught at build time when used to name a storage resource (as only lowercase letters and numbers are allowed) instead of just a length check.

@jeskew Here's my own use case for where it's not clear sufficient folding is happening (and why I perpetually see name length warnings now):

I use a process inspired by another active participant in this group whose name is eluding me at the moment to build resource names in a consistent way. I've got a two-character value identifying the resource itself (shortening each just as a demo here):

var ResourceMapping = {
  keyVault: 'kv'
  cosmosDb: 'cs'
  frontDoor: 'fd'
}

I've got a four-character name for each region:

```bicep
var regionPrefixes = {
  centralus: 'ACU1'
  eastus2: 'AEU2'
}

I build a variable that I use for a prefix going forward with two other strings passed in through parameters:

var DeploymentPrefix = '${RegionPrefixes[LocationLookupValue]}-${Deployment.deployment.orgName}-${Deployment.deployment.tenantAppName}' //EX: AEU2-IVN-COR

Then I pass the DeploymentPrefix, ResourceMapping and a constant suffix for the resource name into each module (the modules then build the actual resource name out so I can put all the constraints in there instead of this outer module):

module EventHub './resources/eventHub.bicep' = {
  name: '${ResourceMapping.deployment}-${ResourceMapping.eventHub}'
  params: {
    Location: Location
    DeploymentPrefix: DeploymentPrefix
    ResourceMapping: ResourceMapping
    EventHubNamespace: 'myeventhubs'
    EventHubName: 'inbound'
    UaiDetails: {
      name: UAI.outputs.Name
      resourceGroupName: UAI.outputs.ResourceGroupName
      subscriptionId: UAI.outputs.SubscriptionId
    }
  }
  dependsOn: [
    WaitForUai
  ]
}

In said module, I seek to spin up an Event Hub namespace:

resource EHNamespace 'Microsoft.EventHub/namespaces@2023-01-01-preview' = {
  name: '${DeploymentPrefix}-${replace(toLower(EventHubNamespace), '-', '')}-${ResourceMapping.eventHub}'
  location: Location
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${UAI.id}': {}
    }
  }
  sku: {
    name: 'Standard'
    tier: 'Standard'
    capacity: 1
  }
  properties: {
    minimumTlsVersion: '1.2'
    publicNetworkAccess: 'Enabled'
    disableLocalAuth: false
    zoneRedundant: true
    isAutoInflateEnabled: false
    maximumThroughputUnits: 2
    kafkaEnabled: false
  }
  dependsOn: [
    UAI
  ]
}

But it's on this name I have a yellow squiggly:
| The provided value can have a length as small as 2 and may be too short to assign to a target with a configured minimum length of 6. bicep(BCP334)

This is my concern about the constant folding then - I've definitely got at least 12 characters through the DeploymentPrefix value alone before the other values (10 after the dashes are removed), but it's not picking up on that or the remaining dash and additional two characters from the resource mapping when evaluating the validation - all entailed values are known at build.

@microsoft-github-policy-service microsoft-github-policy-service bot added Needs: Triage 🔍 and removed Needs: Author Feedback Awaiting feedback from the author of the issue labels Dec 19, 2023
@alex-frankel
Copy link
Collaborator

@WhitWaldo - I'm having a little bit of trouble following this latest response. Should this issue be broken down into smaller pieces? So far I think I see three features asks:

  • regex validation
  • more naming restriction information in the generated bicep types (i.e. allowed characters in a storage account name)
  • better constant folding

Is that a correct summary or am I missing something?

@alex-frankel alex-frankel added Needs: Author Feedback Awaiting feedback from the author of the issue and removed Needs: Triage 🔍 labels Jan 17, 2024
@alex-frankel alex-frankel removed their assignment Jan 17, 2024
@WhitWaldo
Copy link
Author

Most of the latest response was meant to explain to @jeskew why I didn't think constant folding was yet implemented.

Put simply, my greatest concern and purpose of the issue is that an already-taken resource name is cause for a deployment failure and there's no real pre-checking of name validity ahead of a full deployment. This isn't so much about what to do when a name is already taken so much as ensuring that the name used at least matches the resource requirements (e.g. which symbols are or aren't allowed, must everything be lowercase, are numbers permitted, what's the maximum length of the name, etc.).

Anthony's approach is a fair start, but at the end of the day, it doesn't do any per-resource name validation checking so I could very well produce a 25 character storage account name or a Key Vault secret with underscores, or a Kusto cluster with a three-character name and only find out after the deployment failed.

So the ask here is probably more of a strengthening of the constant folding to evaluate (as much as possible) what a given string will be at compile time based on what's known then + a more robust version of what came with #9870 (to add valid characters on a per-resource basis to the check).

Again, I'd love to see a function I could just provide the prefix, suffix and idempotency key and get a guaranteed valid name, but in lieu of that, I'd love to see compile-time validation warning me of a name that might be tentatively invalid at deployment time and why.

@alex-frankel
Copy link
Collaborator

Thanks for the response @WhitWaldo. For better tracking, I am going to close this one and recommend we do the following:

In theory, once all of those are done, you should be able to right a more complete name generation function.

@github-project-automation github-project-automation bot moved this from Todo to Done in Bicep Mar 13, 2024
@WhitWaldo
Copy link
Author

@alex-frankel I just wanted to loop back here and share my thanks for the improvement done since I opened this ticket. While putting together a collection of values for a storage account name, the local Bicep checker was properly able to count the number of characters of the various locally-provided objects to tell me that my proposed name would exceed the maximum length by 2:
StorageAccountName: 'prrd${ResourceMapping.batch}${substring(uniqueString(DeploymentPrefix, 'prrd'), 0, 6)}${Deployment.deployment.isDevTest ? 'dev' : 'prd'}'

...where ResourceMapping.batch was a 2-character value, it properly worked out that this would be 4 + 2 + 6 + 3 = 15 characters and warned me that the length (which would always exceed 13 characters) was greater than the allowed 11 characters.

This is exactly the sort of dev-time experience I've been hoping for, so I just wanted to circle back with a big thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Needs: Triage 🔍
Projects
Archived in project
Development

No branches or pull requests

5 participants