Generalize naming of common pipeline-related artifacts

[mthalman]

The common infrastructure located in the eng/common folder had names of projects, repositories, variables, branches, and paths that were specific to the .NET implementation. In order to make the infrastructure suitable for general purpose, these names need to be generalized.

• Defined parameters for internal/public AzDO project names.
• Defined parameters for a reference to the internal/public pipeline repository resource. This allows the pipeline implementer to be able to declare their own repository resource that represents a versions repo and pass in its identifying name as a parameter to the build-test-publish-repo.yml template.
• Replace all usages of variables derived from secrets of the .NET key vault with generalized names (e.g. app-dotnetdockerbuild-client-secret => acr.servicePrincipalPassword).
• Defined a new officialRepoPrefix so that the pipeline logic can know whether a given publishRepoPrefix value represents an official repo location.
• Defined a new azdoVersionsRepoInfo.path and gitHubVersionsRepoInfo.path variable to provide a way to specify the directory path where image info files are located in those respective repos.
• Added parameters allowing pipeline implementers to inject custom init steps and custom publish variables in order to customize the logic in the build and publish stages. Specifically, this allows .NET to customize the publicSourceBranch and additionalPublishMcrDogsArgs variables and reference a .NET-specific variable group in the publish stage.
• Create a new build-test-publish-repo.yml in the eng/pipelines/stages folder (outside of common) to act as a wrapper around the same-named template in the common folder. This allows for a shared implementation of many of the common settings that are used amongst the .NET pipelines.

Related to dotnet/docker-tools#843

[dagood]

Related to #843

Meant to be a link to dotnet/docker-tools#843 (vs. dotnet/dotnet-docker#843)?

Is this PR in the right repo to change the common files? It does make sense to test the results in this repo though.

[mthalman]

Related to #843

Meant to be a link to dotnet/docker-tools#843 (vs. dotnet/dotnet-docker#843)?

Is this PR in the right repo to change the common files? It does make sense to test the results in this repo though.

Yes, thanks. I've fixed the link. I'm making these changes in the nightly branch here because I feel it gives better context to the changes to see what changes are necessary to the consuming pipelines which aren't in the docker-tools repo.

[dagood]

I haven't dug through the entire thing, but the bits being pulled out make sense to me. A few suggestions.

[dagood]

It seems kind of questionable to me to manually maintain this file in each .NET repo--would it make sense to add a common dotnet configuration template that repos can opt into? (Also for publicProjectName / internalProjectName.)

[mthalman]

One thing not being addressed in this PR is the usage of DotNet in the names for agent pools. Fixing that will require more changes than I wanted to mix in here.

[dagood]

I think this fixes dotnet/docker-tools#835 too. (Well, sort of: it opens up a workaround-seeming fix that looks good enough to me!) For an internal build that I want to prevent from pushing tags to my ACR (like a PR validation build), I can make the pipeline file leave internalProjectName as null, so this never evaluates to true. 👍

[dagood]

Looking at the file again, is this saying that internal builds can only publish image info to the AzDO/internal repo? Don't we want to publish it to the public dotnet/versions repo, as long as we aren't doing a truly private build? (We can't publish anywhere at all in public-project builds, right?)

[mthalman]

This template sets the variable for the purposes of reading the image info file. It's not relevant to the publishing of the image info file. For publish, it publishes to both the public and internal repos. It pushes the commit to both to avoid latency issues (see dotnet/docker-tools#642). For public project builds, it runs with the dry-run option, so nothing actually gets published.

[dagood]

Ah, right... and I guess having them different is just an error right now (https://github.com/dotnet/docker-tools/blob/2dddfe26ef6faca5f110fe1f6c179e47a51f3433/src/Microsoft.DotNet.ImageBuilder/src/Commands/PublishImageInfoCommand.cs#L35-L39) but this could be used in the future.