New Tool: update-go-azure-sdk
#23498
Conversation
This tool is intended to update the `hashicorp/go-azure-sdk` dependency within the Provider - and then update the codebase to take advantage of any new API Versions that are present within this version of `hashicorp/go-azure-sdk`, if used in the Provider and the API versions are compatible (e.g. the tests pass when updating the API Version). This tool makes a considerable number of assumptions about the working directory and environment this is being run in and is primarily intended to be run in automation - and whilst will not be automatically configured initially, in time should mean that both SDK upgrades and new API Version upgrades are (mostly) automated. In the event that an SDK/API Version is _not_ compatible, the automated process will fail - and manual intervention is required - which is desired (since this should be the edge-case and/or requires additional thought).
There was a problem hiding this comment.
@tombuildsstuff This looks great! Should we also run make test after updating the module version, to catch any breaking changes outside of the services/ directory (e.g. in internal/clients or internal/provider)?
…-version` is now auto-detected
This allows for reusing an existing checkout, rather than cloning a fresh copy of `hashicorp/go-azure-sdk` useful both for automation, and when working with an offline copy
I considered that, honestly if that's happened we have larger problems - either intentional or not - I suspect that we'll make those types of changes infrequently enough that it's not worth the cost of the extra CPU cycles to check that - since the tests would be run when the PR's sent to this repository anyway (where we'd catch it)? |
I was mainly thinking of scenarios where an intentional breaking change is introduced, for example in |
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
This PR introduces a new tool
update-go-azure-sdkwhich is primarily intended to be run in automation.This tool obtains the changes between two versions of
hashicorp/go-azure-sdk, then updates the version ofhashicorp/go-azure-sdkused in this repository - and when a new API Version is available in the latest SDK, attempts to update the API Version for each Service.Both the SDK Upgrade and each API Version update are done in individual commits - and prior to both being committed the unit tests are run to ensure these pass. This means that when an API Version is compatible, that is contains no breaking changes to the API Payloads - the API Version upgrade can be done automatically. However in the event that the API Payload is not compatible - the API Version will be ignored, with the issues outlined in the PR description. Examples of both of these can be seen below.
This will be conditionally run by
hashicorp/go-azure-sdkafter a release is made, to both update the version ofhashicorp/go-azure-sdkthat's being used in the Provider - but also to update the API Version being used in the Provider when a new API Version becomes available (and the build passes).There's a bunch of assumptions built into this tool and it's not going to be infallible - however this should both ensure that
hashicorp/go-azure-sdkis up-to-date in this repository and remove the manual work needed to update from one API Version to another.Examples
Console output:
Output in a PR:
An earlier build log containing example failures (with slightly less detail) can be seen here: https://gist.github.com/tombuildsstuff/d6c153d484697961b6519a3c1cec4882#file-output-txt-L156-L337
Questions
Why not extend the existing tool
update-api-version?In short, there's value in both - updating the API Version being used, and updating
go-azure-sdkare subtly different use-cases - with the former being primarily run by a human and the latter primarily being run in automation.It's worth calling out that some of the changes made in this PR should also be made to that tool, but that'll follow as a separate PR.
Why not just use Dependabot?
Dependabot would work great for updating the SDK dependency - however this wouldn't allow us to update the API Versions being used in the Provider. In addition, using a custom tool allows us to provide additional logic, such as running the (unit) tests for each API Version update to ensure these pass prior to committing the changes - else rolling this back (and documenting it in the PR description).
Whilst there's value in running this on every release of
hashicorp/go-azure-sdk- for the moment we need this tool to be conditionally run, as when the base layer is migrated fromAzure/go-autoresttohashicorp/go-azure-sdkadditional changes must be made by hand.Why not use a bash script?
In short, it's far simpler to get the output we want from a tool.