Allow `pulumictl get version` on bare repositories #72

iwahbe · 2023-09-13T01:43:23Z

Previously, pulumictl get version required a full clone of the repository it was in. This was necessary so we could traverse history to find the nearest parent of the current commit.

This PR enables us to perform the same algorithm on a repository with non-local commits, lazily fetching commits as needed.

This will allow us to avoid full clones of provider repositories in CI. For pulumi-aws, this will take ~20 minutes off of the time from new PR to release.

thomas11

This is great! Only nit, can we get a bit of -v logging to help diagnose issues at runtime?

pkg/gitversion/gitversion.go

t0yv0 · 2023-09-13T14:11:16Z

pkg/gitversion/gitversion.go

+					RefSpecs: []config.RefSpec{config.RefSpec(
+						fmt.Sprintf("%[1]s:%[1]s", p.String()),
+					)},
+					Tags:     git.TagFollowing,


Does this fetch commits only or it fetches tags from the remote also if these commits are tagged on the remote?

Tags: git.TagFollowing will fetch commits and any tag applied to them.

t0yv0 · 2023-09-13T14:20:07Z

pkg/gitversion/gitversion.go

+	if err != nil && !errors.Is(err, plumbing.ErrObjectNotFound) {
+		return false, nil, fmt.Errorf("could not find local object: %w", err)
+	}
+	// We have found a local tag that is a parent, so return it.


Curious is this behavior what pulumi build system needs? This would prefer a local tag. Suppose you forgot to do fetch tags. So origin has more recent tags than local. For example:

local: c1=v0.9 --> c2 --> c3=HEAD origin: c1=v0.9 --> c2 --> c3=v1.0

Current algo will decide we're at 0.9-pre.something

If you had this though:

local: c1 --> c2 --> c3=HEAD origin: c1=v0.9 --> c2 --> c3=v1.0

The system would decide we're at v1.0?

In practice, I think the only way for pulumictl to see a local and remote history as in your example (👇) is on a dev machine that has not fetched tags in a while.

local: c1 --> c2 --> c3=HEAD origin: c1=v0.9 --> c2 --> c3=v1.0

I can change when we execute a remote search to only trigger when we are missing history, which would prevent this corner case.

To step back why is it consulting both local and remote and tries to merge it? Can it just look in remote?

Hypothesis.

The operation you need in CI builds is this. Fetch me the bare --depth 1 commit X from remote to get the code, and compute the version. To compute the version you only need to look in remote/origin.

The operation you need locally is this. I have a full non-bare repo and I need to compute version based on local tags ONLY. If I introduce tags I want it to see them. It's my job as dev to sync tags with remotes and so forth.

Can these two use cases perhaps be separate commands instead of coding up merging remote and local?

I'll add a --remote={yes,no} flag that defaults to no. This will let us:

Enable this behavior separate from a merge to master.

Disable it by for local builds.

t0yv0 · 2023-09-13T14:25:15Z

Well I sympathize with the desire to shave off 3m from this:

https://github.com/pulumi/pulumi-aws/actions/runs/6166783348/job/16736792578

I however don't think this is a good idea to change pulumictl behavior in-place - at this point pulumictl is critical infrastructure and this can wreck things if there're mistakes rolled out. Are you confident in our testing strategy for pulumictl? I don't see tests added for new behavior, and specifying new behavior seems tricky.

We already have users saying things like "I don't understand how pulumictl decided the version".

t0yv0 · 2023-09-13T14:28:09Z

Let's consider a few alternatives, LMK what do you think:

put some more effort to explain and test this change and roll it out
reconsider your concrete use case and add a new pulumictl command or flag catering to the use case; migrate to it instead of updating in-place
we've had conversations (@guineveresaenger et al) about setting up testing of provider builders on a test provider could we do this first to get end-to-end confidence in versioning, i.e. that prereleases et all work and then we could be a lot more aggressive around making such changes?

t0yv0 · 2023-09-13T14:30:55Z

Also where is the 20 min figure coming from. An alternative to strongly consider also, is that we need to compute the version for the build only once and not recompute it in every build phase. The version can be redistributed to build phases as an artifact in GHA. Like in pulumi/pulumi. This is really important because absent that the builds are non-atomic on versions. If someone tags the repository while a build is in-flight, prerequisites phase will get one version while release phase will get another version.

If pulumictl is invoked only once then this shaves off 3 min not 20 right?

thomas11 · 2023-09-13T15:23:04Z

I however don't think this is a good idea to change pulumictl behavior in-place - at this point pulumictl is critical infrastructure and this can wreck things if there're mistakes rolled out. Are you confident in our testing strategy for pulumictl? I don't see tests added for new behavior, and specifying new behavior seems tricky.

I think the real issue here is that our GH actions don't pin the pulumictl version. It should be possible to evolve a tool without breaking everything.

We already have users saying things like "I don't understand how pulumictl decided the version".

Yep, this is why I asked for some -v logging in another comment.

t0yv0 · 2023-09-13T15:40:48Z

It should be possible to evolve a tool without breaking everything.

That's true. I'm getting some cold feet that this change is not going to break but I can work harder on review. Happy to rescind objection if y'all have confidence here.

iwahbe · 2023-09-13T16:03:15Z

Also where is the 20 min figure coming from.

pulumictl is invokes in prerequisites, build_sdk and test. Removing the full clone shaves off about 2.3 minutes from each consecutive job, so 6.9 minutes per test workflow. We run 3 test workflows to add new code: run-acceptence-tests, master/main and release/prerelease 6.9*3=20.7~=20.

An alternative to strongly consider also, is that we need to compute the version for the build only once and not recompute it in every build phase. The version can be redistributed to build phases as an artifact in GHA. Like in pulumi/pulumi. This is really important because absent that the builds are non-atomic on versions. If someone tags the repository while a build is in-flight, prerequisites phase will get one version while release phase will get another version.

That's a good idea, but it will require a larger re-work of our CI and Makefiles, or adding an unexpected side channel to pulumictl. Ideally, we will have both.

If pulumictl is invoked only once then this shaves off 3 min not 20 right?

t0yv0

Actually can we consider some alternatives here..

I have mentally a small complexity budget for the algorithm that needs to compute versions, and this codebase doesn't fit in this budget. It seems way too complex to me.

As it turns out we have prior art on the @pulumi/providers team of computing versions in what I consider a simple and accessible manner:

https://github.com/pulumi/pulumi-azure-native/blob/5ca6b832860f4050e258fcd1dd96a5892d0f9608/.github/workflows/dev-version.yml#L19-#L19

https://github.com/pulumi/pulumi-azure-native/blob/5ca6b832860f4050e258fcd1dd96a5892d0f9608/.github/workflows/release.yml#L15-#L15

This does a lot right-ish:

In release mode uses the tag.. $GITHUB_REF_NAME as is
In dev mode computes an alpha version based on last release; last release polled from GH
Atomically computes the version to keep it constant across the phases of the build
It still uses pulumictl but only for triggering docs build?

I'm curious @danielrbradley it's not using pulumictl for version converters? What converts the versions to PyPI versions?

The only controversial thing above imo is computing the last release from GitHub metadata. This locks into linear flow so for example if main branch on AWS is on 5.x, I cannot open a v6 branch to publish alphas from and have dev builds on that branch realize they're v6. I wonder how azure-native does that.

I wonder if we considered just checking the baseline release version as .special-file in-repo. Then it's trivial to read and has no constraints on branching. We just need to make sure that releasing updates its contents.

danielrbradley · 2023-09-25T12:51:14Z

This looks like a possibile solution if it's a quick fix to address failures without having to restructure the workflows, but we'd need to weigh this with the added complexity as @t0yv0 mentioned.

The azure-native approach still uses pulumictl, but uses only the convert-version command which I introduced for this exact use-case. This is a pure function converting our "generic" versions to a language-specific version. This allows us to just pass the generic version number around as a variable, then convert it to the language specific form within the makefile when required.

t0yv0 · 2023-09-25T13:56:36Z

Yeah so folks I'm partially being somewhat reluctant to take the time to review this line-by-line to get confidence, it might be fine, but partially I'm just wearing the ops hat - every change, even a small change, has a non-zero risk. Given where our build system is today (open issue count) it's a bad time to take that risk on. This PR takes the risk on for the goal of speeding up builds, that's not a good tradeoff for me. If we push through with this we could reduce risk by:

taking the time to thoroughly review
end-to-end demonstration or perhaps rolling this version of pulumictl to one repo only to before scaling this out

All in all I prefer broadcasting the scheme from azure-native to bridged providers since it has much less surface for bugs to hide and also it's been in production use so that's something going for it.

iwahbe requested review from t0yv0 and a team September 13, 2023 01:43

iwahbe self-assigned this Sep 13, 2023

thomas11 approved these changes Sep 13, 2023

View reviewed changes

pkg/gitversion/gitversion.go Show resolved Hide resolved

t0yv0 reviewed Sep 13, 2023

View reviewed changes

t0yv0 requested review from t0yv0 and thomas11 September 13, 2023 14:25

Refactor cmd/pulumictl/get/version/cli.go

54aee7e

iwahbe force-pushed the iwahbe/tag-lookup-bare-repositories branch from 04dcf39 to 15afdca Compare September 13, 2023 16:47

iwahbe added 3 commits September 13, 2023 10:44

Fetch commits as necessary when in a bare repository

8a2c581

Add test as a make target, then consume it in CI

6797da5

Add --remote flag

e4fb14f

iwahbe force-pushed the iwahbe/tag-lookup-bare-repositories branch from b4a8dc5 to e4fb14f Compare September 13, 2023 17:44

iwahbe added 2 commits September 13, 2023 11:27

Refactor test error handling

1df389e

Addapt a test for remote repositories

ec0158a

t0yv0 requested changes Sep 21, 2023

View reviewed changes

iwahbe marked this pull request as draft September 25, 2023 15:30

iwahbe closed this Nov 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow `pulumictl get version` on bare repositories #72

Allow `pulumictl get version` on bare repositories #72

iwahbe commented Sep 13, 2023

thomas11 left a comment

t0yv0 Sep 13, 2023

iwahbe Sep 13, 2023

t0yv0 Sep 13, 2023 •

edited

Loading

iwahbe Sep 13, 2023

t0yv0 Sep 13, 2023

t0yv0 Sep 13, 2023

iwahbe Sep 13, 2023

t0yv0 commented Sep 13, 2023

t0yv0 commented Sep 13, 2023 •

edited

Loading

t0yv0 commented Sep 13, 2023

thomas11 commented Sep 13, 2023

t0yv0 commented Sep 13, 2023

iwahbe commented Sep 13, 2023

t0yv0 left a comment

danielrbradley commented Sep 25, 2023

t0yv0 commented Sep 25, 2023

Allow pulumictl get version on bare repositories #72

Allow pulumictl get version on bare repositories #72

Conversation

iwahbe commented Sep 13, 2023

thomas11 left a comment

Choose a reason for hiding this comment

t0yv0 Sep 13, 2023

Choose a reason for hiding this comment

iwahbe Sep 13, 2023

Choose a reason for hiding this comment

t0yv0 Sep 13, 2023 • edited Loading

Choose a reason for hiding this comment

iwahbe Sep 13, 2023

Choose a reason for hiding this comment

t0yv0 Sep 13, 2023

Choose a reason for hiding this comment

t0yv0 Sep 13, 2023

Choose a reason for hiding this comment

iwahbe Sep 13, 2023

Choose a reason for hiding this comment

t0yv0 commented Sep 13, 2023

t0yv0 commented Sep 13, 2023 • edited Loading

t0yv0 commented Sep 13, 2023

thomas11 commented Sep 13, 2023

t0yv0 commented Sep 13, 2023

iwahbe commented Sep 13, 2023

t0yv0 left a comment

Choose a reason for hiding this comment

danielrbradley commented Sep 25, 2023

t0yv0 commented Sep 25, 2023

Allow `pulumictl get version` on bare repositories #72

Allow `pulumictl get version` on bare repositories #72

t0yv0 Sep 13, 2023 •

edited

Loading

t0yv0 commented Sep 13, 2023 •

edited

Loading