[CI/infra] Git submodule woes: improve contributor experience

[chalin]

It doesn't happen often, but we've had too many (IMHO) situations where contributors adding content unrelated to any of the Git submodules, end up submitting OTel commits with mis-aligned submodules (i.e., submodules pointing to old commits rather than the updated commits available in main).

I think that this situation is solvable while keeping submodules, but it might make for a better overall contributor experience to eliminate the submodules since it wouldn't require each developers to support local commit hooks (for example).

The initial discussion started in https://cloud-native.slack.com/archives/C040BF9TG9J/p1690372828700639 (accessible only to Comms maintainers), which I quote partially here:

@svrnm - Any ideas / thoughts how we can improve the situation with ppl having changes to the git submodules in their commits.
I had a few ideas but not sure if any of them fly:

• add commit/push hooks that ask ppl to confirm if they have changes to these files
• use a GitHub bot or similar to clean them up
• Use alternatives to submodules (not sure if anything usable exists)

Alternatives that were mentioned:

• https://gohugo.io/hugo-modules/
• A bot action to run npm run get:submodule
• Use Git subtree feature

I think that we can handle this on a case-by-case basis. I'm tempted to start with Docsy.

• [CI] Auto-update-versions: fix action + pin gitmodules #3912

[svrnm]

I think that we can handle this on a case-by-case basis. I'm tempted to start with Docsy.

+1, it's probably the easiest as it might work with hugo modules.

[svrnm]

Could we manage the modules ourselves somehow? e.g. we have a config file that holds the right commits and we have a npm script that pulls exactly that commit on run?

[chalin]

@svrnm @theletterf et al., with the new (not yet officially supported) script introduced by #3912, I was able to "fix" the git submodule skew in:

• wip: Establish Security section for OTel documentation #3652

(I had fixed it "manually", but then reverted the commit, and tried the new script ...)

Simply run the following command and commit the resulting submodule updates will resolve submodule skew:

$ ./scripts/sync-submodules.pl
> cd themes/docsy && git switch --detach v0.8.0-11-gce52bb0
HEAD is now at ce52bb0 RSS button: link to XML via RelPermalink (#1786)
> cd content-modules/opentelemetry-specification && git switch --detach v1.29.0
HEAD is now at c6520a7 Release 1.29.0 (#3808)
> cd content-modules/community && git switch --detach 0cb6dca
HEAD is now at 0cb6dca [editorial] Fix mission-vision-values.md formatting (#1742)
> cd content-modules/opentelemetry-proto && git switch --detach v1.1.0
HEAD is now at 4ca4f03 Prepare release 1.1.0 (#523)
> cd content-modules/semantic-conventions && git switch --detach v1.24.0
HEAD is now at cafda71 Staging the 1.24.0 release (#613)
> cd content-modules/opamp-spec && git switch --detach v0.8.0-5-g0360da8
HEAD is now at 0360da8 [editorial] Drop 404 link and cleanup references (#178)

So, ... this might be a way to help our contributors who have long standing PRs that need to be rebased, and for which the submodules need to be (easily) synced too. My, I've been wanting to have this script for a long time!

Btw, this pushes the burden on us maintainers to ensure that all submodules are pinned to the proper version/commit, as specified in .gitmodules. The auto-version-update action handles some of the updates, but not all. Anyhow, this feels like some progress. 🤷🏼‍♂️😄

[svrnm]

@chalin I love this, great progress indeed! If we have a one click solution like that to fix any submodule issues, we have it solved 90%...

Btw, this pushes the burden on us maintainers to ensure that all submodules are pinned to the proper version/commit, as specified in .gitmodules.

I am totally fine with that.

Question: if one now wants to update(!) any submodules themselves, would it be enough to update that script, run it, and then submit the script+.gitmodules change? It feels that way you only have to change things at one place?

[chalin]

For now the process I see for updating, say Docsy, would be to update the pinned version in .gitmodlues and run the script. Right, only one place to change.

[svrnm]

Awesome, looking forward to see this in action :-)

[chalin]

I'm going to mark this as closed. IMHO, the sync script has been vetted and is accessible to committers via a /fix:* command.