[krel] Refactor existing bash scripts into go tools

[saschagrunert]

The main target is to port the existing bash scripts into golang based versions on top of the existing krel command line tool. The scripts should be added as subcommands to krel whereas common utilities should be re-used over the whole package.

Things to be done

• (@justaugustus) Create initial skeleton of krel and krel ff (port of branchff) - cmd/krel/cmd/ff: Initial commit of Go-based branchff tool #869
• (@justaugustus, @saschagrunert ) resolve all TODOs in cmd/krel/ff.go
  • Add DescribeTag method for Repo #938, Make branch ff ready to be used #939
• (@saschagrunert, @hasheddan) migrate relates notes git package (pkg/notes/git.go) into the pkg/util/git.go package
  • Move notes git implementation into dedicated package #920, Bind git branch verification to Repo #936, Refactor the rest of the git methods #937, Add git package unit tests #942
• refactor utils to contain a dedicated command package (Add command package and tests #922)
• migrate the bash scripts
  • make a plan in which order the scripts should be ported
  • push-build.sh → cmd/krel/cmd/push.go
    • push build: initial migration to krel #941, [krel] Add release artifacts staging to krel push #1105
  • compile-release-tools
    • (compile-release-tools: Update script to compile all release tools #1047) Compile all release tools in compile-release-tools
  • find_green_build
    • Remove find_green_build script in favor of dl.k8s.io #1078
  • (@justaugustus - Fork test-infra gcbuilder and initial commit of krel gcbmgr #1081) drop gcbmgr in favor of the builder tool
  • release-notify
    • Remove release-notify script #1323, Remove krel patch-announce #1315, Add krel announce subcommand #1173
  • changelog-update
    • Remove unused changelog-update script #983
  • relnotes - use Go implementation from @saschagrunert et al
    • Add binary downloads table to release notes #940, Define krel changelog subcommand scope #982, Add table of contents support #980, Use string builder to avoid copies on notes rendering #978, Add support for patch-to-patch discovery mode #975, Move release notes options into notes package #987, Add first part of changelog command implementation #991, Add markdown creation support for krel changelog #992, Add changelog remote lookup feature #1003, Add HTML output to krel changelog #1005, Add commit support for changelog subcommand #1014, Add first patch release changelog integration test #1023, Add git -rm CHANGELOG-*.md to krel changelog #1043, Add dedicated --html-file output option to krel changelog #1042, Use krel changelog instead of relnotes bash script #1044, Patch release announce in golang #1040, Add krel changelog integration tests #1068, Uppercase the first character for each markdown note #1075, Bind RenderMarkdown function to *Document #1074, Add dedicated release notes section for 'action required' notes #1073, Trim leading and trailing whitespace from rendered markdown #1072, Sort release notes alphabetically #1071, Rename ListCommitsWithNotes and make method private #1070, Make SIG list more compact #1069
  • (@justaugustus / @kubernetes/patch-release-team to supervise) anago
    • [krel] anago: Initial commit #1106
• Add integration tests to verify the tooling from a use-case perspective (see Make branch ff ready to be used #939 (comment))

Related: #834, #959

/cc @justaugustus @hasheddan

[justaugustus]

Thanks for opening this! I'm going to come back and shift some things around later as well as drop notes on our #release-management Slack discussion (ref: #869 (comment))

/assign @saschagrunert @justaugustus @hasheddan
/milestone v1.18
/priority important-longterm
/kind feature cleanup
/sig release
/area release-eng

[justaugustus]

/assign @saschagrunert @justaugustus @hasheddan

[justaugustus]

@saschagrunert @hasheddan -- Please work on reconciling pkg/notes/git.go into the pkg/util/git.go and krel push first. Feel free to break up the work however you see fit.

I'll be working on krel ff.

[hasheddan]

Small nitpick: I am of the opinion that we should try to avoid putting things into a util package as it has potential to grow and become bloated with functionality. Currently, the functions in common.go seems like a fair candidate to be part of a util package (although maybe they could go into a cli package or something), but I think the functionality in pkg/notes/git.go should likely go into a pkg/git package which feels more descriptive. My feelings on this are pretty accurately captured here: https://dave.cheney.net/2019/01/08/avoid-package-names-like-base-util-or-common

[saschagrunert]

Small nitpick: I am of the opinion that we should try to avoid putting things into a util package as it has potential to grow and become bloated with functionality. Currently, the functions in common.go seems like a fair candidate to be part of a util package (although maybe they could go into a cli package or something), but I think the functionality in pkg/notes/git.go should likely go into a pkg/git package which feels more descriptive. My feelings on this are pretty accurately captured here: https://dave.cheney.net/2019/01/08/avoid-package-names-like-base-util-or-common

We have the same mind, please take a look at #920

[justaugustus]

Perfect! This was going to be part of my notes :)

[saschagrunert]

What would be the next steps to move forward here? I suggest to finish the git refactoring (and branch ff adaptions) before porting other scripts. WDYT? Is anyone already working on the git side? (If not, I could)

[saschagrunert]

@hoegaarden regarding the release notes tool we could add another --discover mode like introduced here: 532f7df

[saschagrunert]

I think the git package is fine for now, so we can continue to work on ff.go where I expect that more methods from there could go into the git package.

[justaugustus]

Sascha and Dan have this well in hand, so I'm unassigning myself.
Please still tag me for reviews on these PRs though.
/unassign

[justaugustus]

anago deprecation announced on k-dev: https://groups.google.com/d/topic/kubernetes-dev/Mhpx-loSBns/discussion

[saschagrunert]

anago deprecation announced on k-dev: https://groups.google.com/d/topic/kubernetes-dev/Mhpx-loSBns/discussion

Alright we should probably start converting anago itself into a krel subcommand. Something like krel run? Then krel could call the internal subcommand functions directly...

[justaugustus]

Alright we should probably start converting anago itself into a krel subcommand. Something like krel run? Then krel could call the internal subcommand functions directly...

@saschagrunert -- Yep. I'm planning on taking the first pass on the anago rewrite before handing it off to y'all for optimizations.

I've begun work on krel gcbmgr here: #1081

[saschagrunert]

Update, the release-notify part is done with #1323

[justaugustus]

Great progress team!

[saschagrunert]

We're pretty far. I created new issues for the rest of the migration:

• [krel] Introduce krel subcommand for image promotion #1445
• [krel] Introduce krel subcommand for pushing git objects #1446
• [krel] Introduce krel subcommand for building the Annoucement #1452
• [krel] Finish-up krel push #1459
• [krel] Add subcommand to replace find_green_build script #1460
• [krel] Add subcommand to refactor build-version retrieval from anago #1461

Let's close this one since it has been open for quite a while and we can split-up the work better
/close

[k8s-ci-robot]

@saschagrunert: Closing this issue.

In response to this:

We're pretty far. I created new issues for the rest of the migration:

• [krel] Introduce krel subcommand for image promotion #1445
• [krel] Introduce krel subcommand for pushing git objects #1446
• [krel] Introduce krel subcommand for building the Annoucement #1452
• [krel] Finish-up krel push #1459
• [krel] Add subcommand to replace find_green_build script #1460
• [krel] Add subcommand to refactor build-version retrieval from anago #1461

Let's close this one since it has been open for quite a while and we can split-up the work better
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.