Document tools required to work in this repo and deploy hubs successfully

[sgibson91]

Summary

As a new starter, I gleaned most of the tools I'd need installed to successfully deploy a hub from either past hub experience (kubectl, helm, terraform and the specific cloud cli are safe bets, for example) or from reading different guides (e.g. sops and kops), and I'm sure as I dig in further I'll discover more. I wonder could we begin to curate a reference list of tools in the docs that are needed to successfully deploy a new hub from this repo? I understand this framework is still under development and may change as new technologies emerge, standards change, or our cloud platform stories change - so could we categorise these tools as follows.

• Critical requirements
• Desirables
• Deprecating
• Incoming (perhaps linked to the deprecating as 'X superseding Y')

It might also be a nice opportunity for the engineering team to share tools they like that make their day-to-day tasks easier under a "Misc." category. Thoughts?

Actions

• Curate a list of common tools
  • kubectl
  • helm
  • terraform
  • kops
  • eksctl
  • sops
  • gcloud
  • awscli
  • docker
  • jsonnet
  • Package management for Python (pip, conda, etc)
  • Package management for R
  • Azure stuff? az cli
• Develop an appropriate set of categories that reflect the dependency strength between the tool and current workflow implementation
• Organise the tools accordingly
• Decide on the appropriate place in the docs to host such a list
  • Perhaps at the beginning of the "operating the hubs" how-to guide
• Publish the list at that location

[GeorgianaElena]

Critical requirements
Desirables
Deprecating
Incoming (perhaps linked to the deprecating as 'X superseding Y')

I believe documenting these tools is a great idea. The categorization you proposed makes a lot of sense IMO. I think it could also serve as a progress marker, something that could help us and others learn things like, why we switched to a different tool, if there are specific reasons why a certain tool was chosen and not another.

It might also be a nice opportunity for the engineering team to share tools they like that make their day-to-day tasks easier under a "Misc." category. Thoughts?

I would love to see such a list and think it would be a great learning opportunity :)

Thanks for proposing this @sgibson91!

[yuvipanda]

I opened #497 for a take on this. We can move the structure around to fit - am excited to learn about tools and workflows from others :)

[choldgraf]

With @yuvipanda's first pass at this in #497, are there things that we are obviously missing for this issue? What extra pieces of information would we need in order to consider this one addressed and close it?

[damianavila]

In the PR I suggested filling this in an iterative way... so maybe having something already merged is good enough to close this one?

[yuvipanda]

I think we should add at least bits about AWS (kops, awscli, etc) before we can close this

[choldgraf]

Could we convert whatever remaining steps into "to do" items in the checklist in the top comment, so that we know what needs to be done to close it out?

[damianavila]

Amended the first (partially done) item (with the missing AWS stuff) and ticked the rest of the todo items because we already had those with the merged PR.

[yuvipanda]

I've updated the issue with a list of tools I think we should document, and opened #511 to add to the docs :)

[damianavila]

Should we add something about jsonnet? conda?

[yuvipanda]

@damianavila done.

[yuvipanda]

I think this is good enough for now, we can incrementally improve documentation as we go along