roachprod: support --secure flag for start-tenant

[stevendanna]

The in-development tenant streaming features require a secure tenant
to test them properly.

This change adds support for the --secure flag when starting a
tenant. When passed, we use the host cluster to generate tenant-client
certificates and distribute them to the tenant cluster, along with the
CA and previously created client certificates.

For simplicity, we've stuck with using a single CA for all certificate
creation.

Release note: None

[cockroach-teamcity]

This change is 

[rimadeodhar]

Reviewed 4 of 4 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @otan and @stevendanna)

---

pkg/roachprod/install/cluster_synced.go line 1124 at r2 (raw file):

mkdir -p tenant-certs/certs
cp certs/ca.crt tenant-certs/certs/
cp certs/client.* tenant-certs/certs/

This will not work as expected with the new incoming code around tenant scoped certificates i.e. after this PR - #79065 merges.
Instead of copying the client certificates as is, you will need to recreate these certificates by adding a tenant scope with the specific tenant IDs. The CLI changes to add tenant scopes have already been merged so you should be able to update your PR to add a tenant scope along the lines of what is already being done for roachtests:

cockroach/pkg/cmd/roachtest/tests/multitenant_utils.go

Line 72 in fd26d19

if v.AtLeast(version.MustParse("v22.2.0")) {

.

[stevendanna]

@rimadeodhar Thanks, updated with the version check. PTAL.

[knz]

but also let's get a look from test-eng.

Reviewed 2 of 2 files at r3, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @srosenberg and @stevendanna)

[rimadeodhar]

The certs code

Reviewed all commit messages.
Reviewable status: complete! 2 of 0 LGTMs obtained (waiting on @srosenberg)

[stevendanna]

bors r+

[craig]

Build succeeded:

• GitHub CI (Cockroach)