add the ability to use a secret persistence

[jrhizor]

This PR is meant to enable external secret persistences and optional coordinate-based secret access throughout the system.

It's probably most useful to start by looking at ConfigRepository and following through execution paths for syncs and check/discover workflows.

There are a few things I'd like feedback on here:

• Is there a better way of optionally offering secret access?
• How should we handle the migration process? (see comment)
• Check/discover workflows don't store a config with a set id but they do need to store their secrets outside of temporal. Is there a better alternative to using a TTL?

If you want to play around with it locally I recommend setting SECRET_PERSISTENCE to TESTING_CONFIG_DB_TABLE so you don't need to deal with GCP secrets.

todos:

• offer the option to expose partial configs and do combining at the worker layer
• verify that updating connection information works
• handle making ephemeral secrets in check/discover
• make sure migrations work as expected (open question in todo still)
• figure out why the acceptance tests create superfluous secret versions
• add real feature flagging

out of scope for this pr (create tickets):

• adding to cloud
• add automated testing to verify the secrets aren't stored in the db
• add automated testing to verify the secrets aren't stored in temporal
• removing oauth secrets from temporal
• ticket to track number of secrets (ephemeral and permanent) that we have in our system
• create a debug docker-compose extension file that exposes db ports and also gives the temporal ui

[jrhizor]

For step 1, just running a sync from the config repository while accessing a db-based secrets layer is running!
Secrets are still being passed to temporal and I haven't tested import/export functionality, but it's looking on the right track!

[jrhizor]

todo: remove this before merging

[jrhizor]

todo: fix

[cgardens]

👍

[cgardens]

looks good! I think this iteration makes some nice improvements.

there were a couple questions and naming things i pointed out that i think didn't get covered. it is fine if the answer is that you disagree and are going to skip them. just want to make sure they were seen.