feat(snap): Add support for environment variable injection

[farshidtz]

Supersceding #3924

This PR updates the snap to support environment variable injection as part of introducing the following scheme:

apps.<app>.<type>.<key>

where:

• <app> is the name of the app (service, executable)
• <type> is the type of option with respect to the app
• <key> is key for the option. It could contain a path to set a value inside an object, e.g. x.y=z sets {"x": {"y": "z"}}.

This change effectively makes options prefixed with env. deprecated. Such options will continue to work but be removed as of EdgeX 3.

The reasons for the schema change is as follows:

• The existing options depended on hardcoded mappings from keys to environment variables that was extremely hard to maintain. The new feature allows passing any environment variable dynamically.
• The environment variable passing was mixed with secrets-config executions causing various technical challanges and usability issues. With this change, the app configurations (via env files) use the apps.<app>.config. and config. prefixes exclusively.
• This makes it possible to extend the options for other settings (beyond config and secrets-config proxy), e.g. for controlling default service startup.

Environment Variables

The environment variable injection key uses the the following format:

• apps.<app>.config.<env-var>: setting app-specific env var ENV_VAR (e.g. apps.core-data.config.service-port=1000).
• config.<env-var>: setting global env vars by omitting the apps.<app> prefix (e.g. config.service-host=localhost, config.writable-loglevel=DEBUG)

Other <env-var> mapping examples:

• edgex-startup-duration: EDGEX_STARTUP_DURATION
• add-secretstore-tokens: ADD_SECRETSTORE_TOKENS.

The above design is restricted by snapd which supports only keys with lower-case letters, hypens, and dots. A dot is to define a parent-child relationship (x.y=z = x={"y": "z"}).

Secrets Proxy Options

New secrets-config proxy options:

• apps.secrets-config.proxy.tls.cert equivalent to env.security-proxy.tls-certificate
• apps.secrets-config.proxy.tls.key equivalent to env.security-proxy.tls-private-key
• apps.secrets-config.proxy.tls.snis equivalent to env.security-proxy.tls-sni
• apps.secrets-config.proxy.admin.public-key equivalent to env.security-proxy.public-key and env.security-proxy.user=admin,1,ES256

For usability purposes, the username, userid, algorithm of the single user have been hardcoded to admin, 1, ES256. This should be sufficient since we don't allow setting multiple users using snap options. The algorithm is set to ES256 rather than RS256, because it is more secure. If needed, it can be made configurable in future using keys apps.secrets-config.proxy.admin.{user|id|algorithm}.

Usage

The new scheme is allowed only after opting-in by setting config-enabled=trueapp-options=true. It will be enabled by default as of EdgeX 3.

Setting config-enabled=trueapp-options=true will:

1. unset existing env. options and ignore future sets.
2. convert:
   • env.security-secret-store.add-secretstore-tokens -> apps.security-secretstore-setup.config.add-secretstore-tokens
   • env.security-secret-store.add-known-secrets -> apps.security-secretstore-setup.config.add-known-secrets
   • env.security-bootstrapper.add-registry-acl-roles -> apps.security-bootstrapper.config.add-registry-acl-roles

---

If your build fails due to your commit message not passing the build checks, please review the guidelines here: https://github.com/edgexfoundry/edgex-go/blob/main/.github/Contributing.md

PR Checklist

Please check if your PR fulfills the following requirements:

• I am not introducing a breaking change (if you are, flag in conventional commit message with BREAKING CHANGE: describing the break)
• I am not introducing a new dependency (add notes below if you are)
• I have added unit tests for the new feature or bug fix (if not, why?) - unit tests, smoke tests
• I have fully tested (add details below) this the new feature or bug fix (if not, why?)
• I have opened a PR for the related docs change (if not, why?) - feat: Extend snap getting started docs edgex-docs#751, docs(snap): Move usage instructions to docs #4006

Testing Instructions

Install

snap install edgexfoundry --channel=edge/pr-3986

Run

sudo snap set edgexfoundry apps.core-data.config.service-port=1234

It should be rejected because config-enabled is not set.

Run

sudo snap set edgexfoundry config-enabled=true
sudo snap set edgexfoundry apps.core-data.config.service-port=8080
sudo snap set edgexfoundry config.service-startupmsg="testing env injection"

1. Confirm that all .env files have SERVICE_STARTUPMSG="testing env injection"
2. Confirm that the core-data .env file also has both the SERVICE_STARTUPMSG and SERVICE_PORT settings

$ cat /var/snap/edgexfoundry/current/config/core-data/res/core-data.env 
export SERVICE_STARTUPMSG=testing env injection
export SERVICE_PORT=8080
$ cat /var/snap/edgexfoundry/current/config/core-command/res/core-command.env 
export SERVICE_STARTUPMSG=testing env injection

New Dependency Instructions (If applicable)

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
7.4% Duplication

[MonicaisHer]

Validate error handling:

$ snap install edgexfoundry --channel=edge/pr-3986
$ snap set edgexfoundry config.service-startupmsg="testing env injection"
error: cannot perform the following tasks:
- Run configure hook of "edgexfoundry" snap (run hook "configure": 
-----
edgexfoundry: edgexfoundry:configure could not process config options: 'config.' and 'apps.' options are allowed only when config-enabled is true.

WARNING: Setting config-enabled=true will unset existing 'env.' options and ignore future sets!!
Exception: The following legacy 'env.' options are automatically converted:
	- env.security-secret-store.add-secretstore-tokens
	- env.security-secret-store.add-known-secrets
	- env.security-bootstrapper.add-registry-acl-roles
-----)

$ snap set edgexfoundry config-enabled=true
$ snap set edgexfoundry config.service.startupmsg="testing env injection"
error: cannot perform the following tasks:
- Run configure hook of "edgexfoundry" snap (run hook "configure": edgexfoundry: edgexfoundry:configure could not process config options: error converting config key to environment variable key: config key must not contain dots: service.startupmsg)

Test:

$ snap set edgexfoundry config.service-startupmsg="testing env injection"
$ snap restart edgexfoundry
Started.

$ snap logs -n=all edgexfoundry | grep "testing"
2022-04-26T13:30:27+02:00 edgexfoundry.core-command[10034]: level=INFO ts=2022-04-26T11:30:27.496559088Z app=core-command source=variables.go:352 msg="Variables override of 'Service.StartupMsg' by environment variable: SERVICE_STARTUPMSG=testing env injection"
2022-04-26T13:30:28+02:00 edgexfoundry.core-data[10084]: level=INFO ts=2022-04-26T11:30:28.751877152Z app=core-data source=variables.go:352 msg="Variables override of 'Service.StartupMsg' by environment variable: SERVICE_STARTUPMSG=testing env injection"
2022-04-26T13:30:30+02:00 edgexfoundry.core-metadata[10136]: level=INFO ts=2022-04-26T11:30:30.24381414Z app=core-metadata source=variables.go:352 msg="Variables override of 'Service.StartupMsg' by environment variable: SERVICE_STARTUPMSG=testing env injection"


$ snap set edgexfoundry apps.core-data.config.service-port=11111
$ snap restart edgexfoundry.core-data
Restarted.

$ snap logs -n=all edgexfoundry | grep "11111"
2022-04-26T13:49:49+02:00 edgexfoundry.core-data[18760]: level=INFO ts=2022-04-26T11:49:49.030920692Z app=core-data source=variables.go:352 msg="Variables override of 'Service.Port' by environment variable: SERVICE_PORT=11111"
2022-04-26T13:49:49+02:00 edgexfoundry.core-data[18760]: level=INFO ts=2022-04-26T11:49:49.060236271Z app=core-data source=httpserver.go:123 msg="Web server starting (localhost:11111)"

$ cat /var/snap/edgexfoundry/current/config/core-data/res/core-data.env 
export SERVICE_STARTUPMSG="testing env injection"
export SERVICE_PORT="11111"
$ cat /var/snap/edgexfoundry/current/config/device-virtual/res/device-virtual.env 
export SERVICE_STARTUPMSG="testing env injection"

[MonicaisHer]

@farshidtz Thank you. It works as expected. It also passed the checkbox test security-services-proxy-certs locally with new secrets-config proxy options.

[tonyespy]

Overall this looks good, testing works as expected. That said, there are a couple of comments in-line that should be reviewed before approval.

Also curious why updates to snap/README weren't included (has the doc moved elsewhere)?

[tonyespy]

I'd suggest dropping this as the SMA is considered deprecated and will be removed completely in EdgeX 3.0.

[farshidtz]

Combining the new options with the old ones is disallowed. As a result, it will not be possible for someone to configure SMA using the old options and everything else using the new ones. The only way to respect the deprecation policy is to keep this service functional (and configurable) until EdgeX 3.0 release, at which we can drop that too.

[tonyespy]

I wasn't suggesting combining the options, I was just suggesting that you don't support the SMA at all with the new scheme. And afaik there's nothing in the deprecation policy which states that new features must support deprecated services or applications, that would seem to defeat the purpose of deprecation IMHOP.

[farshidtz]

As explained, this is necessary to allow using the new scheme with applications while still allowing the configuration of SMA. This isn't a new feature for SMA, but a new feature for the snap as a whole and that should not make the SMA incompatible with everything else when using the new scheme.

[tonyespy]

Shouldn't this require be referencing a released version?

[farshidtz]

We'd be doing more beta testing and add a stable version before the release. I believe other 1st party EdgeX dependencies will also get passed the code freeze with dev versions and switch to stable after a week or so. At least that's what happened in the previous release.

[tonyespy]

I don't love the fact we use the service name secrets-config here, as its not really a service, it's an application. Is there a reason you changed this from security-proxy?

[farshidtz]

The new options are grouped per app and here, we are calling the processing of options for the secrets-config app. It should have been --app=secrets-config for accuracy but that'd involve changes to the parts that are being deprecated. I will add an issue to take care of this as part of future refactoring. But for now I would leave it as is because this is an internal CLI typically not seen or used by any user.

[tonyespy]

That doesn't really make sense as you're not configuring the secrets-config application, you're configuring the proxy, secrets-config is a helper application.

[farshidtz]

If at some point the proxy supports provisioning of users and TLS certificates via environment variables, then we can use the apps.secrets-proxy.config namespace to set those. In the meantime and in the scope of the new scheme, a user has to rely on the secrets-config app to perform the operations during runtime by setting e.g. snap set edgexfoundry apps.secrets-config.proxy.admin.public-key=xxx. Note that there is no config in the key because this isn't a configuration override.

The app in the suggested --app=<app> CLI argument is equivalent to apps.<app> namespace for snap options. In other words, as said, we are calling the processing of options for the secrets-config app. For now, only proxy is supported so we don't need to specify that in the CLI.

[farshidtz]

Overall this looks good, testing works as expected. That said, there are a couple of comments in-line that should be reviewed before approval.

Thanks a lot @tonyespy for the much needed code review. I've replied to your comments.

Also curious why updates to snap/README weren't included (has the doc moved elsewhere)?

We will be updating the snap/README and docs in the next two weeks, before the release. I'll update the PR description to make that clear.