From adc5eb2e96a2d67a38021dbeb3b3cddcafa74de8 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Tue, 15 Mar 2022 19:02:44 +0100 Subject: [PATCH 01/29] feat(microservices): add snap config section Signed-off-by: Farshid Tavakolizadeh --- .../configuration/snap/Configuration.md | 20 +++++++++++++++++++ .../configuration/snap/ContentInterfaces.md | 5 +++++ mkdocs.yml | 3 +++ 3 files changed, 28 insertions(+) create mode 100644 docs_src/microservices/configuration/snap/Configuration.md create mode 100644 docs_src/microservices/configuration/snap/ContentInterfaces.md diff --git a/docs_src/microservices/configuration/snap/Configuration.md b/docs_src/microservices/configuration/snap/Configuration.md new file mode 100644 index 0000000000..50ee483d85 --- /dev/null +++ b/docs_src/microservices/configuration/snap/Configuration.md @@ -0,0 +1,20 @@ +# Configuration + + +## Changing config files +- how to locally modify files +- link to config provider page + +## Override configurations +- Environment variable injection +- Link to consul doc page + +### Seed defaults from a gadget + +## Control service startup +snap enable / disable + +### Control defaults from a Gadget +For seeding defaults from a gadget. +- edgexfoundry (service on/off) +- device/app services (autostart bool) diff --git a/docs_src/microservices/configuration/snap/ContentInterfaces.md b/docs_src/microservices/configuration/snap/ContentInterfaces.md new file mode 100644 index 0000000000..d878b15a02 --- /dev/null +++ b/docs_src/microservices/configuration/snap/ContentInterfaces.md @@ -0,0 +1,5 @@ +# Content Interfaces + +## Secret interface + +## Config interface diff --git a/mkdocs.yml b/mkdocs.yml index 9968602411..55d46d0762 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -64,6 +64,9 @@ nav: - './microservices/configuration/CommonEnvironmentVariables.md' - './microservices/configuration/ConfigurationAndRegistry.md' - './microservices/configuration/V2MigrationCommonConfig.md' + - Snaps: + - './microservices/configuration/snap/Configuration.md' + - './microservices/configuration/snap/ContentInterfaces.md' - Core Services: - './microservices/core/Ch-CoreServices.md' - './microservices/core/data/Ch-CoreData.md' From a8a045bb00d38c1837aa9fad2c4d88945d9cd904 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Wed, 16 Mar 2022 18:13:18 +0100 Subject: [PATCH 02/29] feat: Extend the edgexfoundry snap docs Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 403 ++++++++++++++---- .../configuration/snap/Configuration.md | 4 +- .../configuration/snap/ContentInterfaces.md | 4 +- 3 files changed, 313 insertions(+), 98 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 023cb7d6a9..c6383419ef 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -6,170 +6,385 @@ secure, cross‐platform and self-contained. Snaps can be installed on any Linux distribution with [snap support](https://snapcraft.io/docs/installing-snapd). -Snap packages of EdgeX services are published on the [Snap Store](https://snapcraft.io). The list of all EdgeX snaps is available [below](#edgex-snaps). +### Installation +When using the snap CLI, the installation is possible by simply executing: +```bash +snap install +``` + +This is similar to setting `--channel=latest/stable` or shorthand `--stable` and will install the latest stable release of a snap. In this case, `latest/stable` is the [channel](https://snapcraft.io/docs/channels), composed of `latest` track and `stable` risk level. + +To install a specific version with long term support (LTS), or to install a beta +or development release, refer to the store page for the snap, choose install, and +then pick the desired channel. +The store page also provides instructions for installation on different Linux distributions as well as the list of supported CPU architectures. + +Snap packages of EdgeX services are published on the [Snap Store](https://snapcraft.io). +The list of all EdgeX snaps is available [below](#edgex-snaps). + +### Control services +The services of a snap can be started/stopped/restarted using the snap CLI. +When starting/stopping, you can additionally set them to enable/disable which configures whether or not the service should also start on boot. + +To list the services and check their status: +```bash +snap services +``` + +To start and enable services: +```bash +# all services +snap start --enable +# one service +snap start --enable . +``` + +For controlling the default service startup from a gadget snap, see [TBA](tba). + +### Debugging +The service logs can be queried using the `snap log` command. + +For example, to query 100 lines and follow: +```bash +snap logs -n=100 -f +``` +Check `snap logs --help` for details. + +To query not only the service logs, but also the installation logs (from various hooks), use `journalctl`. + +For example, to query 100 lines, follow, and filter: +```bash +journalctl -n 100 -f | grep +``` ## EdgeX Snaps -The following snaps are maintained by the EdgeX working groups: +The following snaps are maintained by the EdgeX working groups. +To find all EdgeX snaps on the public Snap Store, [search by keyword](https://snapcraft.io/search?q=edgex). -Platform snap: +### Platform Snap +[![Get it from the Snap Store][badge]][edgexfoundry] -- [edgexfoundry](https://snapcraft.io/edgexfoundry): the main platform snap containing +The main platform snap, simply called +[edgexfoundry] contains all reference core services along with several other security, supporting, application, and device services. -Development tools: +#### Services + +Upon installation, the following EdgeX services are automatically and immediately started: + +- consul (Registry) +- core-command +- core-data +- core-metadata +- kong-daemon (API Gateway a.k.a. Reverse Proxy) +- postgres (kong's database) +- redis (default Message Bus and database backend for core-data and core-metadata) +- security-bootstrapper-redis (oneshot service) +- security-consul-bootstrapper (oneshot service) +- security-proxy-setup (oneshot service) +- security-secretstore-setup (oneshot service) +- vault (Secret Store) + +The following services are disabled by default: -- [edgex-ui](https://snapcraft.io/edgex-ui) -- [edgex-cli](https://snapcraft.io/edgex-cli) +- app-service-configurable (used to filter events for kuiper) +- device-virtual +- kuiper (Rules Engine) +- support-notifications +- support-scheduler +- sys-mgmt-agent -Application services: +The disabled services can be manually enabled and started; see [Control services](#control-services). -- [edgex-app-service-configurable](https://snapcraft.io/edgex-app-service-configurable) +#### Configuring individual services -Device services: +All default configuration files are shipped with the snap inside `$SNAP/config`, however because `$SNAP` isn't writable, +all of the config files are copied during snap installation to `$SNAP_DATA/config`. -- [edgex-device-camera](https://snapcraft.io/edgex-device-camera) -- [edgex-device-modbus](https://snapcraft.io/edgex-device-modbus) -- [edgex-device-mqtt](https://snapcraft.io/edgex-device-mqtt) -- [edgex-device-rest](https://snapcraft.io/edgex-device-rest) -- [edgex-device-snmp](https://snapcraft.io/edgex-device-snmp) -- [edgex-device-grove](https://snapcraft.io/edgex-device-grove) +!!! Tip + `$SNAP` resolves to the path `/snap/edgexfoundry/current/` and `$SNAP_DATA` resolves to `/var/snap/edgexfoundry/current`. -Other EdgeX snaps do exist on the public Snap Store ([search by keyword](https://snapcraft.io/search?q=edgex)) or private stores under brand accounts. +The preferred way to change the configuration is to use [Configuration Overrides](#configuration-overrides) section below. +It is also possible to change configuration directly via Consul's [UI](http://localhost:8500/ui/) or [kv REST API](https://www.consul.io/api/kv). +Changes made to configuration in Consul require services to be restarted in order for the changes to take effect; +the one exception are changes made to configuration items in a service's `[Writable]` section. +Services that aren't started by default (see [Using the EdgeX snap](#using-the-edgex-snap) section above) +*will* pickup any changes made to their config files when started. -## Installing the `edgexfoundry` snap -[![Get it from the Snap Store](https://snapcraft.io/static/images/badges/en/snap-store-white.svg)](https://snapcraft.io/edgexfoundry) +Also it should be noted that use of Consul is enabled by default in the snap. It is not possible at this time to run the EdgeX services in +the snap with Consul disabled. -This is the main platform snap which -contains all reference core services along with several other security, supporting, application, and device services. -The Snap Store allows access to multiple versions of the [EdgeX Foundry](https://snapcraft.io/edgexfoundry) snap using [channels](https://snapcraft.io/docs/channels). If not specified, snaps are installed -from the default `latest/stable` channel. +#### Configuration Overrides +!!! Note + Deprecated. To be replaced with new scheme allowing env injection. -You can see the current snap channels available for your machine's architecture by running the command: +The EdgeX snap supports configuration overrides via its configure and install hooks which generate service-specific .env files +which are used to provide a custom environment to the service, overriding the default configuration provided by the service's `configuration.toml` file. +If a configuration override is made after a service has already started, then the service must be **restarted** via command-line +(e.g. `snap restart edgexfoundry.`), or [snapd's REST API](https://snapcraft.io/docs/snapd-api). +If the overrides are provided via the snap configuration defaults capability of a gadget snap, +the overrides will be picked up when the services are first started. -```bash -snap info edgexfoundry -``` +The following syntax is used to specify service-specific configuration overrides: -In order to install a specific version of the snap by setting the `--channel` flag. -For example, to install the Jakarta (2.1) release: +`env...` + +For instance, to setup an override of Core Data's port use: ```bash -sudo snap install edgexfoundry --channel=2.1 +sudo snap set edgexfoundry env.core-data.service.port=2112 ``` -To install the latest beta: +And restart the service: + ```bash -sudo snap install edgexfoundry --channel=latest/beta -# or using the shorthand -sudo snap install edgexfoundry --beta +sudo snap restart edgexfoundry.core-data ``` -Replace `beta` with `edge` to get the latest nightly build! +**Note** - at this time changes to configuration values in the [Writable] section are not supported. ---- +For details on the mapping of configuration options to config options, +please refer to [Service Environment Configuration Overrides](#service-environment-configuration-overrides) section. -Upon installation, the following internal EdgeX services are automatically started: -- consul -- vault -- redis -- kong -- postgres -- core-data -- core-command -- core-metadata -- security-services (see [Security Services section](#security-services) below) +#### Security services + +Currently, The EdgeX snap has security (Secret Store and API Gateway) enabled by default. The security services constitute the following components: + +- kong-daemon (API Gateway a.k.a. Reverse Proxy) +- postgres (kong's database) +- vault (Secret Store) + +Oneshot services which perform the necessary security setup and stop, when listed using `snap services`, they show up as `enabled/inactive`: + +- security-proxy-setup (kong setup) +- security-secretstore-setup (vault setup) +- security-bootstrapper-redis (secure redis setup) +- security-consul-bootstrapper (secure consul setup) + +Vault is known within EdgeX as the Secret Store, while Kong+PostgreSQL are used to provide the EdgeX API Gateway. + +For more details please refer to the snap's [Secret Store](https://github.com/edgexfoundry/edgex-go/blob/main/snap/README.md#secret-store) and [API Gateway](https://github.com/edgexfoundry/edgex-go/blob/main/snap/README.md#api-gateway) documentation. -The following services are disabled by default: -- app-service-configurable (required for eKuiper) -- device-virtual -- kuiper -- support-notifications -- support-scheduler -- sys-mgmt-agent -Any disabled services can be enabled and started up using `snap set`: +#### API Gateway user setup + +Before the API Gateway can be used, a user and group must be created and a JWT access token generated. + +1. The first step is to create a public/private keypair for the new user, which can be done with ```bash -sudo snap set edgexfoundry support-notifications=on +# Create private key: +openssl ecparam -genkey -name prime256v1 -noout -out private.pem + +# Create public key: +openssl ec -in private.pem -pubout -out public.pem ``` -To turn a service off (thereby disabling and immediately stopping it) set the service to off: +2. The next step is to create the user. The easiest way to create a single API gateway user is to use `snap set` to set two values as follows: ```bash -sudo snap set edgexfoundry support-notifications=off +# set user=username,user id,algorithm (ES256 or RS256) +sudo snap set edgexfoundry env.security-proxy.user=user01,USER_ID,ES256 + +# set public-key to the contents of a PEM-encoded public key file +sudo snap set edgexfoundry env.security-proxy.public-key="$(cat public.pem)" ``` -All services which are installed on the system as systemd units, which if enabled will automatically start running when the system boots or reboots. +To create multiple users, use the secrets-config command. You need to provide the following: -### Configuring individual services -This snap supports configuration overrides via snap configure hooks which generate service-specific .env files which are used to -provide a custom environment to the service, overriding the default configuration provided by the service's `configuration.toml` -file. If a configuration override is made after a service has already started, then the service must be **restarted** via command-line -(e.g. `snap restart edgexfoundry.`), or [snapd's REST API](https://snapcraft.io/docs/snapd-api). If the overrides are provided via the snap configuration defaults -capability of a gadget snap, the overrides will be picked up when the services are first started. +- The username +- The public key +- The API Gateway Admin JWT token +- (optionally) ID. This is a unique string identifying the credential. It will be required in the next step to +create the JWT token. If you don't specify it, +then an autogenerated one will be output by the secrets-config command +```bash -The following syntax is used to specify service-specific configuration overrides for the edgexfoundry snap: +# get API Gateway/Kong token +JWT_FILE=/var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt +JWT=`sudo cat ${JWT_FILE}` +# use secrets-config to add user +edgexfoundry.secrets-config proxy adduser --token-type jwt --user user01 --algorithm ES256 --public_key public.pem --id USER_ID --jwt ${JWT} ``` -env... + +3. Finally, you need to generate a token using the user ID which you specified: + +```bash +# get token +TOKEN=`edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id USER_ID --expiration=1h` + +# Keep this token in a safe place for future reuse as the same token cannot be regenerated or recovered using the secret-config CLI +echo $TOKEN ``` -For instance, to setup an override of core data's port use: +Alternatively , you can generate the token on a different device using a bash script: ```bash -sudo snap set edgexfoundry env.core-data.service.port=2112 +header='{ + "alg": "ES256", + "typ": "JWT" +}' + +TTL=$((EPOCHSECONDS+3600)) + +payload='{ + "iss":"USER_ID", + "iat":'$EPOCHSECONDS', + "nbf":'$EPOCHSECONDS', + "exp":'$TTL' +}' + +JWT_HEADER=`echo -n $header | openssl base64 -e -A | sed s/\+/-/ | sed -E s/=+$//` +JWT_PAYLOAD=`echo -n $payload | openssl base64 -e -A | sed s/\+/-/ | sed -E s/=+$//` +JWT_SIGNATURE=`echo -n "$JWT_HEADER.$JWT_PAYLOAD" | openssl dgst -sha256 -binary -sign private.pem | openssl asn1parse -inform DER -offset 2 | grep -o "[0-9A-F]\+$" | tr -d '\n' | xxd -r -p | base64 -w0 | tr -d '=' | tr '+/' '-_'` +TOKEN=$JWT_HEADER.$JWT_PAYLOAD.$JWT_SIGNATURE ``` -And restart the service: +4. Once you have the token you can access the API Gateway as follows: -```bash -sudo snap restart edgexfoundry.core-data +The JWT token must be included +via an HTTP `Authorization: Bearer ` header on any REST calls used to access EdgeX services via the API Gateway. + +Example: + +```bash +curl -k -X GET https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" ``` -!!! Note - At this time changes to configuration values in the `[Writable]` section are not supported. -For details on the mapping of configuration options to Config options, please refer to [Service Environment Configuration Overrides](https://github.com/edgexfoundry/edgex-go/blob/main/snap/README.md#configuration-overrides). +#### API Gateway TLS certificate setup + +By default Kong is configured with a self-signed TLS certificate (which you find in `/var/snap/edgexfoundry/current/kong/ssl/kong-default-ecdsa.crt`). +It is also possible to install your own TLS certificate to be used by the gateway. The steps to do so are as follows: -### Viewing logs -To view the logs for all services in an EdgeX snap use the `snap log` command: +1. Start by provisioning a TLS certificate to use. You can use a number of tools for that, such as `openssl` or the `edgeca` snap: ```bash -sudo snap logs edgexfoundry +sudo snap install edgeca +edgeca gencsr --cn localhost --csr csrfile --key csrkeyfile +edgeca gencert -o localhost.cert -i csrfile -k localhost.key ``` -Individual service logs may be viewed by specifying the service name: +2. Then install the certificate: ```bash -sudo snap logs edgexfoundry.consul +sudo snap set edgexfoundry env.security-proxy.tls-certificate="$(cat localhost.cert)" +sudo snap set edgexfoundry env.security-proxy.tls-private-key="$(cat localhost.key)" ``` -Or by using the systemd unit name and `journalctl`: +3. Specify the EdgeCA Root CA certificate with `--cacert` for validation of the new certificate: ```bash -journalctl -u snap.edgexfoundry.consul +curl -v --cacert /var/snap/edgeca/current/CA.pem -X GET https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" ``` -These techniques can be used with any snap including application snap and device services snaps. +Optionally, to specify a server name other than `localhost`, set the `tls-sni` configuration setting first. Example: -### Security services +```bash +# generate certificate and private key +edgeca gencsr --cn server01 --csr csrfile --key csrkeyfile +edgeca gencert -o server.cert -i csrfile -k server.key -Currently, The EdgeX snap has security (Secret Store and API Gateway) enabled by default. The security services constitute the following components: +# To set the certificate again, you first need to clear the current values by setting them to an empty string: +sudo snap set edgexfoundry env.security-proxy.tls-certificate="" +sudo snap set edgexfoundry env.security-proxy.tls-private-key="" -- kong-daemon (API Gateway a.k.a. Reverse Proxy) -- postgres (kong's database) -- vault (Secret Store) +# set tls-sni +sudo snap set edgexfoundry env.security-proxy.tls-sni="server01" -Oneshot services which perform the necessary security setup and stop, when listed using `snap services`, they show up as `enabled/inactive`: +# and then provide the certificate and key +sudo snap set edgexfoundry env.security-proxy.tls-certificate="$(cat server.cert)" +sudo snap set edgexfoundry env.security-proxy.tls-private-key="$(cat server.key)" -- security-proxy-setup (kong setup) -- security-secretstore-setup (vault setup) -- security-bootstrapper-redis (secure redis setup) -- security-consul-bootstrapper (secure consul setup) +# connect +curl -v --cacert /var/snap/edgeca/current/CA.pem -X GET https://server01:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" +``` -Vault is known within EdgeX as the Secret Store, while Kong+PostgreSQL are used to provide the EdgeX API Gateway. -For more details please refer to the snap's [Secret Store](https://github.com/edgexfoundry/edgex-go/blob/main/snap/README.md#secret-store) and [API Gateway](https://github.com/edgexfoundry/edgex-go/blob/main/snap/README.md#api-gateway) documentation. +#### Disabling security +!!! Warning + Disabling security is NOT recommended, unless for demonstration purposes, or when there are other means to secure the services. +!!! Warning + The snap will NOT allow the Secret Store to be re-enabled. The only way to re-enable the Secret Store is to re-install the snap. + +The Secret Store is used by EdgeX for secret management (e.g. certificates, keys, passwords). Use of the Secret Store by all services can be disabled globally, but doing so will also disable the API Gateway, as it depends on the Secret Store. + +Thus the following command will disable both: + +```bash +sudo snap set edgexfoundry security-secret-store=off +``` + +All services in the snap except for the API Gateway are restricted by default to listening on localhost (127.0.0.1). +The API Gateway proxies external requests to internal services. +Since disabling the Secret Store also disables the API Gateway, the service endpoint will no longer be accessible from other systems. +They will be still accessible on the local machine for demonstration and testing. + +If you really need to make an insecure service accessible remotely, the `Service.ServerBindAddr` of each service needs be changed to the IP address of that networking interface on the local machine. If you trust all your interfaces and want the services to accept connections from all, set it to `0.0.0.0`. + +### EdgeX UI +[![Get it from the Snap Store][badge]][edgex-ui] + +The EdgeX GUI snap is a development tool called to help you get started with EdgeX, +whether you've deployed other components natively, using Docker containers, or +with snaps. For installation instructions, refer to [edgex-ui]. + +- Configuration + - Snap + - Snap+docker +- Additional edgexfoundry snap config for UI + +For usage instructions, please refer to the [Graphical User Interface (GUI)](../../getting-started/tools/Ch-GUI/) guide. + +### EdgeX CLI +[![Get it from the Snap Store][badge]](https://snapcraft.io/edgex-cli) + +[edgex-cli](https://snapcraft.io/edgex-cli) + + +### App Service Configurable +[edgex-app-service-configurable](https://snapcraft.io/edgex-app-service-configurable) + +### App RFID LLRP Inventory +### Device Camera +[edgex-device-camera](https://snapcraft.io/edgex-device-camera) +### Device GPIO +[edgex-device-gpio](https://snapcraft.io/edgex-device-gpio) +### Device Grove +[edgex-device-grove](https://snapcraft.io/edgex-device-grove) +### Device Modbus +[edgex-device-modbus](https://snapcraft.io/edgex-device-modbus) +### Device MQTT +[edgex-device-mqtt](https://snapcraft.io/edgex-device-mqtt) +### Device REST +[edgex-device-rest](https://snapcraft.io/edgex-device-rest) +### Device RFID LLRP +[](https://snapcraft.io/edgex-device-rfid-llrp) +### Device SNMP +[edgex-device-snmp](https://snapcraft.io/edgex-device-snmp) + + + + + + + + + +[badge]: https://snapcraft.io/static/images/badges/en/snap-store-white.svg +[edgexfoundry]: https://snapcraft.io/edgexfoundry +[edgex-ui]: https://snapcraft.io/edgex-ui +[edgex-cli]: https://snapcraft.io/edgex-cli +[edgex-app-service-configurable]: https://snapcraft.io/edgex-app-service-configurable +[edgex-device-camera]: https://snapcraft.io/edgex-device-camera +[edgex-device-gpio]: https://snapcraft.io/edgex-device-gpio +[edgex-device-grove]: https://snapcraft.io/edgex-device-grove +[edgex-device-modbus]: https://snapcraft.io/edgex-device-modbus +[edgex-device-mqtt]: https://snapcraft.io/edgex-device-mqtt +[edgex-device-rest]: https://snapcraft.io/edgex-device-rest +[edgex-device-rfid-llrp]: https://snapcraft.io/edgex-device-rfid-llrp +[edgex-device-snmp]: https://snapcraft.io/edgex-device-snmp \ No newline at end of file diff --git a/docs_src/microservices/configuration/snap/Configuration.md b/docs_src/microservices/configuration/snap/Configuration.md index 50ee483d85..82c9d0ca85 100644 --- a/docs_src/microservices/configuration/snap/Configuration.md +++ b/docs_src/microservices/configuration/snap/Configuration.md @@ -1,11 +1,11 @@ # Configuration -## Changing config files +## Change config files - how to locally modify files - link to config provider page -## Override configurations +## Override configurations using Snap options - Environment variable injection - Link to consul doc page diff --git a/docs_src/microservices/configuration/snap/ContentInterfaces.md b/docs_src/microservices/configuration/snap/ContentInterfaces.md index d878b15a02..42bf032f33 100644 --- a/docs_src/microservices/configuration/snap/ContentInterfaces.md +++ b/docs_src/microservices/configuration/snap/ContentInterfaces.md @@ -1,5 +1,5 @@ # Content Interfaces -## Secret interface +## Secret provider -## Config interface +## Config provider From a1e38b382c757c7978881dd9dcf2de75c786c2fb Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 29 Apr 2022 18:43:45 +0200 Subject: [PATCH 03/29] feat: add config options instructions Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 128 ++++++++++++++---- 1 file changed, 101 insertions(+), 27 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index c6383419ef..b88bfc9aa2 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -19,8 +19,69 @@ or development release, refer to the store page for the snap, choose install, an then pick the desired channel. The store page also provides instructions for installation on different Linux distributions as well as the list of supported CPU architectures. -Snap packages of EdgeX services are published on the [Snap Store](https://snapcraft.io). -The list of all EdgeX snaps is available [below](#edgex-snaps). +For the list of EdgeX snaps, please refer [here](#edgex-snaps). + +### Configuration +EdgeX snaps are packaged with default service configuration files. In certain cases, few configuration fields are overridden within the snap for snap-specific deployment requirements. + +#### Configuration files +The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to Consul by default. The configuration file can be modified and applied only if the modifications happen before initial startup. + +#### Configuration registry +The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations are loaded by the service at startup, except for the writable settings which are loaded at runtime. Please refer to the documentation of microservices for details. + +#### Configuration provider snaps +Most EdgeX snaps have a [content interface](https://snapcraft.io/docs/content-interface) which allows another snap to seed the snap with configuration files. +This is useful when replacing entire configuration files via another snap, packaged with the deployment-specific configurations. + +Please refer to [edgex-config-provider](https://github.com/canonical/edgex-config-provider), for an example. + +#### Configuration overrides +!!! edgey "EdgeX 2.2" + The snaps now provide an interface to set any environment variable for supported services. + We call these the *config options* because they use a `config` prefix for the variable names. + The config options are **disabled by default**. + + This functionality supersedes for the snap *env options* (with `env.` prefix) which allows setting certain configurations. Please refer to EdgeX 2.1 (Jakarta) snap READMEs and documentation for details on the deprecated options. + + The env options are deprecated and have incomplete configuration coverage. Existing options will continue to work until the next major EdgeX release. At that point, the config options will become **enabled by default**. + +The EdgeX services allow overriding server configurations using environment variables. Moreover, the services read EdgeX [Common Environment Variables](../../microservices/configuration/CommonEnvironmentVariables/) to change configurations that aren't defined in config files. +The EdgeX snaps provide an interface via [snap configuration options](https://snapcraft.io/docs/configuration-in-snaps) to set environment variables for the services. +We call these the *config options* because they a have `config` prefix for the variable names. + +The snap options for setting environment variable uses the the following format: + +* `apps..config.`: setting an app-specific value (e.g. `apps.core-data.config.service-port=1000`). +* `config.`: setting a global value (e.g. `config.service-host=localhost` or `config.writable-loglevel=DEBUG` + +where: + +* `` is the name of the app (service, executable) +* `` is a lowercase, dash-separated mapping to uppercase, underscore-separate environment variable name (e.g. `x-y`->`X_Y`). The reason for such mapping is that uppercase and underscore characters are not supported as config keys for snaps. + +Mapping examples: + +| Snap config key | Environment Variable | Service configuration TOML | +|------------------------|--------------------------|-----------------------------------------------------| +| service-port | SERVICE_PORT | [Service]
Port | +| clients-core-data-host | CLIENTS_CORE_DATA_HOST | [Clients]
--[Clients.core-data]
--Host | +| edgex-startup-duration | [EDGEX_STARTUP_DURATION] | - | +| add-secretstore-tokens | [ADD_SECRETSTORE_TOKENS] | - | + +[EDGEX_STARTUP_DURATION]: ../../microservices/configuration/CommonEnvironmentVariables/#edgex_startup_duration +[ADD_SECRETSTORE_TOKENS]: ../../security/Ch-Configuring-Add-On-Services/#configure-the-services-secret-store-to-use + +!!! Note + The config options are supported as of EdgeX 2.2 and are disabled by default! + + Setting `config-enabled=true` is necessary to enable their support. + +For example, to change the service port of the core-data service on `edgexfoundry` snap to 8080: +```bash +snap set config-enabled=true +snap set edgexfoundry apps.core-data.service-port=8080 +``` ### Control services The services of a snap can be started/stopped/restarted using the snap CLI. @@ -28,33 +89,36 @@ When starting/stopping, you can additionally set them to enable/disable which co To list the services and check their status: ```bash -snap services +snap services ``` To start and enable services: ```bash # all services -snap start --enable +snap start --enable + # one service -snap start --enable . +snap start --enable . ``` -For controlling the default service startup from a gadget snap, see [TBA](tba). + ### Debugging The service logs can be queried using the `snap log` command. For example, to query 100 lines and follow: ```bash -snap logs -n=100 -f +# all services +snap logs -n=100 -f + +# one service +snap logs -n=100 -f . ``` Check `snap logs --help` for details. -To query not only the service logs, but also the installation logs (from various hooks), use `journalctl`. - -For example, to query 100 lines, follow, and filter: +To query not only the service logs, but also the snap logs (incl. hook apps such as install and configure), use `journalctl`: ```bash -journalctl -n 100 -f | grep +sudo journalctl -n 100 -f | grep ``` ## EdgeX Snaps @@ -87,12 +151,15 @@ Upon installation, the following EdgeX services are automatically and immediatel The following services are disabled by default: -- app-service-configurable (used to filter events for kuiper) -- device-virtual -- kuiper (Rules Engine) - support-notifications - support-scheduler -- sys-mgmt-agent +- sys-mgmt-agent - deprecated +- device-virtual +- kuiper (Rules Engine) - deprecated; see +- app-service-configurable (used to filter events for kuiper) + + + The disabled services can be manually enabled and started; see [Control services](#control-services). @@ -306,6 +373,9 @@ curl -v --cacert /var/snap/edgeca/current/CA.pem -X GET https://server01:8443/co #### Disabling security +!!! TODO: + MOVE TO COMMON SECTION AND EXTEND WITH SERVICE ENV TO DISABLE + !!! Warning Disabling security is NOT recommended, unless for demonstration purposes, or when there are other means to secure the services. !!! Warning @@ -343,43 +413,46 @@ For usage instructions, please refer to the [Graphical User Interface (GUI)](../ ### EdgeX CLI [![Get it from the Snap Store][badge]](https://snapcraft.io/edgex-cli) -[edgex-cli](https://snapcraft.io/edgex-cli) +[edgex-cli] ### App Service Configurable -[edgex-app-service-configurable](https://snapcraft.io/edgex-app-service-configurable) +[edgex-app-service-configurable] ### App RFID LLRP Inventory +[edgex-app-rfid-llrp-inventory] + ### Device Camera -[edgex-device-camera](https://snapcraft.io/edgex-device-camera) +[edgex-device-camera] ### Device GPIO -[edgex-device-gpio](https://snapcraft.io/edgex-device-gpio) +[edgex-device-gpio] ### Device Grove -[edgex-device-grove](https://snapcraft.io/edgex-device-grove) +[edgex-device-grove] ### Device Modbus -[edgex-device-modbus](https://snapcraft.io/edgex-device-modbus) +[edgex-device-modbus] ### Device MQTT -[edgex-device-mqtt](https://snapcraft.io/edgex-device-mqtt) +[edgex-device-mqtt] ### Device REST -[edgex-device-rest](https://snapcraft.io/edgex-device-rest) +[edgex-device-rest] ### Device RFID LLRP [](https://snapcraft.io/edgex-device-rfid-llrp) ### Device SNMP [edgex-device-snmp](https://snapcraft.io/edgex-device-snmp) +### eKuiper +[edgex-ekuiper] - - - + [badge]: https://snapcraft.io/static/images/badges/en/snap-store-white.svg [edgexfoundry]: https://snapcraft.io/edgexfoundry [edgex-ui]: https://snapcraft.io/edgex-ui [edgex-cli]: https://snapcraft.io/edgex-cli [edgex-app-service-configurable]: https://snapcraft.io/edgex-app-service-configurable +[edgex-app-rfid-llrp-inventory]: https://snapcraft.io/edgex-app-rfid-llrp-inventory [edgex-device-camera]: https://snapcraft.io/edgex-device-camera [edgex-device-gpio]: https://snapcraft.io/edgex-device-gpio [edgex-device-grove]: https://snapcraft.io/edgex-device-grove @@ -387,4 +460,5 @@ For usage instructions, please refer to the [Graphical User Interface (GUI)](../ [edgex-device-mqtt]: https://snapcraft.io/edgex-device-mqtt [edgex-device-rest]: https://snapcraft.io/edgex-device-rest [edgex-device-rfid-llrp]: https://snapcraft.io/edgex-device-rfid-llrp -[edgex-device-snmp]: https://snapcraft.io/edgex-device-snmp \ No newline at end of file +[edgex-device-snmp]: https://snapcraft.io/edgex-device-snmp +[edgex-ekuiper]: https://snapcraft.io/edgex-ekuiper \ No newline at end of file From 557b98d9e5bbe1a24bb6a3dd704d18d1b7b3ccae Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Sun, 1 May 2022 23:53:50 +0200 Subject: [PATCH 04/29] feat: Config and edgexfoundry snap updates Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 64 +++++++++++++------ 1 file changed, 45 insertions(+), 19 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index b88bfc9aa2..6647893b2e 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -24,11 +24,16 @@ For the list of EdgeX snaps, please refer [here](#edgex-snaps). ### Configuration EdgeX snaps are packaged with default service configuration files. In certain cases, few configuration fields are overridden within the snap for snap-specific deployment requirements. +Changes made to configurations services to be restarted in order for the changes to take effect; +the one exception are changes made to configuration items in a service's `[Writable]` section. +Services that aren't started by default (see [Using the EdgeX snap](#using-the-edgex-snap) section above) +*will* pickup any changes made to their config files when started. + #### Configuration files -The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to Consul by default. The configuration file can be modified and applied only if the modifications happen before initial startup. +The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to Consul by default. Once the service starts without errors, the local configurations become obsolete and will no longer be read. Any modifications after the initial startup will not be applied. #### Configuration registry -The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations are loaded by the service at startup, except for the writable settings which are loaded at runtime. Please refer to the documentation of microservices for details. +The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations in Consul are loaded by the service at startup. If the service has already started, a restart is required to load new configurations. Configurations that are in the writable section get loaded not only at startup, but also during the runtime. In other words, changes to the writable configurations are loaded automatically without a restart. Please refer to the documentation of microservices for details. #### Configuration provider snaps Most EdgeX snaps have a [content interface](https://snapcraft.io/docs/content-interface) which allows another snap to seed the snap with configuration files. @@ -83,6 +88,8 @@ snap set config-enabled=true snap set edgexfoundry apps.core-data.service-port=8080 ``` +The services load the set config options on startup. If the service has already started, a restart is necessary to load them. + ### Control services The services of a snap can be started/stopped/restarted using the snap CLI. When starting/stopping, you can additionally set them to enable/disable which configures whether or not the service should also start on boot. @@ -101,6 +108,15 @@ snap start --enable snap start --enable . ``` +To restart services, e.g. to load the configurations: +```bash +# all services +snap restart + +# one service +snap restart . +``` + ### Debugging @@ -140,7 +156,7 @@ Upon installation, the following EdgeX services are automatically and immediatel - core-command - core-data - core-metadata -- kong-daemon (API Gateway a.k.a. Reverse Proxy) +- kong-daemon (API Gateway / Reverse Proxy) - postgres (kong's database) - redis (default Message Bus and database backend for core-data and core-metadata) - security-bootstrapper-redis (oneshot service) @@ -153,17 +169,17 @@ The following services are disabled by default: - support-notifications - support-scheduler -- sys-mgmt-agent - deprecated +- sys-mgmt-agent - *deprecated EdgeX component* - device-virtual -- kuiper (Rules Engine) - deprecated; see -- app-service-configurable (used to filter events for kuiper) - - +- kuiper (Rules Engine / eKuiper) - *deprecated; use the standalone [EdgeX eKuiper snap](#edgex-ekuiper)* +- app-service-configurable (used to filter events for kuiper) - *deprecated; use the standalone [App Service Configurable snap](#app-service-configurable)* The disabled services can be manually enabled and started; see [Control services](#control-services). -#### Configuring individual services +For the configuration of services, refer to [Configurations](#configuration) + + -#### Configuration Overrides + #### Security services Currently, The EdgeX snap has security (Secret Store and API Gateway) enabled by default. The security services constitute the following components: -- kong-daemon (API Gateway a.k.a. Reverse Proxy) +- kong-daemon (API Gateway / Reverse Proxy) - postgres (kong's database) - vault (Secret Store) @@ -232,7 +248,7 @@ Oneshot services which perform the necessary security setup and stop, when liste Vault is known within EdgeX as the Secret Store, while Kong+PostgreSQL are used to provide the EdgeX API Gateway. -For more details please refer to the snap's [Secret Store](https://github.com/edgexfoundry/edgex-go/blob/main/snap/README.md#secret-store) and [API Gateway](https://github.com/edgexfoundry/edgex-go/blob/main/snap/README.md#api-gateway) documentation. + @@ -240,17 +256,27 @@ For more details please refer to the snap's [Secret Store](https://github.com/ed Before the API Gateway can be used, a user and group must be created and a JWT access token generated. -1. The first step is to create a public/private keypair for the new user, which can be done with +1. Create a public/private keypair for the new user +Using openssl: ```bash -# Create private key: +# Create private key openssl ecparam -genkey -name prime256v1 -noout -out private.pem -# Create public key: +# Create public key openssl ec -in private.pem -pubout -out public.pem ``` -2. The next step is to create the user. The easiest way to create a single API gateway user is to use `snap set` to set two values as follows: +2. Add the user + +The snap provides a way to set the public key of a single user as an admin. +The admin user has the following hardcoded settings: +- username: admin +- user id: 1 +- algorithm: ES256 + + + To create multiple users, use the secrets-config command. You need to provide the following: From a65d869c8c7a111c7bc2d8a08b458f3429309726 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Mon, 2 May 2022 15:51:28 +0200 Subject: [PATCH 05/29] feat: update security config Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 327 ++++++------------ 1 file changed, 113 insertions(+), 214 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 6647893b2e..26dc35da09 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -7,6 +7,8 @@ secure, cross‐platform and self-contained. Snaps can be installed on any Linux distribution with [snap support](https://snapcraft.io/docs/installing-snapd). ### Installation +[installation]: #installation + When using the snap CLI, the installation is possible by simply executing: ```bash snap install @@ -22,6 +24,8 @@ The store page also provides instructions for installation on different Linux di For the list of EdgeX snaps, please refer [here](#edgex-snaps). ### Configuration +[configuration]: #configuration + EdgeX snaps are packaged with default service configuration files. In certain cases, few configuration fields are overridden within the snap for snap-specific deployment requirements. Changes made to configurations services to be restarted in order for the changes to take effect; @@ -29,19 +33,19 @@ the one exception are changes made to configuration items in a service's `[Writa Services that aren't started by default (see [Using the EdgeX snap](#using-the-edgex-snap) section above) *will* pickup any changes made to their config files when started. -#### Configuration files +#### Config files The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to Consul by default. Once the service starts without errors, the local configurations become obsolete and will no longer be read. Any modifications after the initial startup will not be applied. -#### Configuration registry +#### Config registry The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations in Consul are loaded by the service at startup. If the service has already started, a restart is required to load new configurations. Configurations that are in the writable section get loaded not only at startup, but also during the runtime. In other words, changes to the writable configurations are loaded automatically without a restart. Please refer to the documentation of microservices for details. -#### Configuration provider snaps +#### Config provider snap Most EdgeX snaps have a [content interface](https://snapcraft.io/docs/content-interface) which allows another snap to seed the snap with configuration files. This is useful when replacing entire configuration files via another snap, packaged with the deployment-specific configurations. Please refer to [edgex-config-provider](https://github.com/canonical/edgex-config-provider), for an example. -#### Configuration overrides +#### Config overrides !!! edgey "EdgeX 2.2" The snaps now provide an interface to set any environment variable for supported services. We call these the *config options* because they use a `config` prefix for the variable names. @@ -90,7 +94,32 @@ snap set edgexfoundry apps.core-data.service-port=8080 The services load the set config options on startup. If the service has already started, a restart is necessary to load them. -### Control services + +#### Disabling security +!!! Warning + Disabling security is NOT recommended, unless for demonstration purposes, or when there are other means to secure the services. + + The snap will NOT allow the Secret Store to be re-enabled. The only way to re-enable the Secret Store is to re-install the snap. + +The Secret Store is used by EdgeX for secret management (e.g. certificates, keys, passwords). Use of the Secret Store by all services can be disabled globally. Note that doing so will also disable the API Gateway, as it depends on the Secret Store. + +The following command disables the Secret Store and in turn the API Gateway: +```bash +sudo snap set edgexfoundry security-secret-store=off +``` + +All services in the snap except for the API Gateway are restricted by default to listening on localhost (127.0.0.1). +The API Gateway proxies external requests to internal services. +Since disabling the Secret Store also disables the API Gateway, the service endpoint will no longer be accessible from other systems. +They will be still accessible on the local machine for demonstration and testing. + +If you really need to make an insecure service accessible remotely, the Service ServerBindAddr of each service needs be changed to the IP address of that networking interface on the local machine. If you trust all your interfaces and want the services to accept connections from all, set it to `0.0.0.0`. + +After disabling the Secret Store, the external services should be configured such that they don't attempt to initialize the security. For this purpose, [EDGEX_SECURITY_SECRET_STORE](../../microservices/configuration/CommonEnvironmentVariables/#edgex_security_secret_store) global environment variable should be set to false, using the corresponding snap option: `config.edgex-security-secret-store`. + +### Managing services +[managing services]: #managing-services + The services of a snap can be started/stopped/restarted using the snap CLI. When starting/stopping, you can additionally set them to enable/disable which configures whether or not the service should also start on boot. @@ -99,7 +128,7 @@ To list the services and check their status: snap services ``` -To start and enable services: +To start and optionally enable services: ```bash # all services snap start --enable @@ -108,6 +137,26 @@ snap start --enable snap start --enable . ``` +Similarly, a service can be stopped and optionally disabled using `snap stop --disable`. + +!!! tip "Snap options" + To spin up an EdgeX instance with a different startup configuration (e.g. enabled instead of disabled), the `edgexfoundry` snap provides the following config options that accept values `"on"`/`"off"` to enable/disable a service by default: + + * `consul` + * `redis` + * `core-metadata` + * `core-command` + * `core-data` + * `support-notifications` + * `support-scheduler` + * `device-virtual` + * `security-secret-store` + * `security-proxy` + + Device and app service snaps provide a similar functionality using the `auto-start` option. + + This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. + To restart services, e.g. to load the configurations: ```bash # all services @@ -117,9 +166,9 @@ snap restart snap restart . ``` - - ### Debugging +[debugging]: #debugging + The service logs can be queried using the `snap log` command. For example, to query 100 lines and follow: @@ -148,9 +197,11 @@ The main platform snap, simply called [edgexfoundry] contains all reference core services along with several other security, supporting, application, and device services. +Please refer to common sections above for [installation], [configuration], [managing services], and [debugging]. + #### Services -Upon installation, the following EdgeX services are automatically and immediately started: +Upon installation, the following EdgeX services are automatically started: - consul (Registry) - core-command @@ -159,10 +210,10 @@ Upon installation, the following EdgeX services are automatically and immediatel - kong-daemon (API Gateway / Reverse Proxy) - postgres (kong's database) - redis (default Message Bus and database backend for core-data and core-metadata) -- security-bootstrapper-redis (oneshot service) -- security-consul-bootstrapper (oneshot service) -- security-proxy-setup (oneshot service) -- security-secretstore-setup (oneshot service) +- security-bootstrapper-redis (oneshot service to setup secure Redis) +- security-consul-bootstrapper (oneshot service to setup secure Consul) +- security-proxy-setup (oneshot service to setup Kong) +- security-secretstore-setup (oneshot service to setup Vault) - vault (Secret Store) The following services are disabled by default: @@ -175,169 +226,45 @@ The following services are disabled by default: - app-service-configurable (used to filter events for kuiper) - *deprecated; use the standalone [App Service Configurable snap](#app-service-configurable)* -The disabled services can be manually enabled and started; see [Control services](#control-services). - -For the configuration of services, refer to [Configurations](#configuration) - - - - - - - -#### Security services - -Currently, The EdgeX snap has security (Secret Store and API Gateway) enabled by default. The security services constitute the following components: - -- kong-daemon (API Gateway / Reverse Proxy) -- postgres (kong's database) -- vault (Secret Store) - -Oneshot services which perform the necessary security setup and stop, when listed using `snap services`, they show up as `enabled/inactive`: - -- security-proxy-setup (kong setup) -- security-secretstore-setup (vault setup) -- security-bootstrapper-redis (secure redis setup) -- security-consul-bootstrapper (secure consul setup) - -Vault is known within EdgeX as the Secret Store, while Kong+PostgreSQL are used to provide the EdgeX API Gateway. - - - - - -#### API Gateway user setup - -Before the API Gateway can be used, a user and group must be created and a JWT access token generated. - -1. Create a public/private keypair for the new user - -Using openssl: +Example: ```bash # Create private key openssl ecparam -genkey -name prime256v1 -noout -out private.pem # Create public key openssl ec -in private.pem -pubout -out public.pem -``` - -2. Add the user - -The snap provides a way to set the public key of a single user as an admin. -The admin user has the following hardcoded settings: -- username: admin -- user id: 1 -- algorithm: ES256 - - - - -To create multiple users, use the secrets-config command. You need to provide the following: - -- The username -- The public key -- The API Gateway Admin JWT token -- (optionally) ID. This is a unique string identifying the credential. It will be required in the next step to -create the JWT token. If you don't specify it, -then an autogenerated one will be output by the secrets-config command -```bash - -# get API Gateway/Kong token -JWT_FILE=/var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt -JWT=`sudo cat ${JWT_FILE}` +# read the API Gateway token +KONG_ADMIN_JWT=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` # use secrets-config to add user -edgexfoundry.secrets-config proxy adduser --token-type jwt --user user01 --algorithm ES256 --public_key public.pem --id USER_ID --jwt ${JWT} -``` +# on success, this command with print the user id +edgexfoundry.secrets-config proxy adduser --token-type jwt --user --algorithm ES256 --public_key public.pem --id --jwt $KONG_ADMIN_JWT -3. Finally, you need to generate a token using the user ID which you specified: +# get a JWT token for this user +TOKEN=`edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id --expiration=1h` -```bash -# get token -TOKEN=`edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id USER_ID --expiration=1h` - -# Keep this token in a safe place for future reuse as the same token cannot be regenerated or recovered using the secret-config CLI -echo $TOKEN +# keep this token in a safe place for future reuse +echo $TOKEN > token.jwt ``` -Alternatively , you can generate the token on a different device using a bash script: - -```bash -header='{ - "alg": "ES256", - "typ": "JWT" -}' - -TTL=$((EPOCHSECONDS+3600)) - -payload='{ - "iss":"USER_ID", - "iat":'$EPOCHSECONDS', - "nbf":'$EPOCHSECONDS', - "exp":'$TTL' -}' - -JWT_HEADER=`echo -n $header | openssl base64 -e -A | sed s/\+/-/ | sed -E s/=+$//` -JWT_PAYLOAD=`echo -n $payload | openssl base64 -e -A | sed s/\+/-/ | sed -E s/=+$//` -JWT_SIGNATURE=`echo -n "$JWT_HEADER.$JWT_PAYLOAD" | openssl dgst -sha256 -binary -sign private.pem | openssl asn1parse -inform DER -offset 2 | grep -o "[0-9A-F]\+$" | tr -d '\n' | xxd -r -p | base64 -w0 | tr -d '=' | tr '+/' '-_'` -TOKEN=$JWT_HEADER.$JWT_PAYLOAD.$JWT_SIGNATURE -``` - -4. Once you have the token you can access the API Gateway as follows: +Once you have the token you can access the API Gateway as follows: The JWT token must be included via an HTTP `Authorization: Bearer ` header on any REST calls used to access EdgeX services via the API Gateway. @@ -345,82 +272,58 @@ via an HTTP `Authorization: Bearer ` header on any REST calls used Example: ```bash -curl -k -X GET https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" +$ curl --insecure https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" +{"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"} ``` +!!! tip "Snap options" + To spin up a pre-configured and securely accessible EdgeX instance, the snap provides a way to pass the public key of a single user with snap options. When requested, the user is created with user `admin`, id `1` and JWT signing algorithm `ES256`. The snap option for passing the public key is: + `apps.secrets-config.proxy.admin.public-key`. -#### API Gateway TLS certificate setup - -By default Kong is configured with a self-signed TLS certificate (which you find in `/var/snap/edgexfoundry/current/kong/ssl/kong-default-ecdsa.crt`). -It is also possible to install your own TLS certificate to be used by the gateway. The steps to do so are as follows: + This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. -1. Start by provisioning a TLS certificate to use. You can use a number of tools for that, such as `openssl` or the `edgeca` snap: +##### Consul +Consul API and UI can be accessed using the consul secret id. For the snap, secret is the value of `SecretID` typically placed in a JSON file at `/var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json`. +For example, to get the secret using JQ: ```bash -sudo snap install edgeca -edgeca gencsr --cn localhost --csr csrfile --key csrkeyfile -edgeca gencert -o localhost.cert -i csrfile -k localhost.key +$ sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json | jq '.SecretID' +"ee3964d0-505f-6b62-4c88-0d29a8226daa" ``` -2. Then install the certificate: -```bash -sudo snap set edgexfoundry env.security-proxy.tls-certificate="$(cat localhost.cert)" -sudo snap set edgexfoundry env.security-proxy.tls-private-key="$(cat localhost.key)" -``` - -3. Specify the EdgeCA Root CA certificate with `--cacert` for validation of the new certificate: +##### Custom TLS certificates +The API Gateway setup generates a self-signed certificate by default. To replace that with your own certificate, refer to API Gateway guide: [Using a bring-your-own external TLS certificate for API gateway](../../security/Ch-APIGateway/#using-a-bring-your-own-external-tls-certificate-for-api-gateway) and use the snapped `edgexfoundry.secrets-config` utility. +To get the usage help: ```bash -curl -v --cacert /var/snap/edgeca/current/CA.pem -X GET https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" +edgexfoundry.secrets-config proxy tls -h ``` -Optionally, to specify a server name other than `localhost`, set the `tls-sni` configuration setting first. Example: - +Example: Given certificate `cert.pem`, private key `privkey.pem`, and certificate authority `ca.pem` files: ```bash -# generate certificate and private key -edgeca gencsr --cn server01 --csr csrfile --key csrkeyfile -edgeca gencert -o server.cert -i csrfile -k server.key - -# To set the certificate again, you first need to clear the current values by setting them to an empty string: -sudo snap set edgexfoundry env.security-proxy.tls-certificate="" -sudo snap set edgexfoundry env.security-proxy.tls-private-key="" - -# set tls-sni -sudo snap set edgexfoundry env.security-proxy.tls-sni="server01" - -# and then provide the certificate and key -sudo snap set edgexfoundry env.security-proxy.tls-certificate="$(cat server.cert)" -sudo snap set edgexfoundry env.security-proxy.tls-private-key="$(cat server.key)" +# read the API Gateway token +KONG_ADMIN_JWT=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` -# connect -curl -v --cacert /var/snap/edgeca/current/CA.pem -X GET https://server01:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" +# add the certificate +edgexfoundry.secrets-config proxy tls --incert /path/to/cert.pem --inkey /path/to/privkey.pem --admin_api_jwt $KONG_ADMIN_JWT ``` - -#### Disabling security -!!! TODO: - MOVE TO COMMON SECTION AND EXTEND WITH SERVICE ENV TO DISABLE - -!!! Warning - Disabling security is NOT recommended, unless for demonstration purposes, or when there are other means to secure the services. -!!! Warning - The snap will NOT allow the Secret Store to be re-enabled. The only way to re-enable the Secret Store is to re-install the snap. - -The Secret Store is used by EdgeX for secret management (e.g. certificates, keys, passwords). Use of the Secret Store by all services can be disabled globally, but doing so will also disable the API Gateway, as it depends on the Secret Store. - -Thus the following command will disable both: - +Try it out: ```bash -sudo snap set edgexfoundry security-secret-store=off +$ curl -v --cacert /path/to/ca.pem https://server01:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" +{"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"} ``` -All services in the snap except for the API Gateway are restricted by default to listening on localhost (127.0.0.1). -The API Gateway proxies external requests to internal services. -Since disabling the Secret Store also disables the API Gateway, the service endpoint will no longer be accessible from other systems. -They will be still accessible on the local machine for demonstration and testing. +!!! tip "Snap options" + To spin up an EdgeX instance with custom certificates, the snap provides the following configuration options: + + * `apps.secrets-config.proxy.tls.cert` + * `apps.secrets-config.proxy.tls.key` + * `apps.secrets-config.proxy.tls.snis` (comma-separated values) + + This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. -If you really need to make an insecure service accessible remotely, the `Service.ServerBindAddr` of each service needs be changed to the IP address of that networking interface on the local machine. If you trust all your interfaces and want the services to accept connections from all, set it to `0.0.0.0`. ### EdgeX UI [![Get it from the Snap Store][badge]][edgex-ui] @@ -468,10 +371,6 @@ For usage instructions, please refer to the [Graphical User Interface (GUI)](../ [edgex-ekuiper] - - - - [badge]: https://snapcraft.io/static/images/badges/en/snap-store-white.svg [edgexfoundry]: https://snapcraft.io/edgexfoundry From 08afef9908cd4e5399b9da6dcef637e4d6e9ec1e Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Mon, 2 May 2022 19:20:48 +0200 Subject: [PATCH 06/29] feat: simplify sections Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 80 ++++++++++--------- 1 file changed, 43 insertions(+), 37 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 26dc35da09..e8f463c0cc 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -11,14 +11,12 @@ Snaps can be installed on any Linux distribution with [snap support](https://sna When using the snap CLI, the installation is possible by simply executing: ```bash -snap install +snap install ``` This is similar to setting `--channel=latest/stable` or shorthand `--stable` and will install the latest stable release of a snap. In this case, `latest/stable` is the [channel](https://snapcraft.io/docs/channels), composed of `latest` track and `stable` risk level. -To install a specific version with long term support (LTS), or to install a beta -or development release, refer to the store page for the snap, choose install, and -then pick the desired channel. +To install a specific version with long term support (e.g. 2.1), or to install a beta or development release, refer to the store page for the snap, choose install, and then pick the desired channel. The store page also provides instructions for installation on different Linux distributions as well as the list of supported CPU architectures. For the list of EdgeX snaps, please refer [here](#edgex-snaps). @@ -28,16 +26,14 @@ For the list of EdgeX snaps, please refer [here](#edgex-snaps). EdgeX snaps are packaged with default service configuration files. In certain cases, few configuration fields are overridden within the snap for snap-specific deployment requirements. -Changes made to configurations services to be restarted in order for the changes to take effect; -the one exception are changes made to configuration items in a service's `[Writable]` section. -Services that aren't started by default (see [Using the EdgeX snap](#using-the-edgex-snap) section above) -*will* pickup any changes made to their config files when started. #### Config files The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to Consul by default. Once the service starts without errors, the local configurations become obsolete and will no longer be read. Any modifications after the initial startup will not be applied. #### Config registry -The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations in Consul are loaded by the service at startup. If the service has already started, a restart is required to load new configurations. Configurations that are in the writable section get loaded not only at startup, but also during the runtime. In other words, changes to the writable configurations are loaded automatically without a restart. Please refer to the documentation of microservices for details. +The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations in Consul are loaded by the service at startup. If the service has already started, a restart is required to load new configurations. Configurations that are in the writable section get loaded not only at startup, but also during the runtime. In other words, changes to the writable configurations are loaded automatically without a restart. Please refer to +[Common Configuration](/microservices/configuration/CommonConfiguration/) and +[Configuration and Registry Providers](../../microservices/configuration/ConfigurationAndRegistry/) for more information. #### Config provider snap Most EdgeX snaps have a [content interface](https://snapcraft.io/docs/content-interface) which allows another snap to seed the snap with configuration files. @@ -89,7 +85,7 @@ Mapping examples: For example, to change the service port of the core-data service on `edgexfoundry` snap to 8080: ```bash snap set config-enabled=true -snap set edgexfoundry apps.core-data.service-port=8080 +snap set edgexfoundry apps.core-data.config.service-port=8080 ``` The services load the set config options on startup. If the service has already started, a restart is necessary to load them. @@ -113,9 +109,9 @@ The API Gateway proxies external requests to internal services. Since disabling the Secret Store also disables the API Gateway, the service endpoint will no longer be accessible from other systems. They will be still accessible on the local machine for demonstration and testing. -If you really need to make an insecure service accessible remotely, the Service ServerBindAddr of each service needs be changed to the IP address of that networking interface on the local machine. If you trust all your interfaces and want the services to accept connections from all, set it to `0.0.0.0`. +If you really need to make an insecure service accessible remotely, set the bind address of the service to the IP address of that networking interface on the local machine. If you trust all your interfaces and want the services to accept connections from all, set it to `0.0.0.0`. -After disabling the Secret Store, the external services should be configured such that they don't attempt to initialize the security. For this purpose, [EDGEX_SECURITY_SECRET_STORE](../../microservices/configuration/CommonEnvironmentVariables/#edgex_security_secret_store) global environment variable should be set to false, using the corresponding snap option: `config.edgex-security-secret-store`. +After disabling the Secret Store, the external services should be configured such that they don't attempt to initialize the security. For this purpose, [EDGEX_SECURITY_SECRET_STORE](../../microservices/configuration/CommonEnvironmentVariables/#edgex_security_secret_store) should be set to false, using the corresponding snap option: `config.edgex-security-secret-store`. ### Managing services [managing services]: #managing-services @@ -186,6 +182,9 @@ To query not only the service logs, but also the snap logs (incl. hook apps such sudo journalctl -n 100 -f | grep ``` +!!! tip + The verbosity of service logs is INFO by default. This can be changed by overriding the log level using the `WRITABLE_LOGLEVEL` environment variable using snap config overrides. + ## EdgeX Snaps The following snaps are maintained by the EdgeX working groups. To find all EdgeX snaps on the public Snap Store, [search by keyword](https://snapcraft.io/search?q=edgex). @@ -199,8 +198,6 @@ all reference core services along with several other security, supporting, appli Please refer to common sections above for [installation], [configuration], [managing services], and [debugging]. -#### Services - Upon installation, the following EdgeX services are automatically started: - consul (Registry) @@ -230,19 +227,17 @@ The disabled services can be manually enabled and started; see [managing service For the configuration of services, refer to [configuration]. -#### Secure access +Most services are exposed and accessible on localhost without access control. However, the access from outside is restricted to authorized users. -Most services are exposed and accessible on localhost without access control. - -##### HTTP endpoints -The service endpoints can be accessed securely through the API Gateway. The API Gateway requires a JSON Web Token (JWT) to authenticate requests. Please refer to [Adding EdgeX API Gateway Users Remotely](../../security/Ch-AddGatewayUserRemotely/) and use the snapped `edgexfoundry.secrets-config` utility. +#### Adding API Gateway users +The service endpoints can be accessed securely through the API Gateway. The API Gateway requires a JSON Web Token (JWT) to authorize requests. Please refer to [Adding EdgeX API Gateway Users Remotely](../../security/Ch-AddGatewayUserRemotely/) and use the snapped `edgexfoundry.secrets-config` utility. To get the usage help: ```bash edgexfoundry.secrets-config proxy adduser -h ``` -Example: +Example - create a user: ```bash # Create private key openssl ecparam -genkey -name prime256v1 -noout -out private.pem @@ -256,43 +251,54 @@ KONG_ADMIN_JWT=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-s # use secrets-config to add user # on success, this command with print the user id edgexfoundry.secrets-config proxy adduser --token-type jwt --user --algorithm ES256 --public_key public.pem --id --jwt $KONG_ADMIN_JWT +``` -# get a JWT token for this user +!!! tip "Snap options" + To spin up a pre-configured and securely accessible EdgeX instance, the snap provides a way to pass the public key of a single user with snap options. When requested, the user is created with user `admin`, id `1` and JWT signing algorithm `ES256`. The snap option for passing the public key is: + `apps.secrets-config.proxy.admin.public-key`. + + This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. + +Example - generate a JWT token for this user: +```bash TOKEN=`edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id --expiration=1h` # keep this token in a safe place for future reuse echo $TOKEN > token.jwt ``` +The JWT token can also be created using bash and openssl, but that is beyond the scope of this guide. -Once you have the token you can access the API Gateway as follows: - -The JWT token must be included -via an HTTP `Authorization: Bearer ` header on any REST calls used to access EdgeX services via the API Gateway. - -Example: +Once you have the token, you can access the services via the API Gateway. +Example - use the token: ```bash $ curl --insecure https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" {"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"} ``` -!!! tip "Snap options" - To spin up a pre-configured and securely accessible EdgeX instance, the snap provides a way to pass the public key of a single user with snap options. When requested, the user is created with user `admin`, id `1` and JWT signing algorithm `ES256`. The snap option for passing the public key is: - `apps.secrets-config.proxy.admin.public-key`. - - This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. - -##### Consul -Consul API and UI can be accessed using the consul secret id. For the snap, secret is the value of `SecretID` typically placed in a JSON file at `/var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json`. +#### Accessing Consul +Consul API and UI can be accessed using the consul Secret ID. For the snap, secret is the value of `SecretID` typically placed in a JSON file at `/var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json`. -For example, to get the secret using JQ: +To get the secret: ```bash $ sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json | jq '.SecretID' "ee3964d0-505f-6b62-4c88-0d29a8226daa" ``` +Try it out locally: +```bash +$ curl --insecure --silent http://localhost:8500/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" | jq '.[0].Value' --raw-output | base64 --decode +59880 +``` + +Through the API Gateway: +```bash +$ curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" -H "Authorization: Bearer $TOKEN" | jq '.[0].Value' --raw-output | base64 --decode +59880 +``` + -##### Custom TLS certificates +#### Setting TLS certificates The API Gateway setup generates a self-signed certificate by default. To replace that with your own certificate, refer to API Gateway guide: [Using a bring-your-own external TLS certificate for API gateway](../../security/Ch-APIGateway/#using-a-bring-your-own-external-tls-certificate-for-api-gateway) and use the snapped `edgexfoundry.secrets-config` utility. To get the usage help: From c21479710ae0b2c51192586ba3dfe0f9f071db2e Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Mon, 2 May 2022 20:49:59 +0200 Subject: [PATCH 07/29] feat: add ui instructions Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 43 ++++++++++++++----- 1 file changed, 32 insertions(+), 11 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index e8f463c0cc..0d0b4a4ac7 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -90,7 +90,6 @@ snap set edgexfoundry apps.core-data.config.service-port=8080 The services load the set config options on startup. If the service has already started, a restart is necessary to load them. - #### Disabling security !!! Warning Disabling security is NOT recommended, unless for demonstration purposes, or when there are other means to secure the services. @@ -334,17 +333,31 @@ $ curl -v --cacert /path/to/ca.pem https://server01:8443/core-data/api/v2/ping? ### EdgeX UI [![Get it from the Snap Store][badge]][edgex-ui] -The EdgeX GUI snap is a development tool called to help you get started with EdgeX, -whether you've deployed other components natively, using Docker containers, or -with snaps. For installation instructions, refer to [edgex-ui]. -- Configuration - - Snap - - Snap+docker -- Additional edgexfoundry snap config for UI +For installation instructions, please refer to [edgex-ui]. For usage instructions, please refer to the [Graphical User Interface (GUI)](../../getting-started/tools/Ch-GUI/) guide. +By default, the UI is reachable locally at: +[http://localhost:4000](http://localhost:4000) + +A valid JWT token is required to access the UI; follow [Adding API Gateway users](#adding-api-gateway-users) steps to generate a token. + +To enable all the functionalities of the UI, the following support services should be running: + +* Support Scheduler +* Support Notifications +* System Management Agent +* [EdgeX eKuiper](#edgex-ekuiper) + +For example, to start/install all: +``` +sudo snap start edgexfoundry.support-scheduler +sudo snap start edgexfoundry.support-notifications +sudo snap start edgexfoundry.sys-mgmt-agent +sudo snap install edgex-ekuiper +``` + ### EdgeX CLI [![Get it from the Snap Store][badge]](https://snapcraft.io/edgex-cli) @@ -370,12 +383,20 @@ For usage instructions, please refer to the [Graphical User Interface (GUI)](../ ### Device REST [edgex-device-rest] ### Device RFID LLRP -[](https://snapcraft.io/edgex-device-rfid-llrp) +[edgex-device-rfid-llrp] ### Device SNMP -[edgex-device-snmp](https://snapcraft.io/edgex-device-snmp) -### eKuiper +[edgex-device-snmp] +### EdgeX eKuiper [edgex-ekuiper] +## Useful snippets + +Extend TTL of vault tokens, using [TOKENFILEPROVIDER_DEFAULTTOKENTTL](../../microservices/configuration/CommonEnvironmentVariables/#tokenfileprovider_defaulttokenttl-security-secretstore-setup-service) on security-secretstore-setup: +```bash +sudo snap set edgexfoundry config-enabled=true +sudo snap set edgexfoundry apps.security-secretstore-setup.config.tokenfileprovider-defaulttokenttl=72h +sudo snap restart edgexfoundry.security-secretstore-setup +``` [badge]: https://snapcraft.io/static/images/badges/en/snap-store-white.svg From 7e386bfef694ec0a941945ea0e002edf73411484 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Tue, 3 May 2022 10:39:42 +0200 Subject: [PATCH 08/29] feat: add reference headers Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 32 +++++++++++-------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 0d0b4a4ac7..214a538824 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -189,14 +189,11 @@ The following snaps are maintained by the EdgeX working groups. To find all EdgeX snaps on the public Snap Store, [search by keyword](https://snapcraft.io/search?q=edgex). ### Platform Snap -[![Get it from the Snap Store][badge]][edgexfoundry] +| [Installation][edgexfoundry] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/edgex-go/tree/main/snap) | -The main platform snap, simply called -[edgexfoundry] contains +The main platform snap, simply called `edgexfoundry` contains all reference core services along with several other security, supporting, application, and device services. -Please refer to common sections above for [installation], [configuration], [managing services], and [debugging]. - Upon installation, the following EdgeX services are automatically started: - consul (Registry) @@ -331,12 +328,9 @@ $ curl -v --cacert /path/to/ca.pem https://server01:8443/core-data/api/v2/ping? ### EdgeX UI -[![Get it from the Snap Store][badge]][edgex-ui] - +| [Installation][edgex-ui] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/edgex-ui-go/tree/main/snap) | -For installation instructions, please refer to [edgex-ui]. - -For usage instructions, please refer to the [Graphical User Interface (GUI)](../../getting-started/tools/Ch-GUI/) guide. +For usage instructions, please refer to the [Graphical User Interface (GUI)](../tools/Ch-GUI/) guide. By default, the UI is reachable locally at: [http://localhost:4000](http://localhost:4000) @@ -359,13 +353,23 @@ sudo snap install edgex-ekuiper ``` ### EdgeX CLI -[![Get it from the Snap Store][badge]](https://snapcraft.io/edgex-cli) +| [Installation][edgex-cli] | [Source](https://github.com/edgexfoundry/edgex-cli/tree/main/snap) | -[edgex-cli] +For usage instructions, refer to [Command Line Interface (CLI)](../tools/Ch-CommandLineInterface/) guide. ### App Service Configurable -[edgex-app-service-configurable] +| [Installation][edgex-app-service-configurable] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/app-service-configurable/tree/main/snap) | + +Please refer to [App Service Configurable](../../microservices/application/AppServiceConfigurable/) guide for detailed usage instructions. + +Before you can start the service, you must select one of available profiles, +using snap options. + +For example, to set `mqtt-export` profile using the snap CLI: +```bash +sudo snap set edgex-app-service-configurable profile=mqtt-export +``` ### App RFID LLRP Inventory [edgex-app-rfid-llrp-inventory] @@ -401,7 +405,9 @@ sudo snap restart edgexfoundry.security-secretstore-setup [badge]: https://snapcraft.io/static/images/badges/en/snap-store-white.svg [edgexfoundry]: https://snapcraft.io/edgexfoundry +[edgexfoundry-src]: https://github.com/edgexfoundry/edgex-go/tree/main/snap [edgex-ui]: https://snapcraft.io/edgex-ui +[edgex-ui-src]: https://github.com/edgexfoundry/edgex-ui-go/tree/main/snap [edgex-cli]: https://snapcraft.io/edgex-cli [edgex-app-service-configurable]: https://snapcraft.io/edgex-app-service-configurable [edgex-app-rfid-llrp-inventory]: https://snapcraft.io/edgex-app-rfid-llrp-inventory From 50055f60e9d5b55bd0e9fc80eba06d189eed0a4e Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Wed, 4 May 2022 11:36:03 +0200 Subject: [PATCH 09/29] feat: token interface and reference updates Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 66 ++++++++++++------- 1 file changed, 44 insertions(+), 22 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 214a538824..8d970a2a7a 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -150,7 +150,7 @@ Similarly, a service can be stopped and optionally disabled using `snap stop --d Device and app service snaps provide a similar functionality using the `auto-start` option. - This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. + This is particularly useful when seeding the snap from a [Gadget](https://snapcraft.io/docs/gadget-snap) on an [Ubuntu Core](https://ubuntu.com/core) system. To restart services, e.g. to load the configurations: ```bash @@ -232,6 +232,7 @@ To get the usage help: ```bash edgexfoundry.secrets-config proxy adduser -h ``` +You may also refer to the [secrets-config proxy](../../security/secrets-config-proxy/) documentation. Example - create a user: ```bash @@ -253,22 +254,20 @@ edgexfoundry.secrets-config proxy adduser --token-type jwt --user --algor To spin up a pre-configured and securely accessible EdgeX instance, the snap provides a way to pass the public key of a single user with snap options. When requested, the user is created with user `admin`, id `1` and JWT signing algorithm `ES256`. The snap option for passing the public key is: `apps.secrets-config.proxy.admin.public-key`. - This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. + This is particularly useful when seeding the snap from a [Gadget](https://snapcraft.io/docs/gadget-snap) on an [Ubuntu Core](https://ubuntu.com/core) system. Example - generate a JWT token for this user: ```bash -TOKEN=`edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id --expiration=1h` - -# keep this token in a safe place for future reuse -echo $TOKEN > token.jwt +# on success, a JWT token is printed out and written to user.jwt file +edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id --expiration=1h | tee user.jwt ``` -The JWT token can also be created using bash and openssl, but that is beyond the scope of this guide. +You may alternatively create the JWT token using bash and openssl. That is beyond the scope of this guide. Once you have the token, you can access the services via the API Gateway. Example - use the token: ```bash -$ curl --insecure https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" +$ curl --insecure https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $(cat user.jwt)" {"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"} ``` @@ -283,14 +282,14 @@ $ sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_tok Try it out locally: ```bash -$ curl --insecure --silent http://localhost:8500/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" | jq '.[0].Value' --raw-output | base64 --decode -59880 +$ curl --insecure --silent http://localhost:8500/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" +[{"LockIndex":0,"Key":"edgex/core/2.0/core-data/Service/Port","Flags":0,"Value":"NTk4ODA=","CreateIndex":160,"ModifyIndex":160}] ``` Through the API Gateway: ```bash -$ curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" -H "Authorization: Bearer $TOKEN" | jq '.[0].Value' --raw-output | base64 --decode -59880 +$ curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" -H "Authorization: Bearer $TOKEN" +[{"LockIndex":0,"Key":"edgex/core/2.0/core-data/Service/Port","Flags":0,"Value":"NTk4ODA=","CreateIndex":160,"ModifyIndex":160}] ``` @@ -301,6 +300,7 @@ To get the usage help: ```bash edgexfoundry.secrets-config proxy tls -h ``` +You may also refer to the [secrets-config proxy](../../security/secrets-config-proxy/) documentation. Example: Given certificate `cert.pem`, private key `privkey.pem`, and certificate authority `ca.pem` files: ```bash @@ -324,8 +324,39 @@ $ curl -v --cacert /path/to/ca.pem https://server01:8443/core-data/api/v2/ping? * `apps.secrets-config.proxy.tls.key` * `apps.secrets-config.proxy.tls.snis` (comma-separated values) - This is particularly useful when seeding the snap from a Gadget on an Ubuntu Core system. + This is particularly useful when seeding the snap from a [Gadget](https://snapcraft.io/docs/gadget-snap) on an [Ubuntu Core](https://ubuntu.com/core) system. + +#### Secret Store token interface +The services inside standalone snaps (e.g. device, app snaps) automatically receive a [Secret Store](../../security/Ch-SecretStore/) token when: + +* The standalone snap is downloaded and installed from the store +* The platform snap is downloaded and installed from the store +* Both snaps are installed on the same machine +* The service is registered as an [add-on service](../../security/Ch-Configuring-Add-On-Services/) + +The `edgex-secretstore-token` [content interface](https://snapcraft.io/docs/content-interface) provides the mechanism to automatically supply tokens to connected snaps. + +Execute the following command to check the status of connections: +```bash +sudo snap connections edgexfoundry +``` + +To manually connect the edgexfoundry's plug to a standalone snap's slot: +```bash +snap connect edgexfoundry:edgex-secretstore-token :edgex-secretstore-token +``` + +Note that the token has a limited expiry time of 1h by default. The connection and service startup should happen within the validity period. + +To better understand the snap connections, read the [interface management](https://snapcraft.io/docs/interface-management) +!!! tip "Extend the default Secret Store token TTL" + Set [TOKENFILEPROVIDER_DEFAULTTOKENTTL](../../microservices/configuration/CommonEnvironmentVariables/#tokenfileprovider_defaulttokenttl-security-secretstore-setup-service) for security-secretstore-setup and restart: + ```bash + sudo snap set edgexfoundry config-enabled=true + sudo snap set edgexfoundry apps.security-secretstore-setup.config.tokenfileprovider-defaulttokenttl=72h + sudo snap restart edgexfoundry.security-secretstore-setup + ``` ### EdgeX UI | [Installation][edgex-ui] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/edgex-ui-go/tree/main/snap) | @@ -393,15 +424,6 @@ sudo snap set edgex-app-service-configurable profile=mqtt-export ### EdgeX eKuiper [edgex-ekuiper] -## Useful snippets - -Extend TTL of vault tokens, using [TOKENFILEPROVIDER_DEFAULTTOKENTTL](../../microservices/configuration/CommonEnvironmentVariables/#tokenfileprovider_defaulttokenttl-security-secretstore-setup-service) on security-secretstore-setup: -```bash -sudo snap set edgexfoundry config-enabled=true -sudo snap set edgexfoundry apps.security-secretstore-setup.config.tokenfileprovider-defaulttokenttl=72h -sudo snap restart edgexfoundry.security-secretstore-setup -``` - [badge]: https://snapcraft.io/static/images/badges/en/snap-store-white.svg [edgexfoundry]: https://snapcraft.io/edgexfoundry From 7d91303658790cf7a1065225f54104f3d43e7bfc Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Wed, 4 May 2022 14:24:40 +0200 Subject: [PATCH 10/29] feat: token interface, collapsable notes Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 127 ++++++++++-------- mkdocs.yml | 1 + 2 files changed, 71 insertions(+), 57 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 8d970a2a7a..408b7eff55 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -77,7 +77,7 @@ Mapping examples: [EDGEX_STARTUP_DURATION]: ../../microservices/configuration/CommonEnvironmentVariables/#edgex_startup_duration [ADD_SECRETSTORE_TOKENS]: ../../security/Ch-Configuring-Add-On-Services/#configure-the-services-secret-store-to-use -!!! Note +!!! note The config options are supported as of EdgeX 2.2 and are disabled by default! Setting `config-enabled=true` is necessary to enable their support. @@ -134,7 +134,7 @@ snap start --enable . Similarly, a service can be stopped and optionally disabled using `snap stop --disable`. -!!! tip "Snap options" +??? info "Snap options" To spin up an EdgeX instance with a different startup configuration (e.g. enabled instead of disabled), the `edgexfoundry` snap provides the following config options that accept values `"on"`/`"off"` to enable/disable a service by default: * `consul` @@ -181,7 +181,7 @@ To query not only the service logs, but also the snap logs (incl. hook apps such sudo journalctl -n 100 -f | grep ``` -!!! tip +!!! info The verbosity of service logs is INFO by default. This can be changed by overriding the log level using the `WRITABLE_LOGLEVEL` environment variable using snap config overrides. ## EdgeX Snaps @@ -234,64 +234,70 @@ edgexfoundry.secrets-config proxy adduser -h ``` You may also refer to the [secrets-config proxy](../../security/secrets-config-proxy/) documentation. -Example - create a user: -```bash -# Create private key -openssl ecparam -genkey -name prime256v1 -noout -out private.pem -# Create public key -openssl ec -in private.pem -pubout -out public.pem +!!! example "Creating an example user" + Create private and public keys: + ```bash + openssl ecparam -genkey -name prime256v1 -noout -out private.pem + openssl ec -in private.pem -pubout -out public.pem -# read the API Gateway token -KONG_ADMIN_JWT=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` + ``` -# use secrets-config to add user -# on success, this command with print the user id -edgexfoundry.secrets-config proxy adduser --token-type jwt --user --algorithm ES256 --public_key public.pem --id --jwt $KONG_ADMIN_JWT -``` + Read the API Gateway token: + ```bash + KONG_ADMIN_JWT_FILE=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` + ``` -!!! tip "Snap options" + Use secrets-config to add a user `example` with id `1000`: + ```bash + edgexfoundry.secrets-config proxy adduser --token-type jwt --user example --algorithm ES256 --public_key public.pem --id 1000 --jwt $KONG_ADMIN_JWT + ``` + On success, the above command prints the user id. + +??? tip "Seeding an admin user using snap options" To spin up a pre-configured and securely accessible EdgeX instance, the snap provides a way to pass the public key of a single user with snap options. When requested, the user is created with user `admin`, id `1` and JWT signing algorithm `ES256`. The snap option for passing the public key is: `apps.secrets-config.proxy.admin.public-key`. This is particularly useful when seeding the snap from a [Gadget](https://snapcraft.io/docs/gadget-snap) on an [Ubuntu Core](https://ubuntu.com/core) system. -Example - generate a JWT token for this user: -```bash -# on success, a JWT token is printed out and written to user.jwt file -edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id --expiration=1h | tee user.jwt -``` -You may alternatively create the JWT token using bash and openssl. That is beyond the scope of this guide. +!!! example "Generating a JWT token for the example user" + On success, a JWT token is printed out and written to user.jwt file. + We use the user id `1000` as set in the previous example. + ```bash + edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id 1000 --expiration=1h | tee user.jwt + ``` + +It is also possible to create the JWT token using bash and openssl. But that is beyond the scope of this guide. Once you have the token, you can access the services via the API Gateway. -Example - use the token: -```bash -$ curl --insecure https://localhost:8443/core-data/api/v2/ping? -H "Authorization: Bearer $(cat user.jwt)" -{"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"} -``` +!!! example "Calling an API on behalf of example user" -#### Accessing Consul -Consul API and UI can be accessed using the consul Secret ID. For the snap, secret is the value of `SecretID` typically placed in a JSON file at `/var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json`. + ```bash + curl --insecure https://localhost:8443/core-data/api/v2/ping -H "Authorization: Bearer $(cat user.jwt)" + ``` + Output: `{"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"}` -To get the secret: -```bash -$ sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json | jq '.SecretID' -"ee3964d0-505f-6b62-4c88-0d29a8226daa" -``` +#### Accessing Consul +Consul API and UI can be accessed using the consul token (Secret ID). For the snap, token is the value of `SecretID` typically placed in a JSON file at `/var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json`. -Try it out locally: -```bash -$ curl --insecure --silent http://localhost:8500/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" -[{"LockIndex":0,"Key":"edgex/core/2.0/core-data/Service/Port","Flags":0,"Value":"NTk4ODA=","CreateIndex":160,"ModifyIndex":160}] -``` +!!! example + To get the token: + ```bash + sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json | jq '.SecretID' + ``` + Output: `"ee3964d0-505f-6b62-4c88-0d29a8226daa"` -Through the API Gateway: -```bash -$ curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" -H "Authorization: Bearer $TOKEN" -[{"LockIndex":0,"Key":"edgex/core/2.0/core-data/Service/Port","Flags":0,"Value":"NTk4ODA=","CreateIndex":160,"ModifyIndex":160}] -``` + Try it out locally: + ```bash + curl --insecure --silent http://localhost:8500/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" + ``` + Through the API Gateway: + We need to pass both the Consul token and Secret Store token obtained in [Adding API Gateway users](#adding-api-gateway-users) examples. + ```bash + curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" -H "Authorization: Bearer $(cat user.jwt)" + ``` #### Setting TLS certificates The API Gateway setup generates a self-signed certificate by default. To replace that with your own certificate, refer to API Gateway guide: [Using a bring-your-own external TLS certificate for API gateway](../../security/Ch-APIGateway/#using-a-bring-your-own-external-tls-certificate-for-api-gateway) and use the snapped `edgexfoundry.secrets-config` utility. @@ -302,22 +308,29 @@ edgexfoundry.secrets-config proxy tls -h ``` You may also refer to the [secrets-config proxy](../../security/secrets-config-proxy/) documentation. -Example: Given certificate `cert.pem`, private key `privkey.pem`, and certificate authority `ca.pem` files: -```bash -# read the API Gateway token -KONG_ADMIN_JWT=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` +!!! example + Given certificate `cert.pem`, private key `privkey.pem`, and certificate authority `ca.pem` files: -# add the certificate -edgexfoundry.secrets-config proxy tls --incert /path/to/cert.pem --inkey /path/to/privkey.pem --admin_api_jwt $KONG_ADMIN_JWT -``` + Read the API Gateway token: + ```bash + KONG_ADMIN_JWT=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` + ``` + + Add the certificate, using Kong Admin JWT to authenticate: + ```bash + edgexfoundry.secrets-config proxy tls --incert cert.pem --inkey privkey.pem --admin_api_jwt $KONG_ADMIN_JWT + ``` + + Try it out: + ```bash + curl --cacert ca.pem https://localhost:8443/core-data/api/v2/ping + ``` + Output: `{"message":"Unauthorized"}` + This means that TLS is setup correctly, but the request is not authorized. + Set the `-v` command for diagnosing TLS issues. -Try it out: -```bash -$ curl -v --cacert /path/to/ca.pem https://server01:8443/core-data/api/v2/ping? -H "Authorization: Bearer $TOKEN" -{"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"} -``` -!!! tip "Snap options" +??? tip "Snap options" To spin up an EdgeX instance with custom certificates, the snap provides the following configuration options: * `apps.secrets-config.proxy.tls.cert` diff --git a/mkdocs.yml b/mkdocs.yml index 55d46d0762..6d8e718818 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -158,6 +158,7 @@ markdown_extensions: - admonition - pymdownx.tabbed: alternate_style: true + - pymdownx.details - pymdownx.superfences - pymdownx.mark - pymdownx.critic From 353493f424dbeda7019b37b7a74e12f5f66ac904 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Wed, 4 May 2022 14:29:03 +0200 Subject: [PATCH 11/29] feat: remove temp snap config pages Signed-off-by: Farshid Tavakolizadeh --- .../configuration/snap/Configuration.md | 20 ------------------- .../configuration/snap/ContentInterfaces.md | 5 ----- mkdocs.yml | 3 --- 3 files changed, 28 deletions(-) delete mode 100644 docs_src/microservices/configuration/snap/Configuration.md delete mode 100644 docs_src/microservices/configuration/snap/ContentInterfaces.md diff --git a/docs_src/microservices/configuration/snap/Configuration.md b/docs_src/microservices/configuration/snap/Configuration.md deleted file mode 100644 index 82c9d0ca85..0000000000 --- a/docs_src/microservices/configuration/snap/Configuration.md +++ /dev/null @@ -1,20 +0,0 @@ -# Configuration - - -## Change config files -- how to locally modify files -- link to config provider page - -## Override configurations using Snap options -- Environment variable injection -- Link to consul doc page - -### Seed defaults from a gadget - -## Control service startup -snap enable / disable - -### Control defaults from a Gadget -For seeding defaults from a gadget. -- edgexfoundry (service on/off) -- device/app services (autostart bool) diff --git a/docs_src/microservices/configuration/snap/ContentInterfaces.md b/docs_src/microservices/configuration/snap/ContentInterfaces.md deleted file mode 100644 index 42bf032f33..0000000000 --- a/docs_src/microservices/configuration/snap/ContentInterfaces.md +++ /dev/null @@ -1,5 +0,0 @@ -# Content Interfaces - -## Secret provider - -## Config provider diff --git a/mkdocs.yml b/mkdocs.yml index 6d8e718818..55cc4d8a51 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -64,9 +64,6 @@ nav: - './microservices/configuration/CommonEnvironmentVariables.md' - './microservices/configuration/ConfigurationAndRegistry.md' - './microservices/configuration/V2MigrationCommonConfig.md' - - Snaps: - - './microservices/configuration/snap/Configuration.md' - - './microservices/configuration/snap/ContentInterfaces.md' - Core Services: - './microservices/core/Ch-CoreServices.md' - './microservices/core/data/Ch-CoreData.md' From cc7390cde7cba3af2043dfb53d6396cefe88fdc4 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Wed, 4 May 2022 15:47:28 +0200 Subject: [PATCH 12/29] feat: fill eKuiper section Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 408b7eff55..84067f3938 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -401,6 +401,15 @@ sudo snap install edgex-ekuiper For usage instructions, refer to [Command Line Interface (CLI)](../tools/Ch-CommandLineInterface/) guide. +### EdgeX eKuiper +!!! edgey "EdgeX 2.2" + This version of EdgeX introduces a standalone EdgeX eKuiper snap. + The new snap is now the supported way of using eKuiper with other EdgeX snaps. + + The `kuiper` and `kuiper-cli` apps built into the platform snap are now deprecated! + +For the documentation of standalone EdgeX eKuiper snap, visit the [README](https://github.com/canonical/edgex-ekuiper-snap). + ### App Service Configurable | [Installation][edgex-app-service-configurable] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/app-service-configurable/tree/main/snap) | @@ -417,9 +426,8 @@ sudo snap set edgex-app-service-configurable profile=mqtt-export ### App RFID LLRP Inventory [edgex-app-rfid-llrp-inventory] - -### Device Camera -[edgex-device-camera] + ### Device GPIO [edgex-device-gpio] ### Device Grove @@ -434,8 +442,7 @@ sudo snap set edgex-app-service-configurable profile=mqtt-export [edgex-device-rfid-llrp] ### Device SNMP [edgex-device-snmp] -### EdgeX eKuiper -[edgex-ekuiper] + [badge]: https://snapcraft.io/static/images/badges/en/snap-store-white.svg From 6c00ca76fc7f03fb74f9a5656cdd95c322d78ba8 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Wed, 4 May 2022 16:10:06 +0200 Subject: [PATCH 13/29] feat: add ekuiper, cleanup asc Signed-off-by: Farshid Tavakolizadeh --- .../getting-started/Ch-GettingStartedSnapUsers.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 84067f3938..b1113801e5 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -402,13 +402,16 @@ sudo snap install edgex-ekuiper For usage instructions, refer to [Command Line Interface (CLI)](../tools/Ch-CommandLineInterface/) guide. ### EdgeX eKuiper +| [Installation][edgex-ekuiper] | [Managing Services] | [Debugging] | [Source](https://github.com/canonical/edgex-ekuiper-snap) | + !!! edgey "EdgeX 2.2" This version of EdgeX introduces a standalone EdgeX eKuiper snap. - The new snap is now the supported way of using eKuiper with other EdgeX snaps. + The new snap is the supported way of using eKuiper with other EdgeX snaps. - The `kuiper` and `kuiper-cli` apps built into the platform snap are now deprecated! +For the documentation of the standalone EdgeX eKuiper snap, visit the [README](https://github.com/canonical/edgex-ekuiper-snap). -For the documentation of standalone EdgeX eKuiper snap, visit the [README](https://github.com/canonical/edgex-ekuiper-snap). +!!! note + The standalone EdgeX eKuiper snap documented here should not be confused with the deprecated `edgexfoundry.kuiper` and `edgexfoundry.kuiper-cli` apps built into the platform. The standalone snap can provide similar functionality. ### App Service Configurable @@ -424,6 +427,9 @@ For example, to set `mqtt-export` profile using the snap CLI: sudo snap set edgex-app-service-configurable profile=mqtt-export ``` +!!! note + The standalone App Service Configurable snap documented above should not be confused with the deprecated `edgexfoundry.app-service-configurable`, built into the platform snap. The standalone snap can serve the same functionality of filtering events for eKuiper by using the rules-engine profile. + ### App RFID LLRP Inventory [edgex-app-rfid-llrp-inventory] ### Device GPIO From 5777cb5e5b8c5eb2a5def312a17763fc2056d6f3 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Thu, 5 May 2022 09:40:26 +0200 Subject: [PATCH 17/29] feat: add device-gpio Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 47 ++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index bd2b6455fc..09b00088f1 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -435,6 +435,8 @@ For the documentation of the standalone EdgeX eKuiper snap, visit the [README](h Please refer to [App Service Configurable](../../microservices/application/AppServiceConfigurable/) guide for detailed usage instructions. +**Profile** + Before you can start the service, you must select one of available profiles, using snap options. @@ -449,6 +451,8 @@ sudo snap set edgex-app-service-configurable profile=mqtt-export ### App RFID LLRP Inventory | [Installation][edgex-app-rfid-llrp-inventory] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/app-rfid-llrp-inventory/tree/main/snap) | +**Aliases** + The aliases need to be provided for the service to work. See [Setting the Aliases](https://github.com/edgexfoundry/app-rfid-llrp-inventory/blob/main/README.md#setting-the-aliases). For the snap, this can either be by: @@ -459,7 +463,48 @@ For the snap, this can either be by: ### Device GPIO -[edgex-device-gpio] +| [Installation][edgex-device-gpio] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-gpio/tree/main/snap) | + +**GPIO Access** + +This snap is strictly confined which means that the access to interfaces are subject to various security measures. + +On a Linux distribution without snap confinement for GPIO (e.g. Raspberry Pi OS 11), the snap may be able to access the GPIO directly, without any snap interface and manual connections. + +On Linux distributions with snap confinement for GPIO such as Ubuntu Core, the GPIO access is possible via the [gpio interface](https://snapcraft.io/docs/gpio-interface), provided by a gadget snap. +The official [Raspberry Pi Ubuntu Core](https://ubuntu.com/download/raspberry-pi-core) image includes that gadget. +It is NOT possible to use this snap on Linux distributions that have the GPIO confinement but not the interface (e.g. Ubuntu Server 20.04), unless for development purposes. + +In development environments, it is possible to install the snap in dev mode (using `--devmode` flag which disables security confinement and automatic upgrades) and allows direct GPIO access. + +The `gpio` interface provides slots for each GPIO channel. The slots can be listed using: +```bash +$ sudo snap interface gpio +name: gpio +summary: allows access to specific GPIO pin +plugs: + - edgex-device-gpio +slots: + - pi:bcm-gpio-0 + - pi:bcm-gpio-1 + - pi:bcm-gpio-10 + ... +``` + +The slots are not connected automatically. For example, to connect GPIO-17: +``` +$ sudo snap connect edgex-device-gpio:gpio pi:bcm-gpio-17 +``` + +Check the list of connections: +``` +$ sudo snap connections +Interface Plug Slot Notes +gpio edgex-device-gpio:gpio pi:bcm-gpio-17 manual +… +``` + + ### Device Grove [edgex-device-grove] ### Device Modbus From e1252e780b6fecf839c5c1f75fa7a4725a5e4144 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Thu, 5 May 2022 10:38:10 +0200 Subject: [PATCH 18/29] feat: describe loglevel option Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 09b00088f1..b9eba080ec 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -36,6 +36,8 @@ The configurations that are uploaded to Consul can be modified using Consul's UI [Configuration and Registry Providers](../../microservices/configuration/ConfigurationAndRegistry/) for more information. #### Config provider snap +[config-provider-snap]: #config-provider-snap + Most EdgeX snaps have a [content interface](https://snapcraft.io/docs/content-interface) which allows another snap to seed the snap with configuration files. This is useful when replacing entire configuration files via another snap, packaged with the deployment-specific configurations. @@ -198,7 +200,7 @@ sudo journalctl -n 100 -f | grep ``` !!! info - The verbosity of service logs is INFO by default. This can be changed by overriding the log level using the `WRITABLE_LOGLEVEL` environment variable using snap config overrides. + The verbosity of service logs is INFO by default. This can be changed by overriding the log level using the `WRITABLE_LOGLEVEL` environment variable using snap config overrides `apps..config.writable-loglevel` or globally as `config.writable-loglevel`. ## EdgeX Snaps The following snaps are maintained by the EdgeX working groups. From 78e550c8d43eff8866e5bb5c8db970f8678969bb Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Thu, 5 May 2022 10:38:41 +0200 Subject: [PATCH 19/29] feat: add remaining snaps Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 212 +++++++++++++++++- 1 file changed, 203 insertions(+), 9 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index b9eba080ec..8a391dc62e 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -435,6 +435,28 @@ For the documentation of the standalone EdgeX eKuiper snap, visit the [README](h ### App Service Configurable | [Installation][edgex-app-service-configurable] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/app-service-configurable/tree/main/snap) | +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-app-service-configurable/current/config/ +└── res + ├── external-mqtt-trigger + │   └── configuration.toml + ├── functional-tests + │   └── configuration.toml + ├── http-export + │   └── configuration.toml + ├── metrics-influxdb + │   └── configuration.toml + ├── mqtt-export + │   └── configuration.toml + ├── push-to-core + │   └── configuration.toml + └── rules-engine + └── configuration.toml +``` + Please refer to [App Service Configurable](../../microservices/application/AppServiceConfigurable/) guide for detailed usage instructions. **Profile** @@ -453,20 +475,61 @@ sudo snap set edgex-app-service-configurable profile=mqtt-export ### App RFID LLRP Inventory | [Installation][edgex-app-rfid-llrp-inventory] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/app-rfid-llrp-inventory/tree/main/snap) | +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-app-rfid-llrp-inventory/current/config/ +└── app-rfid-llrp-inventory + └── res + └── configuration.toml +``` + **Aliases** The aliases need to be provided for the service to work. See [Setting the Aliases](https://github.com/edgexfoundry/app-rfid-llrp-inventory/blob/main/README.md#setting-the-aliases). For the snap, this can either be by: -- using a content interface to provide a `configuration.toml` file with the correct aliases, before startup +- using a [config-provider-snap] to provide a `configuration.toml` file with the correct aliases, before startup - setting the values manually in Consul during or after deployment - +### Device Camera +| [Installation][edgex-device-camera] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-camera-go/tree/main/snap) | + +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-device-camera/current/config/ +└── device-camera + └── res + ├── configuration.toml + ├── devices + │   └── camera.toml + └── profiles + ├── camera-axis.yaml + ├── camera-bosch.yaml + └── camera.yaml +``` + ### Device GPIO | [Installation][edgex-device-gpio] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-gpio/tree/main/snap) | +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-device-gpio/current/config +└── device-gpio + └── res + ├── configuration.toml + ├── devices + │   └── device.custom.gpio.toml + └── profiles + └── device.custom.gpio.yaml +``` + **GPIO Access** This snap is strictly confined which means that the access to interfaces are subject to various security measures. @@ -508,17 +571,148 @@ gpio edgex-device-gpio:gpio pi:bcm-gpio-17 manual ### Device Grove -[edgex-device-grove] +| [Installation][edgex-device-grove] | [Source](https://github.com/edgexfoundry/device-grove-c/tree/main/snap) | + +!!! warning "beta" + Device Grove snap is released as beta for `arm64`. It is compatible with EdgeX 1.3 only. + + It does not support the snap configurations described above. + +The default configuration files are under `/var/snap/edgex-device-grove/current/config/`. + +This device service is started by default. +Changes to the configuration files require a restart to take effect: +```bash +sudo snap restart edgex-device-grove +``` + ### Device Modbus -[edgex-device-modbus] +| [Installation][edgex-device-modbus] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-modbus-go/tree/main/snap) | + +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-device-modbus/current/config/ +└── device-modbus + └── res + ├── configuration.toml + ├── devices + │   └── modbus.test.devices.toml + └── profiles + └── modbus.test.device.profile.yml +``` + + ### Device MQTT -[edgex-device-mqtt] +| [Installation][edgex-device-mqtt] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-mqtt-go/tree/main/snap) | + +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-device-mqtt/current/config/ +└── device-mqtt + └── res + ├── configuration.toml + ├── devices + │   └── mqtt.test.device.toml + └── profiles + └── mqtt.test.device.profile.yml +``` + ### Device REST -[edgex-device-rest] +| [Installation][edgex-device-rest] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-rest-go/tree/main/snap) | + +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-device-rest/current/config/ +└── device-rest + └── res + ├── configuration.toml + ├── devices + │   └── sample-devices.toml + └── profiles + ├── sample-image-device.yaml + ├── sample-json-device.yaml + └── sample-numeric-device.yaml + +``` + ### Device RFID LLRP -[edgex-device-rfid-llrp] +| [Installation][edgex-device-rfid-llrp] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-rfid-llrp-go/tree/main/snap) | + +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-device-rfid-llrp/current/config/ +└── device-rfid-llrp + └── res + ├── configuration.toml + ├── devices + ├── profiles + │   ├── llrp.device.profile.yaml + │   └── llrp.impinj.profile.yaml + └── provision_watchers + ├── impinj.provision.watcher.json + └── llrp.provision.watcher.json +``` + +**Subnet setup** + +The `DiscoverySubnets` setting needs to be provided before a device discovery can occur. This can be done in a number of ways: + +- Using `snap set` to set your local subnet information. Example: + + ```bash + $ sudo snap set edgex-device-rfid-llrp apps.device-rfid-llrp.config.app-custom.discovery-subnets="192.168.10.0/24" + + $ curl -X POST http://localhost:59989/api/v2/discovery + ``` + +- Using a [config-provider-snap] to set device configuration + + +- Using the `auto-configure` command. + + This command finds all local network interfaces which are online and non-virtual and sets the value of `DiscoverySubnets` +in Consul. When running with security enabled, it requires a Consul token, so it needs to be run as follows: + + ```bash + # get Consul ACL token + CONSUL_TOKEN=$(sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json | jq ".SecretID" | tr -d '"') + echo $CONSUL_TOKEN + + # start the device service and connect the interfaces required for network interface discovery + sudo snap start edgex-device-rfid-llrp.device-rfid-llrp + sudo snap connect edgex-device-rfid-llrp:network-control + sudo snap connect edgex-device-rfid-llrp:network-observe + + # run the nework interface discovery, providing the Consul token + edgex-device-rfid-llrp.auto-configure $CONSUL_TOKEN + ``` + ### Device SNMP -[edgex-device-snmp] +| [Installation][edgex-device-snmp] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-snmp-go/tree/main/snap) | + +The service is not started by default. Please refer to [configuration] and [managing services]. + +The default configuration files are installed at: +``` +/var/snap/edgex-device-snmp/current/config/ +└── device-snmp + └── res + ├── configuration.toml + ├── devices + │   └── device.snmp.trendnet.TPE082WS.toml + └── profiles + ├── device.snmp.patlite.yaml + ├── device.snmp.switch.dell.N1108P-ON.yaml + └── device.snmp.trendnet.TPE082WS.yaml +``` From ed1715164fbdba6380942274cae489e936a1a5a2 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 09:15:57 +0200 Subject: [PATCH 20/29] fix: revert secret store section title Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 8a391dc62e..fe14f97e9a 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -357,7 +357,8 @@ You may also refer to the [secrets-config proxy](../../security/secrets-config-p This is particularly useful when seeding the snap from a [Gadget](https://snapcraft.io/docs/gadget-snap) on an [Ubuntu Core](https://ubuntu.com/core) system. -#### Secret Store token interface + +#### Secret Store token The services inside standalone snaps (e.g. device, app snaps) automatically receive a [Secret Store](../../security/Ch-SecretStore/) token when: * The standalone snap is downloaded and installed from the store From 869e311b8a12e2c7341f3da81fc5a45fe907506f Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 10:27:57 +0200 Subject: [PATCH 21/29] fix: typo and formatting Signed-off-by: Farshid Tavakolizadeh --- .../Ch-GettingStartedSnapUsers.md | 23 ++++++++++--------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index fe14f97e9a..c64ca55223 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -28,7 +28,8 @@ EdgeX snaps are packaged with default service configuration files. In certain ca #### Config files -The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to Consul by default. Once the service starts without errors, the local configurations become obsolete and will no longer be read. Any modifications after the initial startup will not be applied. +The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to the registry by default. Once the service starts without errors, the local configurations become obsolete and will no longer be read. Any modifications after the initial startup will not be applied. + #### Config registry The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations in Consul are loaded by the service at startup. If the service has already started, a restart is required to load new configurations. Configurations that are in the writable section get loaded not only at startup, but also during the runtime. In other words, changes to the writable configurations are loaded automatically without a restart. Please refer to @@ -436,7 +437,7 @@ For the documentation of the standalone EdgeX eKuiper snap, visit the [README](h ### App Service Configurable | [Installation][edgex-app-service-configurable] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/app-service-configurable/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -476,7 +477,7 @@ sudo snap set edgex-app-service-configurable profile=mqtt-export ### App RFID LLRP Inventory | [Installation][edgex-app-rfid-llrp-inventory] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/app-rfid-llrp-inventory/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -498,7 +499,7 @@ For the snap, this can either be by: ### Device Camera | [Installation][edgex-device-camera] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-camera-go/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -517,7 +518,7 @@ The default configuration files are installed at: ### Device GPIO | [Installation][edgex-device-gpio] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-gpio/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -541,7 +542,7 @@ On Linux distributions with snap confinement for GPIO such as Ubuntu Core, the G The official [Raspberry Pi Ubuntu Core](https://ubuntu.com/download/raspberry-pi-core) image includes that gadget. It is NOT possible to use this snap on Linux distributions that have the GPIO confinement but not the interface (e.g. Ubuntu Server 20.04), unless for development purposes. -In development environments, it is possible to install the snap in dev mode (using `--devmode` flag which disables security confinement and automatic upgrades) and allows direct GPIO access. +In development environments, it is possible to install the snap in dev mode (using `--devmode` flag which disables security confinement and automatic upgrades) to allow direct GPIO access. The `gpio` interface provides slots for each GPIO channel. The slots can be listed using: ```bash @@ -590,7 +591,7 @@ sudo snap restart edgex-device-grove ### Device Modbus | [Installation][edgex-device-modbus] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-modbus-go/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -608,7 +609,7 @@ The default configuration files are installed at: ### Device MQTT | [Installation][edgex-device-mqtt] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-mqtt-go/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -625,7 +626,7 @@ The default configuration files are installed at: ### Device REST | [Installation][edgex-device-rest] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-rest-go/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -645,7 +646,7 @@ The default configuration files are installed at: ### Device RFID LLRP | [Installation][edgex-device-rfid-llrp] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-rfid-llrp-go/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` @@ -699,7 +700,7 @@ in Consul. When running with security enabled, it requires a Consul token, so it ### Device SNMP | [Installation][edgex-device-snmp] | [Configuration] | [Managing Services] | [Debugging] | [Source](https://github.com/edgexfoundry/device-snmp-go/tree/main/snap) | -The service is not started by default. Please refer to [configuration] and [managing services]. +The service is **not started** by default. Please refer to [configuration] and [managing services]. The default configuration files are installed at: ``` From 7c9f2c4092c7c363e4726affd215c1e2d1036877 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 12:05:15 +0200 Subject: [PATCH 22/29] feat: describe device profile uploads Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index c64ca55223..90782475ab 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -30,6 +30,7 @@ EdgeX snaps are packaged with default service configuration files. In certain ca #### Config files The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to the registry by default. Once the service starts without errors, the local configurations become obsolete and will no longer be read. Any modifications after the initial startup will not be applied. +For device services, the configurations of Device and Device Profile are submitted to [Core Metadata](../../microservices/core/metadata/Ch-Metadata) upon startup. Refer to the documentation of [Device Services](../../microservices/device/Ch-DeviceServices/) for details. #### Config registry The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations in Consul are loaded by the service at startup. If the service has already started, a restart is required to load new configurations. Configurations that are in the writable section get loaded not only at startup, but also during the runtime. In other words, changes to the writable configurations are loaded automatically without a restart. Please refer to From a994981823b54cd5fc7b6e53ebe00f0c6e76a5ed Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 12:06:24 +0200 Subject: [PATCH 23/29] fix: KONG_ADMIN_JWT_FILE -> KONG_ADMIN_JWT Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 90782475ab..44d38223b9 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -265,7 +265,7 @@ You may also refer to the [secrets-config proxy](../../security/secrets-config-p Read the API Gateway token: ```bash - KONG_ADMIN_JWT_FILE=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` + KONG_ADMIN_JWT=`sudo cat /var/snap/edgexfoundry/current/secrets/security-proxy-setup/kong-admin-jwt` ``` Use secrets-config to add a user `example` with id `1000`: From ebb4b8843c926aa825fa433d81f7f447cd2133a1 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 12:15:19 +0200 Subject: [PATCH 24/29] fix: device profile descr fix Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 44d38223b9..fd4243869c 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -30,7 +30,7 @@ EdgeX snaps are packaged with default service configuration files. In certain ca #### Config files The default configuration files are typically placed at `/var/snap//current/config`. Upon startup, the server configurations files are uploaded to the registry by default. Once the service starts without errors, the local configurations become obsolete and will no longer be read. Any modifications after the initial startup will not be applied. -For device services, the configurations of Device and Device Profile are submitted to [Core Metadata](../../microservices/core/metadata/Ch-Metadata) upon startup. Refer to the documentation of [Device Services](../../microservices/device/Ch-DeviceServices/) for details. +For device services, the Device and Device Profile files are submitted to [Core Metadata](../../microservices/core/metadata/Ch-Metadata) upon initial startup. Refer to the documentation of [Device Services](../../microservices/device/Ch-DeviceServices/) for details. #### Config registry The configurations that are uploaded to Consul can be modified using Consul's UI or [kv REST API](https://www.consul.io/api/kv). Changes to configurations in Consul are loaded by the service at startup. If the service has already started, a restart is required to load new configurations. Configurations that are in the writable section get loaded not only at startup, but also during the runtime. In other words, changes to the writable configurations are loaded automatically without a restart. Please refer to From e0e7a3cfb001a612bf5ed81f51e07119c8921ee4 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 12:16:27 +0200 Subject: [PATCH 25/29] fix: add missing snap name to set command Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index fd4243869c..1acf70d247 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -104,7 +104,7 @@ Mapping examples: For example, to change the service port of the core-data service on `edgexfoundry` snap to 8080: ```bash -snap set app-options=true +snap set edgexfoundry app-options=true snap set edgexfoundry apps.core-data.config.service-port=8080 ``` From a95f11f090ce7b3b78cff400ca8de34c16185fff Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 14:30:27 +0200 Subject: [PATCH 26/29] fix: pass consul token in file, rename user jwt for consistency Signed-off-by: Farshid Tavakolizadeh --- .../getting-started/Ch-GettingStartedSnapUsers.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 1acf70d247..7f51493f5e 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -281,10 +281,10 @@ You may also refer to the [secrets-config proxy](../../security/secrets-config-p This is particularly useful when seeding the snap from a [Gadget](https://snapcraft.io/docs/gadget-snap) on an [Ubuntu Core](https://ubuntu.com/core) system. !!! example "Generating a JWT token for the example user" - On success, a JWT token is printed out and written to user.jwt file. + On success, a JWT token is printed out and written to `user-jwt.txt` file. We use the user id `1000` as set in the previous example. ```bash - edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id 1000 --expiration=1h | tee user.jwt + edgexfoundry.secrets-config proxy jwt --algorithm ES256 --private_key private.pem --id 1000 --expiration=1h | tee user-jwt.txt ``` It is also possible to create the JWT token using bash and openssl. But that is beyond the scope of this guide. @@ -294,7 +294,7 @@ Once you have the token, you can access the services via the API Gateway. !!! example "Calling an API on behalf of example user" ```bash - curl --insecure https://localhost:8443/core-data/api/v2/ping -H "Authorization: Bearer $(cat user.jwt)" + curl --insecure https://localhost:8443/core-data/api/v2/ping -H "Authorization: Bearer $(cat user-jwt.txt)" ``` Output: `{"apiVersion":"v2","timestamp":"Mon May 2 12:14:17 CEST 2022","serviceName":"core-data"}` @@ -304,19 +304,19 @@ Consul API and UI can be accessed using the consul token (Secret ID). For the sn !!! example To get the token: ```bash - sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json | jq '.SecretID' + $ sudo cat /var/snap/edgexfoundry/current/secrets/consul-acl-token/bootstrap_token.json | jq -r '.SecretID' | tee consul-token.txt ``` - Output: `"ee3964d0-505f-6b62-4c88-0d29a8226daa"` + The output is printed out and written to `consul-token.txt`. Example output: `ee3964d0-505f-6b62-4c88-0d29a8226daa` Try it out locally: ```bash - curl --insecure --silent http://localhost:8500/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" + curl --insecure --silent http://localhost:8500/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:$(cat consul-token.txt)" ``` Through the API Gateway: We need to pass both the Consul token and Secret Store token obtained in [Adding API Gateway users](#adding-api-gateway-users) examples. ```bash - curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:17938174-059e-8b86-122a-9a08a99dcd1c" -H "Authorization: Bearer $(cat user.jwt)" + curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:$(cat consul-token.txt)" -H "Authorization: Bearer $(cat user-jwt.txt)" ``` #### Setting TLS certificates From a1a9159344e29b53eae4cca60e63a0eb54748f82 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 14:35:56 +0200 Subject: [PATCH 27/29] fix: remove $ rfid app in commands Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 7f51493f5e..907a26dcf5 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -671,9 +671,9 @@ The `DiscoverySubnets` setting needs to be provided before a device discovery ca - Using `snap set` to set your local subnet information. Example: ```bash - $ sudo snap set edgex-device-rfid-llrp apps.device-rfid-llrp.config.app-custom.discovery-subnets="192.168.10.0/24" + sudo snap set edgex-device-rfid-llrp apps.device-rfid-llrp.config.app-custom.discovery-subnets="192.168.10.0/24" - $ curl -X POST http://localhost:59989/api/v2/discovery + curl -X POST http://localhost:59989/api/v2/discovery ``` - Using a [config-provider-snap] to set device configuration From 35252c6e234bdc174806ddf785c598485f274525 Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 14:46:07 +0200 Subject: [PATCH 28/29] fix: clarify custom TLS example Signed-off-by: Farshid Tavakolizadeh --- .../getting-started/Ch-GettingStartedSnapUsers.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 907a26dcf5..8b70a74e0f 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -328,8 +328,12 @@ edgexfoundry.secrets-config proxy tls -h ``` You may also refer to the [secrets-config proxy](../../security/secrets-config-proxy/) documentation. -!!! example - Given certificate `cert.pem`, private key `privkey.pem`, and certificate authority `ca.pem` files: +!!! example + Given the following files created outside the scope of this document: + + * `cert.pem` certificate + * `privkey.pem` private key + * `ca.pem` certificate authority file (if not available in root certificates) Read the API Gateway token: ```bash @@ -347,7 +351,8 @@ You may also refer to the [secrets-config proxy](../../security/secrets-config-p ``` Output: `{"message":"Unauthorized"}` This means that TLS is setup correctly, but the request is not authorized. - Set the `-v` command for diagnosing TLS issues. + Set the `-v` command for diagnosing TLS issues. + The `--cacert` can be omitted if the CA is available in root certificates (e.g. CA-signed or pre-installed CA certificate). ??? tip "Seeding a custom TLS certificate using snap options" From a558346e4ea5c0bc7eeeb18e6a96677ba09da0fb Mon Sep 17 00:00:00 2001 From: Farshid Tavakolizadeh Date: Fri, 6 May 2022 14:49:30 +0200 Subject: [PATCH 29/29] fix: change tls cert section title Signed-off-by: Farshid Tavakolizadeh --- docs_src/getting-started/Ch-GettingStartedSnapUsers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md index 8b70a74e0f..a0422bba2e 100644 --- a/docs_src/getting-started/Ch-GettingStartedSnapUsers.md +++ b/docs_src/getting-started/Ch-GettingStartedSnapUsers.md @@ -319,7 +319,7 @@ Consul API and UI can be accessed using the consul token (Secret ID). For the sn curl --insecure --silent https://localhost:8443/consul/v1/kv/edgex/core/2.0/core-data/Service/Port -H "X-Consul-Token:$(cat consul-token.txt)" -H "Authorization: Bearer $(cat user-jwt.txt)" ``` -#### Setting TLS certificates +#### Changing TLS certificates The API Gateway setup generates a self-signed certificate by default. To replace that with your own certificate, refer to API Gateway guide: [Using a bring-your-own external TLS certificate for API gateway](../../security/Ch-APIGateway/#using-a-bring-your-own-external-tls-certificate-for-api-gateway) and use the snapped `edgexfoundry.secrets-config` utility. To get the usage help: