Skip to content

Commit

Permalink
feat: Create System Events ADR
Browse files Browse the repository at this point in the history
closes edgexfoundry#581

Signed-off-by: Leonard Goodell <[email protected]>
  • Loading branch information
Leonard Goodell authored and TomBrennan-Eaton committed Aug 23, 2022
1 parent 9cc473c commit 495b5a3
Show file tree
Hide file tree
Showing 2 changed files with 118 additions and 24 deletions.
49 changes: 25 additions & 24 deletions docs_src/design/TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,28 @@

## Architectural Design Records (ADRs)

| Name/Link | Short Description |
| -------------------------------------------------------------------- | ----------------------------------------------- |
| [0001 Registry Refactor](./adr/0001-Registy-Refactor.md) | Separate out Registry and Configuration APIs |
| [0002 Array Datatypes](./adr/device-service/0002-Array-Datatypes.md) | Allow Arrays to be held in Readings |
| [0003 V2 API Principles](./adr/core/0003-V2-API-Principles.md) | Principles and Goals of V2 API Design |
| [0004 Feature Flags](./adr/0004-Feature-Flags.md) | Feature Flag Implementation |
| [0005 Service Self Config Init](./adr/0005-Service-Self-Config.md) | Service Self Config Init & Config Seed Removal |
| [0006 Metrics Collection](./adr/0006-Metrics-Collection.md) | Collection of service telemetry data |
| [0007 Release Automation](./adr/devops/0007-Release-Automation.md) | Overview of Release Automation Flow for EdgeX |
| [0008 Secret Distribution](./adr/security/0008-Secret-Creation-and-Distribution.md) | Creation and Distribution of Secrets |
| [0009 Secure Bootstrapping](./adr/security/0009-Secure-Bootstrapping.md) | Secure Bootstrapping of EdgeX |
| [0011 Device Service REST API](./adr/device-service/0011-DeviceService-Rest-API.md) | The REST API for Device Services in EdgeX v2.x |
| [0012 Device Service Filters](./adr/device-service/0012-DeviceService-Filters.md) | Device Service event/reading filters |
| [0013 Device Service Events via Message Bus](./adr/013-Device-Service-Events-Message-Bus.md) | Device Services send Events via Message Bus |
| [0014 Secret Provider for All](./adr/014-Secret-Provider-For-All.md) | Secret Provider for All EdgeX Services |
| [0015 Encryption between microservices](./adr/security/0015-in-cluster-tls.md) | Details conditions under which TLS is or is not used |
| [0016 Container Image Guidelines](./adr/security/0016-docker-image-guidelines.md) | Documents best practices for security of docker images |
| [0017 Securing access to Consul](./adr/security/0017-consul-security.md) | Access control and authorization strategy for Consul |
| [0018 Service Registry](./adr/0018-Service-Registry.md) | Service registry usage for EdgeX services |
| [0019 EdgeX-CLI V2](./adr/core/0019-EdgeX-CLI-V2.md) | EdgeX-CLI V2 Implementation |
| [0020 Delay start services (SPIFFE/SPIRE)](./adr/security/0020-spiffe.md) | Secret store tokens for delayed start services |
| [0021 Device Profile Changes](./adr/core/0021-Device-Profile-Changes.md) | Rules on device profile modifications |
| [0022 Unit of Measure](./adr/core/0022-UoM.md) | Unit of Measure|
| [0023 North South Messaging](./adr/0023-North-South-Messaging.md) | Provide for messaging from north side systems through command down to device services |
| Name/Link | Short Description |
|---------------------------------------------------------------------------------------------| ----------------------------------------------- |
| [0001 Registry Refactor](./adr/0001-Registy-Refactor.md) | Separate out Registry and Configuration APIs |
| [0002 Array Datatypes](./adr/device-service/0002-Array-Datatypes.md) | Allow Arrays to be held in Readings |
| [0003 V2 API Principles](./adr/core/0003-V2-API-Principles.md) | Principles and Goals of V2 API Design |
| [0004 Feature Flags](./adr/0004-Feature-Flags.md) | Feature Flag Implementation |
| [0005 Service Self Config Init](./adr/0005-Service-Self-Config.md) | Service Self Config Init & Config Seed Removal |
| [0006 Metrics Collection](./adr/0006-Metrics-Collection.md) | Collection of service telemetry data |
| [0007 Release Automation](./adr/devops/0007-Release-Automation.md) | Overview of Release Automation Flow for EdgeX |
| [0008 Secret Distribution](./adr/security/0008-Secret-Creation-and-Distribution.md) | Creation and Distribution of Secrets |
| [0009 Secure Bootstrapping](./adr/security/0009-Secure-Bootstrapping.md) | Secure Bootstrapping of EdgeX |
| [0011 Device Service REST API](./adr/device-service/0011-DeviceService-Rest-API.md) | The REST API for Device Services in EdgeX v2.x |
| [0012 Device Service Filters](./adr/device-service/0012-DeviceService-Filters.md) | Device Service event/reading filters |
| [0013 Device Service Events via Message Bus](./adr/013-Device-Service-Events-Message-Bus.md) | Device Services send Events via Message Bus |
| [0014 Secret Provider for All](./adr/014-Secret-Provider-For-All.md) | Secret Provider for All EdgeX Services |
| [0015 Encryption between microservices](./adr/security/0015-in-cluster-tls.md) | Details conditions under which TLS is or is not used |
| [0016 Container Image Guidelines](./adr/security/0016-docker-image-guidelines.md) | Documents best practices for security of docker images |
| [0017 Securing access to Consul](./adr/security/0017-consul-security.md) | Access control and authorization strategy for Consul |
| [0018 Service Registry](./adr/0018-Service-Registry.md) | Service registry usage for EdgeX services |
| [0019 EdgeX-CLI V2](./adr/core/0019-EdgeX-CLI-V2.md) | EdgeX-CLI V2 Implementation |
| [0020 Delay start services (SPIFFE/SPIRE)](./adr/security/0020-spiffe.md) | Secret store tokens for delayed start services |
| [0021 Device Profile Changes](./adr/core/0021-Device-Profile-Changes.md) | Rules on device profile modifications |
| [0022 Unit of Measure](./adr/core/0022-UoM.md) | Unit of Measure|
| [0023 North South Messaging](./adr/0023-North-South-Messaging.md) | Provide for messaging from north side systems through command down to device services |
| [0024 System Events](./adr/0024-system-events.md) | Provide for messaging from north side systems through command down to device services |
93 changes: 93 additions & 0 deletions docs_src/design/adr/0024-system-events.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# System Events ADR
### Submitters
- Lenny Goodell (Intel)

## Change Log
- [pending](https://github.com/edgexfoundry/edgex-docs/pull/795) (2022-07-12)

## Referenced Use Case(s)
- [System Events for Devices ](https://docs.edgexfoundry.org/2.3/design/ucr/0001-System-Events-for-Devices/)

## Context
System Events, aka Control Plane Events (CPE), are new to EdgeX. This ADR addresses the [System Events for Devices](https://docs.edgexfoundry.org/2.3/design/ucr/0001-System-Events-for-Devices/) use case with an extensible design that can address other System Event use cases that may be identified in the future. This extensible design approach and the fact that System Events are produced and consumed by different EdgeX services makes it architecturally significant warranting this ADR.

## Proposed Design
To address the [System Events for Devices ](https://docs.edgexfoundry.org/2.3/design/ucr/0001-System-Events-for-Devices/) use case, Core Metadata will publish a new `SystemEvent` DTO to the EdgeX MessageBus when a device is added, updated or deleted. Consumers of these System Events will subscribe to the MessageBus to receive the new `SystemEvent` DTO .

#### Data Transfer Object (DTO)

This new `SystemEvent` DTO will contain the following data describing the System Event:

- Event Type - i.e. device
- Event Action - i.e. add/update/delete
- Event Source - i.e core-metadata as the publisher
- Event Owner - i.e device-onvif-camera as the device owner
- Optional based on the Event Type
- Event Other- i.e. onvif-camera (profile name)
- Optional based on the Event Type
- Event Details - i.e. Device DTO of newly added device or empty for Delete action.
- Optional and varies based on the Event Type and or Event Action.

#### MessageBus

Services that publish System Events (Core Metadata) must connect to the EdgeX MessageBus and have MessageBus configuration similar to that of Core Data's [here](https://github.com/edgexfoundry/edgex-go/blob/main/cmd/core-data/res/configuration.toml#L53-L74). This design assumes that Core Metadata will have this capability and configuration due to planned implementation of Service Metrics.

For this design, the `publish topic prefix` for System Events will be a constant in the code rather than another configuration item that never changes.

#### MessageBus Topic

The new `SystemEvent` DTO will be published to a multi-level topic allowing subscribers to filter by topic. The format of this topic for Device System Events will be:

`edgex/system-event/{event-type}/{event-action}/{event-source}/{event-owner}/{event-other}`

where

- `{event-type}` = device
- `{event-action}` = add, update or delete
- `{event-source}` = core-metadata
- `{event-owner}` = `<device service name>`
- `{event-other}` = `<device profile name>`

As noted above Event Owner and Event Other are optional based on the Event Type and may not be meaningful in some future use cases. Those use cases will omit the Event Owner and Event Other from the publish topic as appropriate

This multi-level topic scheme allows consuming services to filter which System Events they receive base of the topic that is subscribed.

!!! example - "Example - System Event subscription topics"
```
edgex/system-event/# - All system events
edgex/system-event/device/# - only device system events
edgex/system-event/#/#/core-metadata/# - only system events from Core Metadata
edgex/system-event/device/add/#/device-onvif-camera/# - only add device system events for device-onvif-camera
edgex/system-event/device/#/#/#/onvif-camera - only device system events for devices created for the onvif-camera device profile
```

#### Consumers

Consumers of Device System Events will likely be custom applications service as described in [System Events for Devices ](https://docs.edgexfoundry.org/2.3/design/ucr/0001-System-Events-for-Devices/). No changes are required to the App Functions SDK since it already supports processing of different types via the [Target Type](https://docs.edgexfoundry.org/2.2/microservices/application/AdvancedTopics/#target-type) capability. Developers of custom App Services that consume System Events will need to do the following:

- Set the Target Type to `&dtos.SystemEvent{}`when creating instance of `ApplicationService` using the [NewAppServiceWithTargetType](https://docs.edgexfoundry.org/2.2/microservices/application/ApplicationServiceAPI/#newappservicewithtargettype) factory function.
- Write custom pipeline function that expects the new `SystemEvent` DTO and process it accordingly. Similar to how the [ToLineProtocol](https://docs.edgexfoundry.org/2.2/microservices/application/BuiltIn/#tolineprotocol) pipeline function expects the Metric DTO.

### Services/Modules impacted

- Core Metadata service
- This service is the single point for device Add/Update/Delete and will be the producer of Device System Events.
- Core Contracts module
- The new `SystemEvent` DTO will be added to this repository
- Camera Management App Service Example
- Once Device System Event are implemented, the [Camera Management](https://github.com/edgexfoundry/edgex-examples/tree/main/application-services/custom/camera-management) example can be updated to consume the new Device System Events.

## Considerations
- This design approach can be used for future use cases, such as `System Events for Device Profiles` and `System Events for Device Services` when/if they are deemed needed.
- Another design approach that was considered is to use Support Notifications to send the System Events via REST. This would require consumers to create subscriptions to receive the Systems Events via REST to some endpoint on the consuming service. This subscription would be created using the existing Support Notifications [subscription](https://app.swaggerhub.com/apis/EdgeXFoundry1/support-notifications/2.2.0#/default/post_subscription) REST API. Likely each subscription would be for a specific Event Type. System Events would be Published by POSTing them to Support Notification [notification](https://app.swaggerhub.com/apis/EdgeXFoundry1/support-notifications/2.2.0#/default/post_notification) REST API, which would them forward them via REST POST to each service subscribed to the particular System Event. The System Event DTO would still be used, just sent via the REST. This approach is more complex, requires consumer services to have new REST endpoint(s) to receive the System Events and relies on REST rather than messaging, thus this approach was not chosen.

## Decision
This design will satisfy the [System Events for Devices ](https://docs.edgexfoundry.org/2.3/design/ucr/0001-System-Events-for-Devices/) use case as well as most future System Event use cases

## Other Related ADRs
- [Metric Collection ADR](https://docs.edgexfoundry.org/2.2/design/adr/0006-Metrics-Collection/) - Use of EdgeX MessageBus to publish and subscribe service metrics
- [North-South Messaging ADR](https://docs.edgexfoundry.org/2.2/design/adr/0023-North-South-Messaging/) - Use of EdgeX MessageBus to publish and subscribe commands/responses

## References
- Control Plane Events (CPE) (aka System Events) were initially proposed as part of the metrics collection ADR as early as March 2020. You may find discussions relevant to CPE in these ADR and other design/architecture discussions since March 2020.
- [initial ADR on EdgeX service level metrics collection for Hanoi](https://github.com/edgexfoundry/edgex-docs/pull/97)

0 comments on commit 495b5a3

Please sign in to comment.