From 6220d8dfdb878c4286581cd6f7abbefcd2703e61 Mon Sep 17 00:00:00 2001 From: William Lyles <26171886+wilyle@users.noreply.github.com> Date: Wed, 24 Jan 2024 15:09:27 -0800 Subject: [PATCH] document data adapter selector --- docs/design/README.md | 26 ++++++++++++++++--- .../data_adapter_selection_sequence.puml | 12 ++++++++- 2 files changed, 33 insertions(+), 5 deletions(-) diff --git a/docs/design/README.md b/docs/design/README.md index a011917..7d96142 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -14,25 +14,43 @@ The Software-Defined Vehicle will need to connect to the cloud for scenarios suc ## Architecture -At its core, Freyja consists of the following primary components: the **cartographer**, the **emitter**, the **data adapter selector**, and the **signal store**. In addition to these core components, there are multiple interfaces with external components that define how Freyja interacts with the cloud and the rest of the Software Defined Vehicle. There are interfaces for the in-vehicle digital twin (such as Ibeji), the mapping service (provided by customers), the cloud digital twin provider (such as Azure or AWS), and the digital twin providers. Below is a high-level diagram of the Freyja components: +At its core, Freyja consists of the following primary components: the **cartographer**, the **emitter**, the **data adapter selector**, and the **signal store**. In addition to these core components, there are multiple interfaces with external components that define how Freyja interacts with the cloud and the rest of the Software Defined Vehicle. There are interfaces for the in-vehicle digital twin service (such as Ibeji), the mapping service (authored by users), the cloud digital twin provider (such as Azure or AWS), and the digital twin providers (authored by users). Below is a high-level diagram of the Freyja components: ![Component Diagram](./diagrams/freyja_components.svg) -In a typical life cycle, the Freyja application will start up, discover Ibeji via Chariott or a static configuration, then connect to the mapping service to obtain an entity map. This map will define which signals need to be synced with the cloud digital twin, how often they need to be synced, and how the data should be transformed or packaged. Once a mapping is obtained, Freyja will connect to the providers and begin emitting their data according to the mapping. In case of changes on either the device or vehicle side, the mapping is dynamic and can be updated as required to add, remove, or modify the signals that are being emitted. The following is a more detailed diagram illustrating how the components interact and how a mapping results in signal data emissions +In a typical life cycle, the Freyja application will start up, discover Ibeji via Chariott or a static configuration, then connect to the mapping service to obtain an entity map. This map will define which signals need to be synced with the cloud digital twin, how often they need to be synced, and how the data should be transformed or packaged. Once a mapping is obtained, Freyja will connect to the providers and begin emitting their data according to the mapping. In case of changes on either the device or vehicle side, the mapping is dynamic and can be updated as required to add, remove, or modify the signals that are being emitted. + +The following is a more detailed diagram illustrating how the components interact and how a mapping results in signal data emissions: ![Data Flow Sequence Diagram](./diagrams/data_flow_sequence.svg) ### Cartographer -The cartographer is the core component responsible for managing the entity map and tracking which signals should be synchronized to the cloud. The cartographer interfaces with the mapping adapter to poll the mapping service for updates. If there is an update pending, the cartographer will download it and interface with the digital twin adapter to look up the corresponding entity information. Then, the cartographer will use this information to register data adapters with the data adapter selector. Finally, the cartographer will populate the signal store with the signals that should be tracked. If any part of this process fails for a given entity, the cartographer will retry again at a later time. +The cartographer is the core component responsible for managing the entity map and tracking which signals should be synchronized to the cloud. + +The cartographer interfaces with the mapping adapter to poll the mapping service for updates. If there is an update pending, the cartographer will download it and interface with the digital twin adapter to look up the corresponding entity information. Then, the cartographer will use this information to register data adapters with the data adapter selector. Finally, the cartographer will populate the signal store with the signals that should be tracked. If any part of this process fails for a given entity, the cartographer will retry again at a later time. + +The following diagram illustrates the communication between the cartographer and the mapping service: ![Sequence Diagram](./diagrams/mapping_service_to_cartographer_sequence.svg) ### Emitter -The emitter is the core component responsible for actually emitting data. The emitter interfaces with the signal store to determine which signals should be emitted. The signal store is considered to be the source of truth for this information, and every signal present in the store is a signal that should be emitted. The emitter supports intervals at a per-signal level to enable signals to have different requirements on how often they are synced with the cloud. Note that once a signal is added to the mapping and picked up by the cartographer, it can take up to `min(`*`I`*`)` before the signal is emitted, where *`I`* is the set of intervals for signals already being tracked. +The emitter is the core component responsible for actually emitting data. The emitter interfaces with the signal store to determine which signals should be emitted. The signal store is considered to be the source of truth for this information, and every signal present in the store is a signal that will be emitted. + +The emitter supports intervals at a per-signal level to enable signals to have different requirements on how often they are synced with the cloud. Note that once a signal is added to the mapping and picked up by the cartographer, it can take up to `min(`*`I`*`)` before the signal is emitted, where *`I`* is the set of intervals for signals already being tracked. + +### Data Adapter Selector + +The data adapter selector is the core component responsible for managing communication with data adapters. It behaves like a gateway service and allows callers to interact with the correct data adapter for a given entity. + +The data adapter selector's main interface is the `create_or_update_adapter` function, which accepts an entity description as an argument. When calling this function, the data adapter selector will first use the entity's endpoint URI to search for an existing adapter that can handle the requested entity. If no such adapter is found, the entity's protocol and operation are used to search for an adapter type that can handle that entity, and then an adapter is created. In either case, the new entity is registered with the adapter, which then interfaces with that entity's endpoint to obtain data. + +The data adapter selector also supports a "loopback" functionality. When registering an entity with an adapter, the adapter may return a request for a loopback with updated entity info. This indicates to the selector that the matched adapter cannot handle the originally requested entity, but has modified its contents to redirect it to another adapter. This enables scenarios such as managed subscribe to perform pre-processing on entities and recycle other existing data adapters. For more information on the managed subscribe functionality, see the [Eclipse Agemo project](https://github.com/eclipse-chariott/agemo). +Below is a sequence diagram illustrating the data adapter selection process: +![Data Adapter Selector Sequence](./diagrams/data_adapter_selection_sequence.svg) ### External Interfaces diff --git a/docs/design/diagrams/data_adapter_selection_sequence.puml b/docs/design/diagrams/data_adapter_selection_sequence.puml index b24d96f..36ae45d 100644 --- a/docs/design/diagrams/data_adapter_selection_sequence.puml +++ b/docs/design/diagrams/data_adapter_selection_sequence.puml @@ -9,13 +9,23 @@ autonumber "Freyja Cartographer"->"Data Adapter Selector": create_or_update_adapter(entity) -"Data Adapter Selector"->"Data Adapter Selector": Lookup adapter in selector's local cache +"Data Adapter Selector"->"Data Adapter Selector": Look up adapter in selector's local cache alt #Aqua Matching Data Adapter Doesn't Exist "Data Adapter Selector"->"Data Adapter": Create a data adapter associated with the entity endpoint end +loop While adapters return loopback requests "Data Adapter Selector"->"Data Adapter": Register the new entity +"Data Adapter Selector"<-"Data Adapter": Entity registration response +note left +The entity registration response +can indicate success or failure, +or request a loopback with +updated entity information. +end note +end + "Data Adapter Selector"->"Data Adapter Selector": Map entity to adapter in selector's local cache "Data Adapter" -> "In-Vehicle Digital Twin Provider": Call get or subscribe for values