diff --git a/docs/arc42/arc42-bpdm-rel_3_2.md b/docs/arc42/arc42-bpdm.md
similarity index 62%
rename from docs/arc42/arc42-bpdm-rel_3_2.md
rename to docs/arc42/arc42-bpdm.md
index 116f9d1c3..05473a545 100644
--- a/docs/arc42/arc42-bpdm-rel_3_2.md
+++ b/docs/arc42/arc42-bpdm.md
@@ -12,15 +12,19 @@
- [Technical Context](#technical-context)
- [Solution Strategy (High Level Picture)](#solution-strategy-high-level-picture)
- [Building Block View](#building-block-view)
- - [Target Architecture](#target-architecture)
- - [Architecture for Release 3.2](#architecture-for-release-32)
- - [Architecture for Release 3.3](#architecture-for-release-33)
+ - [High-Level Architecture (L/S/A Endpoints)](#high-level-architecture-lsa-endpoints)
+ - [High-Level Architecture (Generic Endpoint)](#high-level-architecture-generic-endpoint)
- [Keycloak Authentication \& Autorization Flow](#keycloak-authentication--autorization-flow)
- [Runtime View](#runtime-view)
- [Upload Business Partner (BPN-L)](#upload-business-partner-bpn-l)
+ - [Upsert Generic Business Partner](#upsert-generic-business-partner)
+ - [Update on Golden Record Change](#update-on-golden-record-change)
- [Deployment View](#deployment-view)
+ - [Applications Deployment without Kubernetes](#applications-deployment-without-kubernetes)
+ - [Single Application Kubernetes Deployment](#single-application-kubernetes-deployment)
- [Crosscutting Concepts](#crosscutting-concepts)
- [Business Partner Data Management Standards](#business-partner-data-management-standards)
+ - [Logging Behavior](#logging-behavior)
- [Architecture Decisions](#architecture-decisions)
- [Quality Requirements](#quality-requirements)
- [Risks and Technical Debts](#risks-and-technical-debts)
@@ -41,7 +45,7 @@ The Golden Record business partner data in combination with the BPN acts as the
> ⚠️ **HINT**: A Business Partner Data cleaning as well as Golden Record Creation Process is **not** part of this reference implementation!
**Additional Information Material**:
-* Visit BPDM on the official Catena-X Website: [bpdm_catenax_website](https://catena-x.net/en/offers/bpdm)
+* Visit BPDM on the official Catena-X Website: [bpdm_catenax_website](https://catena-x.net/en/offers-standards/bpdm)
## Goals Overview
@@ -168,31 +172,34 @@ The following high level view gives a basic overview about the BPDM Components:
**BPN Issuer**
* Every participant in the Catena-X network shall have a unique Business Partner Number (BPN) according to the concept defined by the Catena-X BPN concept. The task of the BPN Generator is to issue such a BPN for a presented Business Partner data object. In that, the BPN Generator serves as the central issuing authority for BPNs within Catena-X.
* Technically, it constitutes a service that is available as a singleton within the network.
-* Currently (Release 3.2) the BPN Issuer is part of the BPDM Pool. After implementing the BPDM Orchestrator, the BPN Issuer should become an independent component.
+* Currently, creation of BPNs is part of the BPDM Pool implementation. After implementing the BPDM Orchestrator, it can be considered if it should be an independent component.
**BPDM Orchestrator**
-* The BPDM Orchestrator is **not** part of Release 3.2.
* Intention of the BPDM Orchestrator is to provide a passive component that offers standardized APIs for the BPDM Gate, BPDM Pool and Data Curation and Enrichment Services to orchestrate the process of Golden Record Creation and handling the different states a business partner record can have during this process.
# Building Block View
-## Target Architecture
+## High-Level Architecture (L/S/A Endpoints)
-
+Due to a transmission phase there are two concepts of Business Partner Upload Models. The target is to only have the generic Business Partner on the BPDM Gate.
-## Architecture for Release 3.2
+
-
+
+## High-Level Architecture (Generic Endpoint)
+
+
**Simulator Service**
* To become more independent in testing the BPDM Application, a Simulator Service was developed.
* The Simulator Services supports the E2E Test Cases to validate the flow from BPDM Gate to BPDM Pool and back again.
-## Architecture for Release 3.3
+**EDC Operator**
+* The diagram above shows two EDCs on Operator side. This is only for visualization purpose. On a technical level there is only one EDC.
-The transition architecture outlines the next goal for current development phase to get one step closer to the target architecture.
+**SME**
+* Currently there is no SME Application available
-In Progress...
## Keycloak Authentication & Autorization Flow
@@ -201,15 +208,15 @@ In Progress...
```mermaid
sequenceDiagram
- participant EDC of CX Member
+ participant BPDM EDC
participant OpenIDConnect Server
participant BPDM Gate
autonumber
- EDC of CX Member-->>OpenIDConnect Server: Send Client Credentials
- OpenIDConnect Server-->>EDC of CX Member: Respond OAuth2 Token
- EDC of CX Member -->> BPDM Gate: Send Request with OAuth2 Token in Authorization Header
+ BPDM EDC -->>OpenIDConnect Server: Send Client Credentials
+ OpenIDConnect Server-->> BPDM EDC: Respond OAuth2 Token
+ BPDM EDC -->> BPDM Gate: Send Request with OAuth2 Token in Authorization Header
BPDM Gate -->> OpenIDConnect Server: Validate Token
OpenIDConnect Server -->> BPDM Gate: Confirms validity of Token
BPDM Gate -->> BPDM Gate: Check "resource_access" section of OAuth Token
@@ -282,24 +289,263 @@ sequenceDiagram
end
```
+## Upsert Generic Business Partner
+
+```mermaid
+sequenceDiagram
+ autonumber
+
+ SharingMember->>Gate: PUT api/catena/input/business-partners Payload: Business Partner Data A
+ Gate-->>Gate: Persist Business Partner Data Input
+ Gate-->>Gate: Set Sharing State to 'Initial'
+ Gate-->>Gate: Add Changelog Entry 'Create' for Business Partner Input
+ Gate->>Orchestrator: POST api/golden-record-tasks Payload: Business Partner Input Data in mode 'UpdateFromSharingMember'
+ Orchestrator-->>Orchestrator: Create Golden Record Task for Business Partner Data
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Result State: 'Pending'
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'CleanAndSync' StepState: 'Queued'
+ Orchestrator-->>Gate: Created Golden Record Task
+ Gate-->>Gate: Set Sharing State Type: 'PENDING' Task ID: Golden Record Task ID
+ Gate-->>SharingMember: Upserted Business Partner
+
+ loop Polling for Step 'CleanAndSync'
+ CleaningServiceDummy->>Orchestrator: POST api/golden-record-tasks/step-reservations Payload: Step 'CleanAndSync'
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'CleanAndSync' StepState: 'Reserved'
+ Orchestrator-->>CleaningServiceDummy: Golden Record Task
+ CleaningServiceDummy-->>CleaningServiceDummy: Set L/S/A and Generic Business Partner Dummy Cleaning Result
+ CleaningServiceDummy-->>CleaningServiceDummy: Set BPN References to L/S/A result
+ CleaningServiceDummy->>Orchestrator: POST api/golden-record-tasks/step-results Payload: Dummy Result
+ Orchestrator-->>Orchestrator: Set Golden Record Task Business Partner Data to Dummy Result
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'PoolSync' StepState: 'Queued'
+ Orchestrator-->>CleaningServiceDummy: Accept
+ end
+
+ loop Polling for Step 'PoolSync'
+ Pool->>Orchestrator: POST api/golden-record-tasks/step-reservations Payload: Step 'PoolSync'
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'PoolSync' StepState: 'Reserved'
+ Orchestrator-->>Pool: Golden Record Task
+ opt Golden Record Legal Entity Data marked as changed
+ Pool-->>Pool: Upsert Legal Entity from Golden Record Task Legal Entity Data
+ Pool-->>Pool: Add Changelog Entry for BPNL
+ end
+ opt Golden Record Site Data marked as changed
+ Pool-->>Pool: Upsert Site from Golden Record Task Site Data
+ Pool-->>Pool: Add Changelog Entry for BPNS
+ end
+ opt Golden Record Address Data marked as changed
+ Pool-->>Pool: Upsert Address from Golden Record Task Address Data
+ Pool-->>Pool: Add Changelog Entry for BPNA
+ end
+ Pool-->>Pool: Set BPNs in Golden Record Task Generic Business Partner Data
+ Pool->>Orchestrator: POST api/golden-record-tasks/step-results Payload: Updated Result
+ Orchestrator-->>Pool: Accept
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'PoolSync' Step State: 'Success'
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Result State: 'Success'
+ end
+
+ loop Polling for finished Golden Record Tasks
+ Gate-->>Gate: Query sharing states in Sharing State Type 'PENDING'
+ Gate->>Orchestrator: POST golden-record-tasks/state/search Payload: Golde Record Task ID
+ Orchestrator-->Gate: Golden Record Task State and Result
+ Gate-->>Gate: Persist Business Partner Output
+ Gate-->>Gate: Set Sharing State 'Success'
+ Gate-->>Gate: Add Changelog Entry 'Create' for Business Partner Output
+ end
+
+ SharingMember->>Gate: POST api/catena/output/changelog/search Payload: From After Last Search Time
+ Gate-->>SharingMember: Changelog entry with Business Partner External ID
+ SharingMember->>Gate: POST api/catena/output/business-partners/search Payload: External ID
+ Gate-->>SharingMember: Business Partner Output
+```
+
+## Update on Golden Record Change
+
+```mermaid
+sequenceDiagram
+ autonumber
+
+ Pool-->Pool: Add Changelog Entry for BPNL
+
+ loop Polling Pool Changelog
+ Gate->>Pool: POST api/catena/changelog/search Payload: From After Last Search Time
+ Pool-->>Gate: Changelog entry for BPNL
+ Gate-->>Gate: Query Business Partner Output with BPNL
+ Gate->>Orchestrator: POST api/golden-record-tasks Payload: Business Partner Output Data in mode 'UpdateFromPool'
+ Orchestrator-->>Orchestrator: Create Golden Record Task for Business Partner Data
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Result State: 'Pending'
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'Clean' StepState: 'Queued'
+ Orchestrator-->>Gate: Created Golden Record Task
+ Gate-->>Gate: Set Sharing State Type: 'PENDING' Task ID: Golden Record Task ID
+ end
+
+ loop Polling for Step 'Clean'
+ CleaningServiceDummy->>Orchestrator: POST api/golden-record-tasks/step-reservations Payload: Step 'Clean'
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'CleanAndSync' StepState: 'Reserved'
+ Orchestrator-->>CleaningServiceDummy: Golden Record Task
+ CleaningServiceDummy->>Orchestrator: POST api/golden-record-tasks/step-results Payload: Golden Record Task Business Partner Data
+ Orchestrator-->>Orchestrator: Set Golden Record Task Business Partner Data to Dummy Result
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Step: 'Clean' Step State: 'Success'
+ Orchestrator-->>Orchestrator: Set Golden Record Task State Result State: 'Success'
+ Orchestrator-->>CleaningServiceDummy: Accept
+ end
+
+ loop Polling for finished Golden Record Tasks
+ Gate-->>Gate: Query sharing states in Sharing State Type 'PENDING'
+ Gate->>Orchestrator: POST golden-record-tasks/state/search Payload: Golden Record Task ID
+ Orchestrator-->Gate: Golden Record Task State and Result
+ Gate-->>Gate: Persist Business Partner Output
+ Gate-->>Gate: Set Sharing State 'Success'
+ Gate-->>Gate: Add Changelog Entry 'Create' for Business Partner Output
+ end
+
+ SharingMember->>Gate: POST api/catena/output/changelog/search Payload: From After Last Search Time
+ Gate-->>SharingMember: Changelog entry with Business Partner External ID
+ SharingMember->>Gate: POST api/catena/output/business-partners/search Payload: External ID
+ Gate-->>SharingMember: Business Partner Output
+
+```
# Deployment View
-How to run the service
+## Applications Deployment without Kubernetes
+
+```mermaid
+C4Context
+
+ Person(bpdm_user, "(Technical) User of the BPDM APIs")
+
+ System(pool_postgres, "Pool Database" "Postgres: 14.5")
+ System(gate_postgres, "Gate Database" "Postgres: 14.5")
+ System(bridge_postgres, "Gate Database" "Postgres: 14.5")
-
+
+ Deployment_Node(pool_machine, "OS Environment", "Linux Alpine 3.16"){
+ Deployment_Node(pool_java, "Runtime Environment", "JAVA RE 17") {
+ Container(pool_container, "Pool Application", "Spring Boot: 3.1")
+ }
+ }
+
+ Deployment_Node(gate_machine, "OS Environment", "Linux Alpine 3.16"){
+ Deployment_Node(gate_java, "Runtime Environment", "JAVA RE 17") {
+ Container(gate_container, "Gate Application", "Spring Boot: 3.1")
+ }
+ }
+
+ Deployment_Node(bridge_machine, "OS Environment", "Linux Alpine 3.16"){
+ Deployment_Node(bridge_java, "Runtime Environment", "JAVA RE 17") {
+ Container(bridge_container, "Bridge Dummy Application", "Spring Boot: 3.1")
+ }
+ }
+
+ Deployment_Node(orchestrator_machine, "OS Environment", "Linux Alpine 3.16"){
+ Deployment_Node(orchestrator_java, "Runtime Environment", "JAVA RE 17") {
+ Container(orchestrator_container, "Orchestrator Application", "Spring Boot: 3.1")
+ }
+ }
+
+ Deployment_Node(dummy_machine, "OS Environment", "Linux Alpine 3.16"){
+ Deployment_Node(dummy_java, "Runtime Environment", "JAVA RE 17") {
+ Container(dummy_container, "Cleaning Service Dummy Application", "Spring Boot: 3.1")
+ }
+ }
+
+ Rel(bpdm_user, pool_container, "HTTP/S")
+ Rel(pool_container, pool_postgres, "TCP/IP")
+
+ Rel(bpdm_user, gate_container, "HTTP/S")
+ Rel(gate_container, gate_postgres, "TCP/IP")
+
+ Rel(bpdm_user, bridge_container, "HTTP/S")
+ Rel(bridge_container, bridge_postgres, "TCP/IP")
+
+ Rel(pool_container, orchestrator_container, "HTTP/S")
+ Rel(gate_container, orchestrator_container, "HTTP/S")
+ Rel(dummy_container, orchestrator_container, "HTTP/S")
+
+ Rel(bridge_container, pool_container, "HTTP/S")
+ Rel(bridge_container, gate_container, "HTTP/S")
+
+```
+
+## Single Application Kubernetes Deployment
+
+```mermaid
+C4Context
+
+ Person(bpdm_user, "(Technical) User of the BPDM APIs")
+
+ Deployment_Node(kubernetes, "Kubernetes Environment", "Kubernetes 1.28"){
+
+ Container(ingress, "Ingress", "Ingress Kubernetes Resource")
+ Container(nginx, "Ingress Controller", "Nginx Reverse Proxy")
+ Container(service, "Service", "Service Kubernetes Resource")
+
+ Container(database, "Database Deployment", "Chart bitnami/postgres:11.9.13")
+ Container(other_bpdm, "Other BPDM Application Deployment", "Helm Chart")
+
+ Deployment_Node(deployment, "Deployment", "Deployment Kubernetes Resource"){
+ Deployment_Node(replicaSet_1, "Replica Set", "Ingress ReplicaSet Resource"){
+ Deployment_Node(pod_1, "Pod", "Pod Kubernetes Resource"){
+ Container(container_1, "BPDM Application Container", "Spring Boot 3 on Linux Alpine 3.6")
+ Container(volume_1, "Config Volume", "Kubernetes Volume Mount")
+ }
+ }
+ }
+
+ Deployment_Node(kubernetes_config, "Kubernetes Configurations", "Logical Grouping"){
+ Container(configMap, "Application Configuration", "Kubernetes ConfigMap Resource")
+ Container(secret, "Secret Configuration", "Kubernetes Secret Resource")
+
+ }
+}
+
+Rel(bpdm_user, nginx, "Sends URL", "HTTPS")
+Rel(ingress, nginx, "Routing Information")
+Rel(nginx, service, "Routes to")
+Rel(service, container_1, "HTTP")
+
+Rel(container_1, volume_1, "mounts")
+Rel(volume_1, configMap, "mounts")
+Rel(volume_1, secret, "mounts")
+
+Rel(container_1, database, "TCP/IP")
+Rel(container_1, other_bpdm, "")
+
+UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="1")
+
+
+
+```
# Crosscutting Concepts
## Business Partner Data Management Standards
[bpdm_standards](https://catena-x.net/de/standard-library)
+## Logging Behavior
+
+As Spring Boot applications BPDM employs Spring
+specific [logging behavior](https://docs.spring.io/spring-boot/docs/3.0.0/reference/htmlsingle/#features.logging)
+
+We enhance the default log entries with user request information including the determined user ID and a generated request ID.
+Not all logs belong to an ongoing user request in which case these entries are empty.
+
+In addition to the Spring standard logs the BPDM applications keep a log of the following events:
+
+* INFO: User requesting resource with resource name and HTTP verb
+* INFO: Request HTTP response
+* INFO: Update/Create Golden Record Business Partners
+* INFO: Creating BPNs
+* ERROR: Uncaught exceptions occurring in the service logic
+
# Architecture Decisions
[Architecture Decision Logs](https://confluence.catena-x.net/display/CORE/BPDM+%7C+Decision+Logs)
(Link will be replaced with ADRs based on Markdown)
* [001-multitenancy_approach](../decision-records/001-multitenancy_approach.md)
* [002-edc_for_pool_api](../decision-records/002-edc_for_pool_api.md)
+* [003-orchestrator_serviceApi_vs_messagebus_approach](../decision-records/003-orchestrator_serviceApi_vs_messagebus_approach.md)
+* [004-openapi_descriptions](../decision-records/004-openapi_descriptions.md)
+* [005-edc-usage-for-third-party-services](../decision-records/005-edc-usage-for-third-party-services.md)
# Quality Requirements
@@ -314,8 +560,6 @@ How to run the service
* Will there by a Proxy EDC concept?
* ...
-**Lack on Developer Resources**
-* Too less developer resources in contrast to the expectations that the BPDM Product and its Golden Record will be a foundation component within Catena-X.
**Semantic Model and SSI Integration of the Golden Record**
* Not in scope.
@@ -327,6 +571,7 @@ How to run the service
**Data Storage and anonymize concept**
* How to anonymize the relations between CX-Member and its belonging Business Partner?
* 💡 Idea: using kind of "ticket numbering"
+* ✔️ Solved via ticketing.
**Accessability for SMEs**
* Uploading via CSV File. Does it requires an EDC?
diff --git a/docs/assets/cx_bpdm_architecture_v3_2.drawio.svg b/docs/assets/cx_bpdm_architecture_v3_2.drawio.svg
index 2f1d394a4..27a935aee 100644
--- a/docs/assets/cx_bpdm_architecture_v3_2.drawio.svg
+++ b/docs/assets/cx_bpdm_architecture_v3_2.drawio.svg
@@ -1,4 +1,4 @@
-
\ No newline at end of file
+
\ No newline at end of file
diff --git a/docs/assets/cx_bpdm_architecture_v3_3.drawio.svg b/docs/assets/cx_bpdm_architecture_v3_3.drawio.svg
new file mode 100644
index 000000000..604e0260a
--- /dev/null
+++ b/docs/assets/cx_bpdm_architecture_v3_3.drawio.svg
@@ -0,0 +1,4 @@
+
+
+
+
\ No newline at end of file