diff --git a/docs/arc42/arc42-bpdm.md b/docs/arc42/arc42-bpdm.md index 529704b51..86d06ddb6 100644 --- a/docs/arc42/arc42-bpdm.md +++ b/docs/arc42/arc42-bpdm.md @@ -12,11 +12,8 @@ - [Technical Context](#technical-context) - [Solution Strategy (High Level Picture)](#solution-strategy-high-level-picture) - [Building Block View](#building-block-view) - - [High-Level Architecture (L/S/A Endpoints)](#high-level-architecture-lsa-endpoints) - [High-Level Architecture (Generic Endpoint)](#high-level-architecture-generic-endpoint) - - [Keycloak Authentication \& Autorization Flow](#keycloak-authentication--autorization-flow) - [Runtime View](#runtime-view) - - [Upload Business Partner (BPN-L)](#upload-business-partner-bpn-l) - [Upsert Generic Business Partner](#upsert-generic-business-partner) - [Update on Golden Record Change](#update-on-golden-record-change) - [Business Partner Data Records - States](#business-partner-data-records---states) @@ -25,13 +22,29 @@ - [Deployment View](#deployment-view) - [Applications Deployment without Kubernetes](#applications-deployment-without-kubernetes) - [Single Application Kubernetes Deployment](#single-application-kubernetes-deployment) +- [Authentication \& Autorization](#authentication--autorization) + - [Roles, Rights, Permissions](#roles-rights-permissions) + - [Relevant concepts](#relevant-concepts) + - [BPDM Permission Groups](#bpdm-permission-groups) + - [Permissions](#permissions) + - [Permissions as client resources](#permissions-as-client-resources) + - [Permissions by permission group](#permissions-by-permission-group) + - [Mapping to Portal user roles for all companies (for all Catena-X members):](#mapping-to-portal-user-roles-for-all-companies-for-all-catena-x-members) + - [Technical Users:](#technical-users) + - [Resulting technical users to be creatable in the Portal:](#resulting-technical-users-to-be-creatable-in-the-portal) + - [Demo Configuration](#demo-configuration) + - [Keycloak Example Configuration](#keycloak-example-configuration) + - [EDC Communication](#edc-communication) + - [Data Offer Configuration](#data-offer-configuration) + - [Verified Credentials](#verified-credentials) + - [Purposes](#purposes) + - [Keycloak Authentication \& Autorization Flow](#keycloak-authentication--autorization-flow) - [Crosscutting Concepts](#crosscutting-concepts) - [Business Partner Data Management Standards](#business-partner-data-management-standards) - [Logging Behavior](#logging-behavior) - [Architecture Decisions](#architecture-decisions) - [Quality Requirements](#quality-requirements) - [Risks and Technical Debts](#risks-and-technical-debts) -- [Glossary](#glossary) - [NOTICE](#notice) # Introduction and Goals @@ -183,12 +196,6 @@ The following high level view gives a basic overview about the BPDM Components: # Building Block View -## High-Level Architecture (L/S/A Endpoints) - -Due to a transmission phase there are two concepts of Business Partner Upload Models. The target is to only have the generic Business Partner on the BPDM Gate. - -![bpdm_current_architecture_LSA](assets/cx_bpdm_architecture_v3_2.drawio.svg) - ## High-Level Architecture (Generic Endpoint) @@ -205,94 +212,8 @@ Due to a transmission phase there are two concepts of Business Partner Upload Mo * Currently there is no SME Application available - -## Keycloak Authentication & Autorization Flow - - -```mermaid - -sequenceDiagram - participant BPDM EDC - participant OpenIDConnect Server - participant BPDM Gate - - autonumber - - BPDM EDC -->>OpenIDConnect Server: Send Client Credentials - OpenIDConnect Server-->> BPDM EDC: Respond OAuth2 Token - BPDM EDC -->> BPDM Gate: Send Request with OAuth2 Token in Authorization Header - BPDM Gate -->> OpenIDConnect Server: Validate Token - OpenIDConnect Server -->> BPDM Gate: Confirms validity of Token - BPDM Gate -->> BPDM Gate: Check "resource_access" section of OAuth Token - - -``` - # Runtime View -## Upload Business Partner (BPN-L) - -The diagram below describes the flow of uploading a business partner of type "legal entity". -For further information about the different business partner types (Legal Entity, Site, Address), please have a look on the Standards [CX - 0010 Business Partner Number](https://catena-x.net/de/standard-library) - -```mermaid - -sequenceDiagram - participant CX Member - participant BPDM Gate - participant BPDM Simulator - participant BPDM Pool - autonumber - - rect rgb(125, 184, 240) - Note over CX Member,BPDM Gate: EDC enabled - CX Member->>BPDM Gate: Upload Business Partner - Note left of BPDM Gate: PUT api/catena/input/legal-entities - # loop Healthcheck - # John->>John: Fight against hypochondria - # end - # Note right of John: Rational thoughts! - end - BPDM Gate-->BPDM Gate: Update input changelog - BPDM Simulator->>BPDM Gate: Poll for Business Partner changes - Note right of BPDM Gate: POST api/catena/input/changelog/search - BPDM Gate -->> BPDM Simulator: Send Changelog for Business Partners - BPDM Simulator->>BPDM Gate: Request Business Partner based on Changelog (external ID and business partner type) - Note right of BPDM Gate: POST /api/catena/input/legal-entities/search - BPDM Gate -->> BPDM Simulator: Send Business Partners - BPDM Simulator->>BPDM Gate: Request Sharing State for Business Partner based on Changelog - Note right of BPDM Gate: GET api/catena/sharing-state - BPDM Gate -->> BPDM Simulator: Send Sharing State (BPN) of Business Partners (Sharing state determines if BPN exist or not) - BPDM Simulator-->> BPDM Simulator: Update last sync changelog timestamp - BPDM Simulator->>BPDM Pool: Forward Business Partner - Note left of BPDM Pool: POST/PUT api/catena/legal-entities - opt BPN does not exist - BPDM Pool-->>BPDM Pool: Create BPN - end - BPDM Pool-->>BPDM Pool: Update Changelog - BPDM Pool-->>BPDM Simulator: Respond Business Partner with BPN - BPDM Simulator-->>BPDM Gate: Forward Business Partner with BPN - Note right of BPDM Gate: PUT api/catena/output/legal-entities - BPDM Simulator-->>BPDM Gate: Update Sharing State - Note right of BPDM Gate: PUT api/catena/sharing-state - BPDM Gate-->BPDM Gate: Link External ID with BPN - BPDM Gate-->BPDM Gate: Update Output Changelog - - loop Every x hour - rect rgb(125, 184, 240) - Note over CX Member,BPDM Gate: EDC enabled - CX Member->>BPDM Gate: Poll output changelog and fetch updates - Note left of BPDM Gate: POST api/catena/output/changelog/search - BPDM Gate-->>CX Member: Send changelog - CX Member->>BPDM Gate: Fetch updates - Note left of BPDM Gate: POST api/catena/output/legal-entities/search - BPDM Gate-->>CX Member: Send Business Partner changes - end - CX Member-->>CX Member: Update data - CX Member-->>CX Member: Update last polled timestamp - end -``` - ## Upsert Generic Business Partner > [!NOTE] @@ -472,8 +393,6 @@ C4Context System(pool_postgres, "Pool Database" "Postgres: 14.5") System(gate_postgres, "Gate Database" "Postgres: 14.5") - System(bridge_postgres, "Gate Database" "Postgres: 14.5") - Deployment_Node(pool_machine, "OS Environment", "Linux Alpine 3.16"){ Deployment_Node(pool_java, "Runtime Environment", "JAVA RE 17") { @@ -487,12 +406,6 @@ C4Context } } - Deployment_Node(bridge_machine, "OS Environment", "Linux Alpine 3.16"){ - Deployment_Node(bridge_java, "Runtime Environment", "JAVA RE 17") { - Container(bridge_container, "Bridge Dummy Application", "Spring Boot: 3.1") - } - } - Deployment_Node(orchestrator_machine, "OS Environment", "Linux Alpine 3.16"){ Deployment_Node(orchestrator_java, "Runtime Environment", "JAVA RE 17") { Container(orchestrator_container, "Orchestrator Application", "Spring Boot: 3.1") @@ -511,15 +424,9 @@ C4Context Rel(bpdm_user, gate_container, "HTTP/S") Rel(gate_container, gate_postgres, "TCP/IP") - Rel(bpdm_user, bridge_container, "HTTP/S") - Rel(bridge_container, bridge_postgres, "TCP/IP") - Rel(pool_container, orchestrator_container, "HTTP/S") Rel(gate_container, orchestrator_container, "HTTP/S") Rel(dummy_container, orchestrator_container, "HTTP/S") - - Rel(bridge_container, pool_container, "HTTP/S") - Rel(bridge_container, gate_container, "HTTP/S") ``` @@ -571,6 +478,312 @@ UpdateLayoutConfig($c4ShapeInRow="3", $c4BoundaryInRow="1") +``` + +# Authentication & Autorization + +## Roles, Rights, Permissions + +The authorization concept of the golden record process services (BPDM) has evolved. This impacts the permissions of portal users as well as as the creation of technical users in the Portal. + +### Relevant concepts + +The golden record process contains sharing members which need to share their data (input) to the golden record process and read the result of that process (output). The Pool is a central place that offers golden records that have been created from the shared business partner data. Golden records are distinguished between whether they belong to Catena-X members or not. + +### BPDM Permission Groups + +We defined the following relevant permission groups in BPDM: + +1. Gate Admin: Create, update and read sharing member business partner input data as well as read the output data of the golden record process +2. Gate Input Manager: Create, update and read sharing member business partner input data +3. Gate Input Consumer: Read sharing member business partner input data +4. Gate Output Consumer: Read sharing member business partner output data +5. Pool Admin: Read, create and update golden records as well as meta data in the Pool +6. Pool Cx Member: Read golden records that belong to Catena-X members from the Pool +7. Pool Sharing Member: Read golden records of Catena-X members and the overall changelog +8. Orchestrator Admin: Full access to Golden Record Tasks +9. Orchestrator Task Creator: Create Golden Record Tasks, view task results and status +10. Orchestrator Clean And Sync Task Processor: Reserve and resolve Golden Record Tasks in step 'Clean And Sync' +11. Orchestrator Clean Task Processor: Reserve and resolve Golden Record Tasks in step 'Clean' +12. Orchestrator Pool Task Processor: Reserve and resolve Golden Record Tasks in step 'Pool' + +### Permissions + +#### Permissions as client resources + + + + + + + + + + + + + + +
BPDM PoolBPDM GateBPDM Orchestrator
+
    +
  • read_partner
    • +
  • write_partner
    • +
  • read_partner_member
    • +
  • read_changelog
    • +
  • read_changelog_member
    • +
  • read_metadata
    • +
  • write_metadata
    • +
+ 		+
    +
  • read_input_partner
    • +
  • write_input_partner
    • +
  • read_input_changelog
    • +
  • read_output_partner
    • +
  • read_output_changelog
    • +
  • read_sharing_state
    • +
  • write_sharing_state
    • +
  • read_stats
    • +
+ 		+
    +
  • create_task
    • +
  • read_task
    • +
  • create_reservation_clean
    • +
  • create_result_clean
    • +
  • create_reservation_cleanAndSync
    • +
  • create_result_cleanAndSync
    • +
  • create_reservation_poolSync
    • +
  • create_result_poolSync
    • +
+
+ +#### Permissions by permission group + +Gate permissions: + + + + + + + + + + + + + + + +
AdminInput ManagerInput ConsumerOutput Consumer
+ All of BPDM Gate + +
    +
  • read_input_partner
    • +
  • write_input_partner
    • +
  • read_input_changelog
    • +
  • read_sharing_state
    • +
  • write_sharing_state
    • +
  • read_stats
    • +
+ 		+
    +
  • read_input_changelog
    • +
  • read_input_partner
    • +
  • read_input_changelog
    • +
  • read_sharing_state
    • +
  • read_stats
    • +
+ 		+
    +
  • read_output_changelog
    • +
  • read_output_partner
    • +
  • read_output_changelog
    • +
  • read_sharing_state
    • +
  • read_stats
    • +
+
+ +Pool Permissions: + + + + + + + + + + + + + +
AdminCx MemberSharing Member
+ All of BPDM Pool + +
    +
  • read_partner_member
    • +
  • read_changelog_member
    • +
  • read_metadata
    • +
+ 		+
    +
  • read_partner_member
    • +
  • read_changelog_member
    • +
  • read_metadata
    • +
  • read_changelog
    • +
+
+ +Orchestrator Permissions: + + + + + + + + + + + + + + + + + + +
AdminTask CreatorClean And Sync Task ProcessorClean Task ProcessorPool Task Processor
+ All of BPDM Orchestrator + +
    +
  • create_task
    • +
  • read_task
    • +
+ 		+
    +
  • create_reservation_clean
    • +
  • create_result_clean
    • +
+ 		+
    +
  • create_reservation_cleanAndSync
    • +
  • create_result_cleanAndSync
    • +
+ 		+
    +
  • create_reservation_poolSync
    • +
  • create_result_poolSync
    • +
+
+ +### Mapping to Portal user roles for all companies (for all Catena-X members): + +| BPDM Permission Group | Portal Role | +|--|--| +| Gate Admin | Service Manager | +| Pool Catena-X Member | CX User | + +### Technical Users: + +The golden record service provider needs to be able to generate technical users for each permission group (1 - 8). The technical users for sharing member roles 1 - 4 should be associated with the sharing member's BPNL (So that resulting tokens will have the sharing member's BPNL for authorization purposes). Furthermore, there needs to be one technical user option per Pool and Orchestrator permission group. + +#### Resulting technical users to be creatable in the Portal: + +For BPDM service: + +- Gate BPNLX Admin (for each Sharing Member) +- Pool Admin +- Pool Cx Member +- Pool Sharing Member + +For VAS: + +- Gate BPNLX Consumer: Having both roles 'Gate BPNLX Input Consumer' and 'Gate BPNLX Output Consumer ' (for each Sharing Member) + +Companies which have booked the golden record service should not be able to create any technical users for BPDM. Any such feature to create technical users for companies that are not the golden record service provider should be removed. + +### Demo Configuration + +BPDM is configurable to have arbitrary configurations when it comes to redirect URLs and clients. As long as the above requirements are implemented, BPDM can be configured to be compatible with any Portal environment. + +Still, for the sake of defining a demo configuration, here is a proposal: + +**Clients:** + +`BPDM Pool` + +`BPDM Gate` + +**BPDM Pool:** + +Valid Origin: `https://business-partners.{env}.demo.catena-x.net/pool/*` + +Description: BPDM Pool + +**BPDM Gate:** + +Valid Origin: `https://business-partners{env}.demo.catena-x.net/companies/*` + +Description: BPDM Gate + +### Keycloak Example Configuration + +This example configuration includes the roles, clients and client scopes that BPDM currently expects. +The actual client IDs are subject to change depending on the name they receive in the Portal Keycloak configuration. +[CX-Central.json](../../bpdm-common-test/src/main/resources/keycloak/CX-Central.json) + +For more details see: https://github.com/eclipse-tractusx/sig-release/issues/565 + +## EDC Communication + +### Data Offer Configuration + +Communication with BPDM application must be via EDC. The standards for EDC Assets are defined as follows: + +* [BPDM Pool API Asset Structure](https://github.com/catenax-eV/product-standardization-prod/blob/main/standards/CX-0012-BusinessPartnerDataPoolAPI/4.0.0/CX-0012-BusinessPartnerDataPoolAPI-v4.0.0.md#223-data-asset-structure) +* [BPDM Gate API Asset Structure](https://github.com/catenax-eV/product-standardization-prod/blob/main/standards/CX-0074-BusinessPartnerGateAPI/3.0.0/CX-0074-BusinessPartnerGateAPI-v3.0.0.md#223-data-asset-structure) + + +An example postman collection for Asset definition you can find [here](<../postman/EDC Provider Setup.postman_collection.json>) + +### Verified Credentials + +**Gate** + +To enable communication for uploading and downloading from the gate through EDC, it's essential to have a Verifiable Credential stored in the wallet for BPDM Framework Agreement. This credential will be verified during EDC communication. Additionally, the BPN-Verifiable Credential needs to be validated to ensure that only the sharing member has access to its own gate. + +**Pool** + +To enable communication for downloading from the pool through EDC, it's essential to have a Verifiable Credential stored in the wallet for BPDM Framework Agreement. This credential will be verified during EDC communication. Additionally, the Membership Credential needs to be validated to ensure that only onboarded catena-x members have access to the pool. + +### Purposes + +Additionally each of the purposes need to be checked. You can find them [here](https://github.com/catenax-eV/cx-odrl-profile/blob/main/profile.md#usagepurpose). All purposes beginning with `cx.bpdm.gate` and `cx.bpdm.pool` are relevant. + + +## Keycloak Authentication & Autorization Flow + + +```mermaid + +sequenceDiagram + participant BPDM EDC + participant OpenIDConnect Server + participant BPDM Gate + + autonumber + + BPDM EDC -->>OpenIDConnect Server: Send Client Credentials + OpenIDConnect Server-->> BPDM EDC: Respond OAuth2 Token + BPDM EDC -->> BPDM Gate: Send Request with OAuth2 Token in Authorization Header + BPDM Gate -->> OpenIDConnect Server: Validate Token + OpenIDConnect Server -->> BPDM Gate: Confirms validity of Token + BPDM Gate -->> BPDM Gate: Check "resource_access" section of OAuth Token + + ``` # Crosscutting Concepts @@ -611,17 +824,6 @@ In addition to the Spring standard logs the BPDM applications keep a log of the # Risks and Technical Debts -**Open EDC Questions** -* What capabilities will be provided in the future? -* How to deal with APIs provided behind an EDC? Swagger documentation cannot be seen by the requesting service? -* How can we authorize and authenticate a User/System with individual permissions after it passes the EDC? -* Will there by a Proxy EDC concept? -* ... - - -**Semantic Model and SSI Integration of the Golden Record** -* Not in scope. - **Dependency on third party service provider** * Currently we are not flexible enough to easily change the third party service provider for golden record creation. Therefore the next step will be to introduce an own data persistence layer, getting more independent. * ✔️Solved via "Simulator Application" @@ -635,11 +837,6 @@ In addition to the Spring standard logs the BPDM applications keep a log of the * Uploading via CSV File. Does it requires an EDC? * ⚠️Current State: Yes, is needed. -# Glossary - -The Glossary is currently under development and will be added below after internal approval ([DRAFT](https://confluence.catena-x.net/display/CORE/BPDM+Glossary+-+Internal+-+DRAFT)). - -The current version you can find in the Catena-X Standards. ## NOTICE