Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Golden Record Process: Orchestration Layer #455

Closed
nicoprow opened this issue Sep 13, 2023 · 4 comments
Closed

Golden Record Process: Orchestration Layer #455

nicoprow opened this issue Sep 13, 2023 · 4 comments
Labels
epic This issue is an epic to other issues golden record Issues relating to the Golden Record process

Comments

@nicoprow
Copy link
Contributor

Currently, it is completely on the cleaning service provider to figure out how to connect the BPDM Pool and the Gate applications to form the golden record process: The process on how a record in the Gate becomes a golden record in the Pool which, in turn, gets distributed back to all Gates.

We should develop an orchestration layer between Pool and Gates that outlines the golden record process in which a cleaning service provider can integrate against. In this way the cleaning service provider can concentrate on its core functionality: Processing and cleaning records.

There have already been discussions about the concept of an orchestration layer. I have taken the results from the previous discussions in #377 and want to condense them here for readability purposes.
@nicoprow
Copy link
Contributor Author

nicoprow commented Sep 13, 2023 •
edited
Loading

Use Case 1: Business Partner Update from Sharing Member

The sharing member updates a business partner input in the Gate.

sequenceDiagram
    SharingMember->>Gate: Upsert business partner A input
    Gate-->>Gate: Persist business partner A input
    Gate->>Orchestrator: (1) Request cleaning of business partner A input
    Orchestrator-->>Orchestrator: (2) Create cleaning task B for business partner A input
    Orchestrator-->>Orchestrator: (2) Update Cleaning Request B State: QUEUED_GENERIC_CLEANING
    Orchestrator-->>Gate: Created cleaning task B
    Gate-->>Gate: (3) Associate cleaning task ID with business partner A
    Gate-->>Gate: (3) Upsert sharing state of business partner A
    Gate-->>SharingMember: Upserted business partner A input
    loop Poll Generic Cleaning
        CleaningServiceDummy->>Orchestrator: (4) Fetch and reserve next cleaning requests in 'Generic Cleaning' queue
        Orchestrator-->>Orchestrator: (5) Update state of cleaning task B: RESERVED_GENERIC_CLEANING
        Orchestrator-->>CleaningServiceDummy: Cleaning task B with business partner input
        CleaningServiceDummy->>CleaningServiceDummy: (6) Create dummy cleaning results
        opt BPN does not exist
            CleaningServiceDummy->>CleaningServiceDummy: (6) Add BPN request IDs into results
        end
        CleaningServiceDummy->>Orchestrator: (7) Send cleaning result for task B
        Orchestrator-->>Orchestrator: (8) Update Cleaning Result of task B
        Orchestrator-->>Orchestrator: (8) Update state of cleaning task B: QUEUED_BPN_PROCESSING
        Orchestrator-->>CleaningServiceDummy: Accept
    end
    loop Poll BPN Processing
        Pool->>Orchestrator: (9) Fetch and reserve next cleaning task in 'BPN Processing' queue
        Orchestrator-->>Orchestrator: (10) Update state of cleaning task B: RESERVED_BPN_PROCESSING
        Orchestrator-->>Pool: Cleaning task B cleaning result
        opt L/S/A result has changed
            Pool-->>Pool: (11) Upsert L/S/A business partners with result
        end
        Pool-->>Pool: (11) Add BPNs from BPN task IDs to cleaning result
        Pool->>Orchestrator: (12) Send BPN cleaning result for task B
        Orchestrator-->>Pool: Accept
        Orchestrator-->>Orchestrator: Update state of cleaning task B: FINISHED
    end
    loop Poll for finished Cleaning Tasks
        Gate->>Orchestrator: (13) Fetch cleaning result by task ID
        Orchestrator-->Gate: Cleaning task B cleaning result
        Gate-->>Gate: (14) Resolve cleaning task B
        Gate-->>Gate: (14) Update sharing state of business partner A
    end
Loading

The following issues detail the steps shown in the sequence diagram above:

@nicoprow
Copy link
Contributor Author

nicoprow commented Sep 13, 2023 •
edited
Loading

Use Case 2: Business Partner Update from Golden Record

The Pool updates one of its golden records, so the update needs to be propagated to the Gates.

Each Gate checks the Pool's Changelog for new BPN updates. If an update to a BPN is recoognized, the Gate queries the corresponding golden record from the Pool and updates the outputdata which is connected to that BPN.

sequenceDiagram
    Pool-->Pool: Update legal entity with BPNL A
    loop Poll Pool BPN Changes
        Gate->>Pool: (1) Fetch BPN changelog entries from last timestamp
        Pool-->>Gate: Changelog entry for BPNL A
        opt Business Partner B in Gate with BPNL A and newer timestamp (2)
            Gate->>Orchestrator: (3) Request golden record update for business partner B
            Orchestrator-->>Orchestrator: (4) Create cleaning task C with business partner output B
            Orchestrator-->>Orchestrator: (4) Update Cleaning task C State: QUEUED_GOLDEN_RECORD_UPDATE
            Orchestrator-->>Gate: Created cleaning task C
            Gate-->>Gate: (5) Associate cleaning task C with business partner B
            Gate-->>Gate: (5) Upsert sharing state of business partner B
        end
    end
    loop Golden Record Update Queue
        CleaningServiceDummy->>Orchestrator: (6)Fetch and reserve next cleaning tasks in 'Golden Record Update' queue
        Orchestrator-->>Orchestrator: (7) Update state of cleaning task C: RESERVED_GOLDEN_RECORD_UPDATE
        Orchestrator-->>CleaningServiceDummy: Cleaning task C with business partner output B
        CleaningServiceDummy->>CleaningServiceDummy: (8) Create dummy cleaning result for business partner B
        CleaningServiceDummy->>Orchestrator: (9) Send cleaning result for task C
        Orchestrator-->>Orchestrator: (10) Update cleaning result for task C
        Orchestrator-->>Orchestrator: (10) Update state of cleaning task C: FINISHED
        Orchestrator-->>CleaningServiceDummy: Accept
    end
    loop Poll for finished Cleaning Tasks
        Gate->>Orchestrator: (11) Fetch cleaning result by task ID
        Orchestrator-->>Gate: Task C cleaning result
        Gate-->>Gate: (12) Resolve cleaning task C
        Gate-->>Gate: (12) Update sharing state of business partner B
    end
Loading

The following issues detail the steps shown in the sequence diagram above:

@nicoprow
Copy link
Contributor Author

nicoprow commented Sep 13, 2023 •
edited
Loading

Issue Dependencies

@nicoprow nicoprow mentioned this issue Sep 13, 2023
Closed
@nicoprow nicoprow added golden record Issues relating to the Golden Record process and removed Orchestrator labels Sep 13, 2023
@nicoprow nicoprow pinned this issue Sep 13, 2023
@nicoprow nicoprow added the epic This issue is an epic to other issues label Sep 13, 2023
@nicoprow nicoprow moved this to 🆕 New in BPDM Kanban Sep 14, 2023
@nicoprow nicoprow removed this from BPDM Kanban Sep 19, 2023
This was referenced Sep 20, 2023
Closed
Closed
@HeyHardy HeyHardy mentioned this issue Oct 18, 2023
Closed
1 task
@Siegfriedk Siegfriedk unpinned this issue Dec 8, 2023
@nicoprow
Copy link
Contributor Author

nicoprow commented Jan 4, 2024

Orchestration Layer is created

@nicoprow nicoprow closed this as completed Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
epic This issue is an epic to other issues golden record Issues relating to the Golden Record process
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nicoprow