From e5072d9714f600b2d5336749f02a66f166356be6 Mon Sep 17 00:00:00 2001 From: Sean Wiseman <30619751+seanwiseman@users.noreply.github.com> Date: Tue, 8 May 2018 09:40:10 +0100 Subject: [PATCH] Workflow and workflow integration points ADRs (#16) * Draft workflow integration points adr * Add workflow system adr * Remove messaging protocol and version specifics * Clarify who manages the included integrations and plugins * Change decision wording to indicate a proposed solution * Add links to workflow system definition * Typo fix * Move context out of decision * Remove client facing queue from list of requirements * workflow is optional and add event bus to decision * Fix typo --- adr/0002-workflow-integration-points.md | 35 +++++++++++++++ adr/0003-workflow-system.md | 59 +++++++++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 adr/0002-workflow-integration-points.md create mode 100644 adr/0003-workflow-system.md diff --git a/adr/0002-workflow-integration-points.md b/adr/0002-workflow-integration-points.md new file mode 100644 index 0000000..2348eb8 --- /dev/null +++ b/adr/0002-workflow-integration-points.md @@ -0,0 +1,35 @@ +# 2. Provide workflow integration points + +Date: 2018-04-27 + +## Status + +Proposed + +## Context + +A [workflow system](https://en.wikipedia.org/wiki/Workflow_management_system) to handle the creation, +processing and orchestration of customized workflows is an optional component of Libero's currently proposed design. + +Users will each have their individual needs and requirements and therefore may decide to use their +workflow solution of choice. + +It is therefore important to define clear integration points between a [workflow system](https://en.wikipedia.org/wiki/Workflow_management_system) +and the rest of the Libero architecture. + +## Decision + +The workflow solution must provide: + +- HTTP endpoint(s) to allow the triggering of workflows +- Publishing of workflow and activity state event messages via the event bus +- Adhere to the [Libero event schema](https://github.com/libero/walking-skeleton/blob/master/events/schema.md) + +## Consequences + +The [workflow system](https://en.wikipedia.org/wiki/Workflow_management_system) in a Libero deployment will +be independent and swappable. + +Existing systems may need a customized plugin or interface to adhere to the required integration points. + +Does not force specific language choice, service provider or workflow solution. diff --git a/adr/0003-workflow-system.md b/adr/0003-workflow-system.md new file mode 100644 index 0000000..a00c0a2 --- /dev/null +++ b/adr/0003-workflow-system.md @@ -0,0 +1,59 @@ +# 3. Provide a workflow system + +Date: 2018-04-27 + +## Status + +Proposed + +## Context + +In order to maintain portability and avoid dependency on any one service provider we wanted to explore +[workflow systems](https://en.wikipedia.org/wiki/Workflow_management_system) that did not depend on a specific platform. + +Although a Libero deployment should be workflow system agnostic, we want to provide a tried and tested solution as an option. + +There are many off the shelf solutions which offer vastly different levels of features and functionality. + +Each have their pros and cons, the key factors considered where: + +Usage: +- Workflow and activity creation +- Triggering workflows +- Tracking workflow state + +Developer experience: +- API complexity overhead +- Minimal required system configuration +- System deployment +- System extension + +Integration: +- How other services will communicate with it + +Libraries and frameworks that were considered: + +- [SpiffWorkflow](http://spiffworkflow.readthedocs.io/en/latest/) +- [Toil](https://toil.readthedocs.io/en/3.15.0/) +- [Luigi](http://luigi.readthedocs.io/en/stable/index.html) +- [Conductor](https://netflix.github.io/conductor/) +- [Airflow](https://airflow.incubator.apache.org/project.html) + +We also explored using a custom implementation consisting of the minimal components required to make up a lightweight +workflow system written in pure Python. + +## Decision + +Our proposed solution is to use [Airflow](https://airflow.incubator.apache.org/project.html) with a Libero wrapper to fulfil the required workflow integration points and manage the custom workflow and activities. + +## Consequences + +Only have to manage the code base for the wrapper and custom workflows and activities, not the overall system. + +Access to many pre made integrations and plugins with common services managed by the project. + +Must track progress of the [Airflow](https://airflow.incubator.apache.org/project.html) project and keep up to date. + +Handle the larger deployment requirements (when compared with other solutions). + +