imixs-Workflow with Raspberry Pi #871
Replies: 6 comments 4 replies
-
|
Hi @gmillinger, But there is definitely an interesting variant to use Imixs workflow also in a small environment that may meet your requirements in a perfect way! As you can see from the project structure, Imixs-Workflow is divided into modules. And the Core-Module is the interesting part for you. It is a plain java module providing the most of the workflow processing logic. The module is completely independent from the other modules. All what you need is the WorkflowKernel and the ModelManager. The core concept is explained here: https://www.imixs.org/doc/core/workflowkernel.html With version 6.1.0 I am currently refactoring a lot of this code to integrate the OpenBPMN Meta module that allows more functionality working with BPMN 2.0 models. And you do no longer need a Database. You can manage models and process instances in-memory. The JUnit class TestWorkflowKernelBasic gives you an impression how this works. I use the But to make it short: All we need to do is implementing a kind of InMemoryWorkflowEngine in the core module providing you with functionality to manage models (already implemented by the So, if you are interested we can do this together? |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much for this detailed explanation. Yes I am very interested in what you are suggesting. I will have a look at the core-module and get going on my java skills. I will contact you soon and we can get started. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Ralph,
Again thank you for responding to my question on the forum. First, I respect your time and will try not to ask questions that cannot be answered by investigation on my part.
I found open-bpmn when looking for a BPMN modeler, and then to imixs-workflow. This is a sincere compliment, you have done brilliant work with imixs-workflow and everything associated with it. As I have looked through the repository I stopped many times and said to myself “Wow, great work”.
Short history:
- I have been programming and architecting small and large automation systems for over 35 years. Yes I am old.
- I have had development budgets of over $13 million USD, and managed global development teams with up to 200 software engineers reporting to me.
- I started my manufacturing automation career at a very small company named Matrix Automation in Huron, Ohio, USA. I designed and built an industrial workflow platform called ProductionWare and it is still running manufacturing operations at many large plants after 25 years.
- I retired over 5 years ago after doing Product Management and System Architecture for General Electric and Rockwell Automation.
- I started programming in assembly and C, progressed to C++. Now I program using python, javacript, backends in nodejs, and most web-technologies/frameworks
- I have designed and built many workflow engines and industrial automation state machines.
- I have spent the last 2 years experimenting and building workflow engine prototypes to execute on inexpensive and small footprint micro-processors such as Raspberry Pi.
- After getting really bored with retirement I went back to part-time work 3 months ago at Matrix Automation to design and build their next generation industrial workflow platform.
I have only begun to venture into open source and have never been a contributor. I see I have much to learn but I will invest the time to accelerate what needs to be learned. Unfortunately I have not programmed in Java but I have started, I am finding it very much like C++ which I have programmed with for many years. I have been working through imixs-workflow-core as you have recommended. I am setup in vscode and using maven for the build. I have a lot of free hours to do this work and if you can tolerate someone just starting in Java then I can help out.
If you are interested in learning more about industrial workflow and how it naturally flows to human-centric workflow at the manufacturing operations and enterprise levels I can help with that. I don’t know your line of business but I am certain industrial workflow can expand the services you offer if that is what you are interested in.
Best Regards, Greg Millinger
From: Ralph Soika ***@***.***>
Sent: Sunday, September 1, 2024 3:13 PM
To: imixs/imixs-workflow ***@***.***>
Cc: Greg Millinger ***@***.***>; Mention ***@***.***>
Subject: Re: [imixs/imixs-workflow] imixs-Workflow with Raspberry Pi (Discussion #871)
Hi @gmillinger<https://github.com/gmillinger>,
thanks for your interest in our project.
Yes, you are right, the focus of the Imixs-Workflow engine (and in particular the WorkflowService<https://www.imixs.org/doc/engine/workflowservice.html>) is clearly on human centric enterprise processes. The WorkflowService is a Jakarta EE EJB implementation and this service expects a transactional EJB Container and a SQL database. Typically such a container is provided by a Jakrata EE Application server like Wildfly<https://www.wildfly.org/> or Payara<https://www.payara.fish/>. And I guess this is to much stuff for your target environment.
But there is definitely an interesting variant to use Imixs workflow also in a small environment that may meet your requirements in a perfect way!
As you can see from the project structure<https://github.com/imixs/imixs-workflow/tree/master>, Imixs-Workflow is divided into modules. And the Core-Module<https://www.imixs.org/doc/core/index.html> is the interesting part for you. It is a plain java module providing the most of the workflow processing logic. The module is completely independent from the other modules. All what you need is the WorkflowKernel<https://www.imixs.org/doc/core/workflowkernel.html> and the ModelManager<https://www.imixs.org/doc/core/model.html>. The core concept is explained here: https://www.imixs.org/doc/core/workflowkernel.html
With version 6.1.0 I am currently refactoring a lot of this code to integrate the OpenBPMN Meta module<https://github.com/imixs/open-bpmn/tree/master/open-bpmn.metamodel> that allows more functionality working with BPMN 2.0 models. And you do no longer need a Database. You can manage models and process instances in-memory.
The JUnit class TestWorkflowKernelBasic<https://github.com/imixs/imixs-workflow/blob/master/imixs-workflow-core/src/test/java/org/imixs/workflow/kernel/TestWorkflowKernelBasic.java> gives you an impression how this works. I use the MockWorkflowEngine for testing only and it is based on the Mockito test framework<https://site.mockito.org/> which is again a part you would not wish to have in your project I guess :-)
But to make it short: All we need to do is implementing a kind of InMemoryWorkflowEngine in the core module providing you with functionality to manage models (already implemented by the ModelManager) and process WorkItems through the WorkflowKernel.
So, if you are interested we can do this together?
—
Reply to this email directly, view it on GitHub<#871 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BJPDHZL2YVBRL4W5BTZFLWLZUNRMXAVCNFSM6AAAAABNPBLMN6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJRGQYTIOA>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
Beta Was this translation helpful? Give feedback.
-
|
Hello Greg, thank you for the compliments :-) I really appreciate your experience in industry environments. I never worked in this area. And I think about your situation with learning Java and - at the same time - diving deep into the Imixs-Workflow stack. This is not so easy and I want to avoid getting you frustrated. During my latest work integrating Open-BPMN deeper into the core of Imixs-Workflow, I realized that with the Open-BPMN Meta project it is much easier to run imixs-workflow in a In-Memory-Szenario. Much of the code was really old - but also very stable and bullet proved ;-) Now with the new design I am fascinated from the idea to offer a kind of 'Micro-Workflow-Engine' running on plain Java VMs without the need of an EJB Container and a Database. And I think this is what you are searching for when working on your industry workflow project. So, what I'm thinking about at the moment is the following: So from the code foot print this all should look on your Raspberry platform somehow like this. MicroWorkflowService myWorkflowService = new MicroWorkflowService();
myWorkflowService.loadModel("/models/my-workflow.bpmn");
ItemCollection workitem=new ItemCollection();
workitem.model('1.0').task(100).event(1);
workitem.setItemValue("somePayload","abc....");
workitem.setItemValue("someOtherValues",42.0);
myWorkflowService.process(workitem);As you can see - very very simple java code base. I am able to implement the MicroWorkflowService, but I am not able to explain to anyone outside what you can do with it ;-) So I would be dependent on your help. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Ralph,
This is great news.
I am attaching a document that my wife and I co-authored when we worked together at General Electric. Within Manufacturing and the Process industries we created a new market category which we called "Work Process Management". We named it this because the workflow execution was a mix of tasks executed by automation equipment and users. Reading this will give you a step towards what we will do with the engine you have created.
I have a design for how to provide real-time data binding to the events and other workflow properties. The connection to real-time data sources is what changes the workflow from user-centric to a hybrid of automation and user tasks. I will pull together the design documents I have and send them to you so we can exchange ideas for how this will work. I am good with building all the real-time aspects that are added to what you have built. I have years of code that handle the real-time data so it will not be starting from nothing. I have re-written most of it with javascript using a nodejs backend but I will look at how it can be possibly adapted to java.
This also has broad application for IoT and IIoT.
Regards, Greg
…________________________________
From: Ralph Soika ***@***.***>
Sent: Tuesday, September 3, 2024 6:12 PM
To: imixs/imixs-workflow
Cc: Greg Millinger; Mention
Subject: Re: [imixs/imixs-workflow] imixs-Workflow with Raspberry Pi (Discussion #871)
Hello Greg,
thank you for the compliments :-)
I'm also a bit older. I started the Imixs workflow project more than 15 years ago and we serve many medium-sized companies with building business applications based on Imixs-Workflow.
I really appreciate your experience in industry environments. I never worked in this area. And I think about your situation with learning Java and - at the same time - diving deep into the Imixs-Workflow stack. This is not so easy and I want to avoid getting you frustrated.
During my latest work integrating Open-BPMN deeper into the core of Imixs-Workflow, I realized that with the Open-BPMN Meta project it is much easier to run imixs-workflow in a In-Memory-Szenario. Much of the code was really old - but also very stable and bullet proved ;-)
Now with the new design I am fascinated from the idea to offer a kind of 'Micro-Workflow-Engine' running on plain Java VMs without the need of an EJB Container and a Database. And I think this is what you are searching for when working on your industry workflow project.
So, what I'm thinking about at the moment is the following:
Start a new separate project (e.g. Imixs-Micro) and in this project we realize just a simple in-memory-worflow engine. But 'just simple' is not the right wording, because with the existing concepts of the Imixs Workflow Kernel and the micro-kernel architecture it will be a very powerful engine.
So from the code foot print this all should look on your Raspberry platform somehow like this.
MicroWorkflowService myWorkflowService = new MicroWorkflowService();
myWorkflowService.loadModel("/models/my-workflow.bpmn");
ItemCollection workitem=new ItemCollection();
workitem.model('1.0').task(100).event(1);
workitem.setItemValue("somePayload","abc....");
workitem.setItemValue("someOtherValues",42.0);
myWorkflowService.process(workitem);
As you can see - very very simple java code base.
I am able to implement the MicroWorkflowService, but I am not able to explain to anyone outside what you can do with it ;-)
So I would be dependent on your help.
—
Reply to this email directly, view it on GitHub<#871 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BJPDHZNIDM5EN5HTZF7WMM3ZUYX3NAVCNFSM6AAAAABNPBLMN6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJTHAZDANA>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Hi Greg, the document is not attached. Can you send me the document to my company info box [email protected] - I will answer you than with my personal email. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Ralph,
I have sent the background documents to you at ***@***.******@***.***>. Let me know if you have not received them. I am also trimming down a design spec for what I built many years ago so that you can understand the thought process of how an industrial workflow works. It is just meant for communication and is not how I would do it today because the technology has changed so much. I will send it soon.
Regards, Greg
From: Ralph Soika ***@***.***>
Sent: Wednesday, September 4, 2024 3:20 PM
To: imixs/imixs-workflow ***@***.***>
Cc: Greg Millinger ***@***.***>; Mention ***@***.***>
Subject: Re: [imixs/imixs-workflow] imixs-Workflow with Raspberry Pi (Discussion #871)
Hi Greg, the document is not attached. Can you send me the document to my company info box ***@***.******@***.***> - I will answer you than with my personal email.
—
Reply to this email directly, view it on GitHub<#871 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BJPDHZLBZHZ5LDGT7QCMAWDZU5MNJAVCNFSM6AAAAABNPBLMN6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJVGAYDCOI>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
Beta Was this translation helpful? Give feedback.
-
|
Hi Greg, I will send you an invitation too. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Greg, I think we are dealing with two aspects here.
Or in short: human-machine interaction. Both process types can be described as a BPMN process model. As you have already written, you have a lot of experience in programming fast and lightweight processing code running on devices such as automation equipment. The Imixs-Micro Project is maybe just another variant for such a lightweight processing engine. The point is, and this is my understanding of such process-controlled production automation, that in some cases (the exceptions, so to speak) the micro-processes have to trigger a human process. For example, a service technician need to replaces a component on a machine. Usually such a meta-task suspends the micro processing on the device. So one question is how to model something like this and the other question is how to build this from the architecture perspective. BPMN Modelling From the modeling view, I would always separate the micro processes from the meta processes and work with separate BPMN models. Both process need to communicate with each other even if this is more a visual modeling aspact to document how the whole system works. The Architecture The micro process should be run on a device - in-memory - fast and lightweight. This is something that can be run with imixs-micro or with your own NodeJS Engine. The meta process, in contrast, is a transactional highly available business process. So the open question for the micro process is how to communicate with the meta process level? Example:
You can download the example model from here |
Beta Was this translation helpful? Give feedback.
-
|
Hi Ralph,
Hope all is well. Very good email you have sent, and I will address my thoughts by embedding them within. Attached you will find two more reference documents.
WPM – Purdue Model.pdf, positions the Micro-Workflow within manufacturing operations
ProductionWare 1.pdf, from wayback in 2001. Explains the design and thinking for the original workflow engine targeted at controlling a manufacturing process. Architecture drawings will give you an idea of how things worked then. Ancient technology and architecture. We used the MS VBA engine to do the heavy lifting of the workflow engine. This document is not considered confidential because of the technology used is not viable today but please don’t share the document with anyone else. Sharing the concepts is ok.
Thoughts below.
Regards, Greg
From: Ralph Soika ***@***.***>
Sent: Saturday, September 7, 2024 9:08 AM
To: imixs/imixs-workflow ***@***.***>
Cc: Greg Millinger ***@***.***>; Mention ***@***.***>
Subject: Re: [imixs/imixs-workflow] imixs-Workflow with Raspberry Pi (Discussion #871)
Hi Greg,
I read through the documents and thought about the architecture of such a system.
I think we are dealing with two aspects here.
1. The machine-centered micro processing
2. and the management of human-centered processes.
Or in short: human-machine interaction. Both process types can be described as a BPMN process model.
Agree
As you have already written, you have a lot of experience in programming fast and lightweight processing code running on devices such as automation equipment. The Imixs-Micro Project is maybe just another variant for such a lightweight processing engine.
Agreed.
The point is, and this is my understanding of such process-controlled production automation, that in some cases (the exceptions, so to speak) the micro-processes have to trigger a human process. For example, a service technician need to replaces a component on a machine. Usually such a meta-task suspends the micro processing on the device.
Yes!
So one question is how to model something like this and the other question is how to build this from the architecture perspective.
BPMN Modelling
From the modeling view, I would always separate the micro processes from the meta processes and work with separate BPMN models.
As the Micro Process is very fast and maybe runs only view minutes or seconds the meta process can be a long running process for hours or even days.
Yes!
Both process need to communicate with each other even if this is more a visual modeling aspact to document how the whole system works.
The Architecture
The micro process should be run on a device - in-memory - fast and lightweight. This is something that can be run with imixs-micro or with your own NodeJS Engine. The meta process, in contrast, is a transactional highly available business process.
The concept of a meta process is very good. I have also been thinking in this way but did not know how to express it. What is ideal is to use imix-micro to execute the meta process across all platforms.
And for this meta process the Jakarta EE Platform is the perfect choice. It is transactional, scalable and very robust. For example we typical archive closed business process in a cassandra cluster to guaranty High availability over a long period of time (years).
So the open question for the micro process is how to communicate with the meta process level?
Example:
1. The micro-process running on the device detects a hardware problem (e.g. the temperature is to high).
2. The micro process triggers a REST API call to the meta process to start a service task.
3. The micro process sends all relevant device information up to the meta process.
4. An engineer verifies the data and decides how to progress
5. The meta process calls the micro process to suspend the processing. (e.g. stopping the machine)
6. After the service technician confirms the repair of the machine the meta process sends a event to the micro process to continue the processing.
7. The meta process can be highly formal. E.g. according to a 4-eyes principle. The Imixs-Workflow Engine also creates a human readable protocol and can create for example additional documents (e.g. PDF) to be used for later review by the management.
This is a valid use case and there are many other common use cases I can document that will help add to your thought process.
grafik.png (view on web)<https://github.com/user-attachments/assets/fab8fc27-0cc1-4a6b-b26d-6a097613c0d8>
You can download the example model from here<https://github.com/imixs/imixs-micro/tree/main/doc>
Everything you have previously stated is aligned with my thoughts.
What I have done at this point with modern technology is built a simple workflow engine using a node.js module which is instanced within a node.js/express.js website. The website only exposes a page that reflects the operational state of the workflow engine, it is really just a log of the execution of the engine. The website also exposes a websocket that is directly connected to the workflow engine. The workflow engine accepts commands to receive a process definition (BPMN xml), start the process, stop the process, accept external data that advances the execution of a task. Very simple and most importantly it can run on a small micro-processor. The purpose of using the websocket vs a REST API is that the communication to the workflow engine requires bi-directional real-time interaction with the engine, not for this particular version of the engine but future. It is a proof-of-concept for real-time interaction with a web-site that is important for later work.
The fundamental goals of my project are:
- reduce the cost of software and hardware required to execute station level work processes (workflows)
- reduce the software and hardware architectural complexity of the solution
- create natural redundancy through the use of peer-to-peer computing and mesh networking. remove internet and/or WAN dependency within the walls of the facility.
- use mobile-first thinking for all task lists and operational management user interfaces
- use visual modeling for all aspects of work process definition
- provide a natural flow from a work station level workflow to a traditional user-centric workflow using the same technology with mixed architecture. For example, the work station level workflows would use peer-to-peer micro-processors as the computing infrastructure but the executing workflow can be transferred to a traditional centralized application server infrastructure to carry-on with the user-centric workflow tasks.
- the work station level workflows communicate directly with industrial equipment to send and receive data for the completion of tasks. User-centric workflows rely on user interaction and data entry while work station level workflows rely on equipment interaction and sometimes mixed with user interaction.
What is important to note is that work station level workflows may have 100 tasks! But may only take 1 minute to complete the workflow with no user interaction, or they could take 30 minutes to complete with a user using industrial tooling that communicates with the workflow. I have done this for years but with big hardware and infrastructure. My proof-of-concepts has shown I can do most work on a micro-processor that costs $50 USD with better scalability and redundancy than infrastructure that costs $1000s USD.
Personally I think open source projects are the most powerful tool in advancing technology, not just software. I have convinced Matrix Automation to be a consumer, contributor, and to create open source projects. This has been a battle to change their thinking but they have accepted the concepts. It will be interesting to see how you and I can help advance workflow to cover many layers of use within industry and business.
—
Reply to this email directly, view it on GitHub<#871 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BJPDHZP7PLCJNU7IZBKTCADZVL3D7AVCNFSM6AAAAABNPBLMN6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANJXG4YDAOI>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
Beta Was this translation helpful? Give feedback.
-
Hey everyone,
This project looks awesome! Great job on everything. I am new to java, but been programming for many years with C, C++, html, javascript, typescript, and embedded system languages.
I have designed a system to run workflow-like processes from a Raspberry Pi. Up to this point I have a workflow engine that is a node.js module and it executes BPMN 2.0 definitions. Very fast and lightweight. Most of the time the workflow processes are running service tasks and talking with various types of devices such as automation equipment but there are cases where human interaction is necessary through a web-based task list. There also exists a web-based process monitoring and management through express.js.
I was looking for a modeler when I run across this project. The workflow engine looks to be positioned for enterprise level human workflow execution and as such using EE technology. Excuse my non-existent understanding of java, but can I execute the imixs-workflow engine in a very simple and lightweight environment on the Raspberry Pi micro-processor? Can the engine be separated from the EE dependencies? Even if it takes some work on my part to re-build code in some way to just use the engine I am game to do that. Thank you for your thoughts on this.
Beta Was this translation helpful? Give feedback.
All reactions