[BIOMAGE-1054] Pipeline flow does not follow the state machine definition

[marcellp]

Background

Link to issue

https://biomage.atlassian.net/browse/BIOMAGE-1054

Link to staging deployment URL

https://ui-marcellp-api153.scp-staging.biomage.net/experiments/marcellp-api153-e52b39624588791a7889e39c617f669e/data-processing

Links to any Pull Requests related to this

N/A

Anything else the reviewers should know about the changes here

N/A

Changes

Code changes

This should fix the issue. As the problem is related to requests from previous runs persisting and messing up the current run, the source is likely just AWS Step Functions activities being reused across runs, and they retain tasks until a success/failure message is sent. Even after a failure message is sent, it may still retain some of the tasks for a bit longer, but we don't know this for sure. Either way, making sure each pipeline run starts with an empty activity should be enough to fix the problem.

Definition of DONE

Your changes will be ready for merging after each of the steps below have been completed:

Testing

• Unit tests written
• Tested locally with Inframock (with latest production data downloaded with biomage experiment pull)
• Deployed to staging

To set up easy local testing with inframock, follow the instructions here: https://github.com/biomage-ltd/inframock
To deploy to the staging environment, follow the instructions here: https://github.com/biomage-ltd/biomage-utils

Documentation updates

Is all relevant documentation updated to reflect the proposed changes in this PR?

• Relevant Github READMEs updated
• Relevant wiki pages created/updated

Approvers

• Approved by a member of the core engineering team
• (UX changes) Approved by vickymorrison (this is her username, tag her if you need approval)

Just before merging:

• After the PR is approved, the unstage script in here: https://github.com/biomage-ltd/biomage-utils is executed. This script cleans up your deployment to staging

Optional

• Photo of a cute animal attached to this PR

[marcellp]

Now staged at https://ui-marcellp-api153.scp-staging.biomage.net/experiments/marcellp-api153-e52b39624588791a7889e39c617f669e/data-processing

[aerlaut]

Code looks good. Tested deleting a running container in staging via Lens, and the new container restarts from the beginning.