[BIOMAGE-1965] - Fix rerunning of embedding in old experiments

[aerlaut]

Description

Problem

Embedding resets on some experiments when the page is refreshed.

Cause

This happens because the UI fails to generate the correct ETag to get previously calculated embedding. A parameter of the calculation is the time the QC pipeline was last run. This is usually retrieved from the pipeline execution history from AWS. Unfortunately, AWS only keeps 90 days of execution history. To remedy this, the API dynamically generates a date. This results in the pipeline last run date to be different everytime the page is refreshed for experiments whose execution history is erased, i.e. pipelines that are executed more than 90 days ago.

Solution

API V2 actually stores the last execution history of the pipeline in the database and will return this if the pipeline is not found. However, there are experiments which has been run more than 90 days before v2 was released. For these exepriments, we return a fixed value of the date.

Aside from this, some randomness also arise from the UMAP function provided by Seurat. Alex's PR to update the Seurat version fixes this issue. Sara has written some note on that in the ticket.

More details fleshed out the ticket: https://biomage.atlassian.net/browse/BIOMAGE-1965

Staging link: https://ui-agi-beady-gibbon.scp-staging.biomage.net/

Details

URL to issue

https://biomage.atlassian.net/browse/BIOMAGE-1965

Link to staging deployment URL (or set N/A)

N/A

Links to any PRs or resources related to this PR

N/A

Integration test branch

master

Merge checklist

Your changes will be ready for merging after all of the steps below have been completed.

Code updates

Have best practices and ongoing refactors being observed in this PR

• Migrated any selector / reducer used to the new format.

Manual/unit testing

• Tested changes using InfraMock locally or no tests required for change, e.g. Kubernetes chart updates.
• Validated that current unit tests for code work as expected and are sufficient for code coverage or no unit tests required for change, e.g. documentation update.
• Unit tests written or no unit tests required for change, e.g. documentation update.

Integration testing

You must check the box below to run integration tests on the latest commit on your PR branch.
Integration tests have to pass before the PR can be merged. Without checking the box, your PR
will not pass the required status checks for merging.

• Started end-to-end tests on the latest commit.

Documentation updates

• Relevant Github READMEs updated or no GitHub README updates required.
• Relevant Wiki pages created/updated or no Wiki updates required.

Optional

• Staging environment is unstaged before merging.
• Photo of a cute animal attached to this PR.

[codecov]

Codecov Report

Merging #372 (67070cd) into master (e2da4d6) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #372   +/-   ##
=======================================
  Coverage   87.81%   87.81%           
=======================================
  Files         166      166           
  Lines        4496     4498    +2     
  Branches      480      480           
=======================================
+ Hits         3948     3950    +2     
  Misses        509      509           
  Partials       39       39           

Impacted Files	Coverage Δ
src/api.v2/helpers/pipeline/constants.js	100.00% <100.00%> (ø)
src/api.v2/helpers/pipeline/getPipelineStatus.js	95.20% <100.00%> (+0.03%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2da4d6...67070cd. Read the comment docs.

[ivababukova]

for more fun, you can set it to Biomage's birthday: 19th May 2019 :)

[ivababukova]

left a comment which is optional for addressing.