Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: automated MACAW tests #840

Merged
merged 19 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/check-metabolictasks.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Test metabolic tasks

on: [push, pull_request]
on: [push]

jobs:
check-metabolictasks:
Expand Down
9 changes: 9 additions & 0 deletions .github/workflows/commentsFromTests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
This PR has been [automatically tested with GH Actions](https://github.com/SysBioChalmers/Human-GEM/actions/runs/{GH_ACTION_RUN}). Here is the output of the macaw test:

<pre>
{TEST_RESULTS}
</pre>

A more detailed output from this test run is also committed to `data/macawResults/macaw_results.csv`.

> _Note: In the case of multiple test runs, this post will be edited._
58 changes: 58 additions & 0 deletions .github/workflows/macaw-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
name: Run macaw tests

on: [pull_request]

jobs:
macaw-tests:
runs-on: ubuntu-latest
timeout-minutes: 60

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Test checkout
run: ls -la && cd code && ls -la

- name: Set up Python 3
uses: actions/setup-python@v4
with:
python-version: "3.10"

- name: Install macaw
run: pip install git+https://github.com/Devlin-Moyer/macaw.git@main numpy==1.26.4

- name: Run macaw
id: macaw-run
run: |
TEST_RESULTS=$(python code/test/macawTests.py)
echo $TEST_RESULTS
PARSED_RESULTS="${TEST_RESULTS//$'\n'/'<br>'}"
PARSED_RESULTS="${PARSED_RESULTS//$'\r'/'<br>'}"
echo $PARSED_RESULTS
echo "results=$PARSED_RESULTS" >> $GITHUB_OUTPUT

- name: Mention PR# in README.md
env:
PR_NUMBER: ${{ github.event.number }}
run: sed -i -e "s/#[[:digit:]]\{3,4\}/#$PR_NUMBER/g" data/macawResults/README.md

- name: Auto-commit results
uses: stefanzweifel/git-auto-commit-action@v4
with:
commit_user_name: memote-bot
commit_message: "chore: add macaw test result"
file_pattern: data/macawResults/*
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.number }}

- name: Post comment
uses: NejcZdovc/comment-pr@v2
with:
file: "commentsFromTests.md"
env:
GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}
TEST_RESULTS: ${{steps.macaw-run.outputs.results}}
GH_ACTION_RUN: ${{github.run_id}}

2 changes: 1 addition & 1 deletion .github/workflows/yaml-conversion.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Test YAML conversion

on: [push, pull_request]
on: [push]

jobs:
yaml-conversion:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/yaml-validation.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: YAML validation

on: [push, pull_request]
on: [push]

jobs:
yaml-validation:
Expand Down
8 changes: 8 additions & 0 deletions code/test/macawTests.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import cobra
from macaw.main import dead_end_test, duplicate_test

model = cobra.io.load_yaml_model("model/Human-GEM.yml")
(dead_end_results, dead_end_edges) = dead_end_test(model)
(duplicate_results, duplicate_edges) = duplicate_test(model)
output = dead_end_results.merge(duplicate_results)
output.to_csv('data/macawResults/macaw_results.csv', index = False)
18 changes: 18 additions & 0 deletions data/macawResults/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# MACAW results

The file here contains results from the [MACAW](https://github.com/Devlin-Moyer/macaw) `dead_end_test` and `duplicate_test` tests.

The test results shown here were obtained by the GitHub Actions run in **PR #840**, and will be updated by any subsequent PR. Summary results are shown as a comment in the corresponding PR.

### `dead_end_test`
Looks for metabolites in Human-GEM that can only be produced by all reactions they participate in or only consumed, then identifies all reactions that are prevented from sustaining steady-state fluxes because of each of these dead-end metabolites. The simplest case of a dead-end metabolite is one that only participates in a single reaction. Also flags all reversible reactions that can only carry fluxes in a single direction because one of their metabolites can either only be consumed or only be produced by all other reactions it participates in.

### `duplicate_test`
Identifies sets of reactions that may be duplicates of each other because they:

- Involve exactly the same metabolites with exactly the same stoichiometric coefficients (but potentially different associated genes).
- Involve exactly the same metabolites, but go in different directions and/or some are reversible and some are not.
- Involve exactly the same metabolites, but with different stoichiometric coefficients.
- Represent the oxidation and/or reduction of the same metabolite, but use different electron acceptors/donors from the given list of pairs of oxidized and reduced forms of various electron carriers (e.g. NAD(H), NADP(H), FAD(H2), ubiquinone/ubiquinol, cytochromes).

It is possible for a single reaction to fit in multiple of the above categories. There are sometimes cases where sets of reactions that fall into one of the above categories are completely legitimate representations of real biochemistry (e.g. separate irreversible reactions for importing vs exporting the same metabolite because two different transporters encoded by different genes are each responsible for transporting that metabolite in only one direction, enzymes that can use NAD(H) or NADP(H) interchangeably to catalyze the same redox reaction), but reactions that meet these criteria are generally worth close examination to ensure that they should actually all exist as separate reactions.
Loading
Loading