ENH: b0 reference and skullstrip workflow #25

josephmje · 2019-10-16T14:33:18Z

My previous workflow used MRtrix's dwibiascorrect but for some images, dwi2mask, which is called by dwibiascorrect would produce a mask around the cerebellum and only that region would be enhanced. This PR adapts init_bold_reference_wf from niworkflows.

However, I noticed that the mask produced by BET in the skullstrip_first_pass node includes every voxel. The output is fine but not sure if this is also a problem in niworkflows.

oesteban

Had a first pass. I still need to go through the workflow.

dmriprep/interfaces/images.py

josephmje · 2019-10-22T18:31:35Z

I'll modify the workflow to use the new dwi.tsv file instead of bvec and bval.

oesteban

This is looking good. I particularly like that you documented the workflows very well. Some further comments:

Please consider the more robust calculation of the final B0 map that I suggested.
We need to add some tests
We need to generate the reportlet so that we can refer future contributors to this PR when they ask about the reports system.

dmriprep/workflows/dwi/util.py

oesteban · 2019-10-22T18:50:53Z

dmriprep/workflows/dwi/util.py

+    :abbr:`INU (intensity non-uniformity)` bias field and calculates a signal
+    mask.
+
+    Steps of this workflow are:


I've been thinking about this, and I would like to hear opinions (@arokem, @jelleveraart) about a more robust b0 computation:

Using the rough mask (I'll call it pre_mask) calculated before, what about recalculating the reference as follows:

bzeros = dwi_data[..., b0vals_mask] bzeros = 1000 * bzeros / bzeros[pre_mask, ...].mean(axis=-1) # Normalize to have mean intensity of 1000 within mask b0map = np.median(bzeros, axis=-1) # median should be more robust than average, esp. with few bzeros

The first rescaling is to deal with the average signal decay you generally see when b zeros are distributed along the scan. We could also store the average signal series (that bzeros[pre_mask, ...].mean(axis=-1)) to have an estimation of signal drop and maybe allow some exponential decay fitting down the line.

We could also robustify that calculation using median instead of mean.

Would it also be worth registering each b0 to the first in case there is significant motion between them? I'm not sure what opinions are on this.

Yes. I would start by registering all the b0 to each other, and then registering the others to the mean of these registered b0 volumes.

I've temporarily gotten FSL mcflirt setup but would this be something handled by antsMultivariateTemplateConstruction?

oesteban · 2019-11-04T17:10:03Z

@josephmje can you rebase to upstream/master?

pep8speaks · 2019-11-04T17:29:31Z

Hello @josephmje, Thank you for updating!

Cheers! There are no style issues detected in this Pull Request. 🍻 To test for issues locally, pip install flake8 and then run flake8 dmriprep.

Comment last updated at 2019-12-16 16:38:28 UTC

oesteban · 2019-11-04T17:31:53Z

mmm this doesn't look quite right, how have you done the rebase?

josephmje · 2019-11-04T17:32:28Z

mmm this doesn't look quite right, how have you done the rebase?

I used:

git fetch upstream
git rebase upstream/master

oesteban · 2019-11-04T17:32:59Z

can you post: git remote -v?

josephmje · 2019-11-04T17:33:38Z

can you post: git remote -v?

origin	https://github.com/josephmje/dmriprep.git (fetch)
origin	https://github.com/josephmje/dmriprep.git (push)
upstream	https://github.com/nipreps/dmriprep.git (fetch)
upstream	https://github.com/nipreps/dmriprep.git (push)

oesteban · 2019-11-04T17:35:23Z

Weird, did you have to force-push?

josephmje · 2019-11-04T17:36:01Z

Nope, I pushed normally. But I see now, it's including everything, not just my changes.

oesteban · 2019-11-04T17:39:42Z

Can you add me as a collaborator to your fork?

oesteban · 2019-11-04T17:45:44Z

I'm going to push to your branch. Before any kind of git-fetch or git-pull, could you follow these steps after I push?

git checkout -b enh/skullstrip-backup enh/skullstrip
git branch -D enh/skullstrip
git fetch
git checkout enh/skullstrip

josephmje · 2019-11-04T17:46:58Z

Thanks @oesteban . Looks much better. What was the solution (for future reference)?

oesteban · 2019-11-04T17:47:31Z

I believe you undid the rebase at some point, and merged some other branch in.

dmriprep/interfaces/images.py

dmriprep/workflows/dwi/util.py

dmriprep/interfaces/images.py

codecov-io · 2019-11-04T21:30:56Z

Codecov Report

Merging #25 into master will decrease coverage by 16.4%.
The diff coverage is 31.5%.

@@             Coverage Diff             @@
##           master      #25       +/-   ##
===========================================
- Coverage   40.57%   24.16%   -16.41%     
===========================================
  Files           9       11        +2     
  Lines         589      662       +73     
  Branches       92       94        +2     
===========================================
- Hits          239      160       -79     
- Misses        349      501      +152     
  Partials        1        1

Impacted Files	Coverage Δ
dmriprep/workflows/dwi/util.py	`20.93% <20.93%> (ø)`
dmriprep/interfaces/images.py	`46.66% <46.66%> (ø)`
dmriprep/cli/version.py	`16.98% <0%> (-75.48%)`	⬇️
dmriprep/cli/run.py	`7.39% <0%> (-21.84%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3c5d0f1...0e29500. Read the comment docs.

pull-assistant · 2019-11-26T22:48:58Z

Best reviewed: commit by commit

Optimal code review plan (3 warnings)

enh: initial implementation of b0 reference and skull-stripping workfl...

Update dmriprep/workflows/dwi/util.py

Update dmriprep/interfaces/images.py

update to use inputs according to #26

initial b0 registration adapted from niworkflows

dmriprep/workflows/dwi/util.py 73% changes removed in add mcflirt and fix ...

fix masking

dmriprep/workflows/dwi/util.py 47% changes removed in add mcflirt and fix ...

update docs and doctests

dmriprep/workflows/dwi/util.py 50% changes removed in docs: revise docstri...

add workflow test

add pytest fixtures

docs: revise docstrings for napoleon parsing

add mcflirt and fix rescale_b0 inputs and outputs

fix imputs again

Powered by Pull Assistant. Last update fb83460 ... dd4af8d. Read the comment docs.

oesteban · 2019-12-13T00:12:50Z

@josephmje the addition of documentation is starting to give us good feedback. The build_docs job is failing because some of the connections in the workflow are wrong:

Warning, treated as error:
/root/project/dmriprep/workflows/dwi/util.py:docstring of dmriprep.workflows.dwi.util.init_dwi_reference_wf:11:Exception occurred in plotting dmriprep-workflows-dwi-util-1
 from /root/project/docs/api/dmriprep.workflows.dwi.util.rst:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/nipype/sphinxext/plot_workflow.py", line 485, in run_code
    exec(code, ns)
  File "<string>", line 2, in <module>
  File "/root/project/dmriprep/workflows/dwi/util.py", line 118, in init_dwi_reference_wf
    ('outputnode.skull_stripped_file', 'ref_image_brain')]),
  File "/usr/local/lib/python3.7/site-packages/nipype/pipeline/engine/workflows.py", line 214, in connect
    '\n'.join(['Some connections were not found'] + infostr))
Exception: Some connections were not found
Module rescale_b0 has no input called pre_mask

Module rescale_b0 has no output called ref_image

I've force-pushed to your branch after rebasing. I believe that:

git checkout enh/skullstrip
git pull --rebase

should work for you.
Let me know if you find any problems.

initial port from niworkflows add initial pre-mask step fix doc add pre_mask address review comments add period extract b0s first fix function name typo initial port from niworkflows add initial pre-mask step fix doc add pre_mask address review comments add period extract b0s first fix function name typo remove MatchHeader

Co-Authored-By: Oscar Esteban <[email protected]>

oesteban · 2019-12-17T20:25:46Z

Okay, it seems to make more sense now:
https://163-168723324-gh.circle-artifacts.com/0/root/project/docs/_build/no_version_html/html/api/dmriprep.workflows.dwi.util.html#dmriprep.workflows.dwi.util.init_dwi_reference_wf

I'll check why THP002 is failing.

oesteban reviewed Oct 22, 2019

View reviewed changes

oesteban force-pushed the enh/skullstrip branch from 1a9ed45 to e6f9eb9 Compare November 4, 2019 17:45

oesteban reviewed Nov 4, 2019

View reviewed changes

oesteban mentioned this pull request Dec 12, 2019

DOC: Build versioned docs and deploy them to gh-pages #45

Merged

oesteban force-pushed the enh/skullstrip branch from 8608605 to 18b6a56 Compare December 12, 2019 23:27

josephmje and others added 8 commits December 12, 2019 22:11

Update dmriprep/workflows/dwi/util.py

b8a7548

Co-Authored-By: Oscar Esteban <[email protected]>

Update dmriprep/interfaces/images.py

ea01373

Co-Authored-By: Oscar Esteban <[email protected]>

Update dmriprep/interfaces/images.py

1e55d73

Co-Authored-By: Oscar Esteban <[email protected]>

update to use inputs according to nipy#26

4a2ca58

initial b0 registration adapted from niworkflows

43d6cd0

fix masking

e1afd66

update docs and doctests

1f31b65

josephmje and others added 3 commits December 12, 2019 22:12

add workflow test

7b5bbf6

add pytest fixtures

9632b04

docs: revise docstrings for napoleon parsing

215e936

oesteban force-pushed the enh/skullstrip branch from e076051 to 215e936 Compare December 13, 2019 06:12

josephmje added 2 commits December 16, 2019 11:25

add mcflirt and fix rescale_b0 inputs and outputs

0528f21

fix imputs again

dd4af8d

oesteban mentioned this pull request Jan 13, 2020

Concatenating different runs #43

Closed

josephmje closed this Jan 13, 2020

oesteban mentioned this pull request Jan 17, 2020

[WIP] Eddy Motion Correction (EMC) #62

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: b0 reference and skullstrip workflow #25

ENH: b0 reference and skullstrip workflow #25

josephmje commented Oct 16, 2019

oesteban left a comment

josephmje commented Oct 22, 2019

oesteban left a comment

oesteban Oct 22, 2019

oesteban Oct 22, 2019

josephmje Oct 22, 2019

arokem Oct 22, 2019

josephmje Nov 19, 2019

oesteban commented Nov 4, 2019

pep8speaks commented Nov 4, 2019 •

edited

Loading

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

codecov-io commented Nov 4, 2019

pull-assistant bot commented Nov 26, 2019 •

edited

Loading

oesteban commented Dec 13, 2019

oesteban commented Dec 17, 2019

ENH: b0 reference and skullstrip workflow #25

ENH: b0 reference and skullstrip workflow #25

Conversation

josephmje commented Oct 16, 2019

oesteban left a comment

Choose a reason for hiding this comment

josephmje commented Oct 22, 2019

oesteban left a comment

Choose a reason for hiding this comment

oesteban Oct 22, 2019

Choose a reason for hiding this comment

oesteban Oct 22, 2019

Choose a reason for hiding this comment

josephmje Oct 22, 2019

Choose a reason for hiding this comment

arokem Oct 22, 2019

Choose a reason for hiding this comment

josephmje Nov 19, 2019

Choose a reason for hiding this comment

oesteban commented Nov 4, 2019

pep8speaks commented Nov 4, 2019 • edited Loading

Comment last updated at 2019-12-16 16:38:28 UTC

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

oesteban commented Nov 4, 2019

josephmje commented Nov 4, 2019

oesteban commented Nov 4, 2019

codecov-io commented Nov 4, 2019

Codecov Report

pull-assistant bot commented Nov 26, 2019 • edited Loading

oesteban commented Dec 13, 2019

oesteban commented Dec 17, 2019

pep8speaks commented Nov 4, 2019 •

edited

Loading

pull-assistant bot commented Nov 26, 2019 •

edited

Loading