Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parallelization option to CorrelationDecoder and CorrelationDistributionDecoder #738

Merged
merged 5 commits into from
Aug 8, 2022

Conversation

JulioAPeraza
Copy link
Collaborator

Closes None.

Currently, CorrelationDecoder takes a long time to be trained. Depending on the number of studies per feature, it takes around 13 min per feature (i.e., per iteration). For large databases like Neurosynth, which has 3228 features, it will take ~29 days. Adding the parallelization option reduces the computation time by a factor of n_cores.

Changes proposed in this pull request:

  • Add n_cores parameter to use for parallelization.

@JulioAPeraza JulioAPeraza added the enhancement New feature or request label Aug 3, 2022
@JulioAPeraza JulioAPeraza requested a review from tsalo August 3, 2022 14:47
@codecov
Copy link

codecov bot commented Aug 3, 2022

Codecov Report

Merging #738 (79a8c95) into main (87c3ce3) will increase coverage by 0.01%.
The diff coverage is 93.75%.

@@            Coverage Diff             @@
##             main     #738      +/-   ##
==========================================
+ Coverage   88.60%   88.61%   +0.01%     
==========================================
  Files          40       40              
  Lines        4449     4455       +6     
==========================================
+ Hits         3942     3948       +6     
  Misses        507      507              
Impacted Files Coverage Δ
nimare/decode/continuous.py 94.79% <93.54%> (+0.34%) ⬆️
nimare/decode/base.py 91.11% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Member

@tsalo tsalo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@JulioAPeraza
Copy link
Collaborator Author

I would like to run another test before we merge this. The attribute self.images_ was returned empty in one of the tests I ran yesterday.

@tsalo
Copy link
Member

tsalo commented Aug 4, 2022

Actually, could you also fix the type at:

LGR.info(f"Retaining {len(features)}/({n_features_orig} features.")

There's an extra opening parenthesis that should be removed, but I don't think it's worth a separate PR.

@JulioAPeraza
Copy link
Collaborator Author

Sure!
Could we also change the vocab in the fetch_neuroquery example to neuroquery6308 in this PR?

@tsalo
Copy link
Member

tsalo commented Aug 4, 2022

That would be great! Thanks!

@tsalo
Copy link
Member

tsalo commented Aug 4, 2022

I don't know why the linter only started complaining in this PR, given that you didn't touch the problematic code, but it definitely is correct. The flagged line is definitely too long:

"""Create complementary binary images to identify areas of likely significance and nonsignificance.

@JulioAPeraza
Copy link
Collaborator Author

I tried splitting the summary line into two lines and it also failed the Style check test (see the last commit). Do you think we should shorten the summary sentence?

@tsalo
Copy link
Member

tsalo commented Aug 5, 2022

That's correct. Summary sentences need to fit on one line. I think you can just drop the word "complementary" and it will fit.

@JulioAPeraza
Copy link
Collaborator Author

I think this is ready to merge. I ran a few tests locally and self.images_ now contains the meta-analytic maps.

Copy link
Member

@tsalo tsalo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Feel free to merge.

@JulioAPeraza JulioAPeraza merged commit a4a7d16 into neurostuff:main Aug 8, 2022
@JulioAPeraza JulioAPeraza deleted the parallel-decode branch August 8, 2022 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants