Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: addition of carbon trackers to estimate compute costs #2834

Merged
merged 37 commits into from
Dec 3, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
c2303f1
prelim tests with CodeCarbon complete
nikhil153 Mar 22, 2022
9471fdf
Merge branch 'nipreps:master' into carbon-trackers
nikhil153 Mar 22, 2022
e953e47
added basic readme for running fmriprep with carbon trackers
nikhil153 Mar 22, 2022
44b9a12
added basic readme for running fmriprep with carbon trackers
nikhil153 Mar 22, 2022
eaac8ae
tracker test with sudo complete
nikhil153 Mar 22, 2022
b5ba399
Update carbon_trackers_readme.md
nikhil153 Mar 22, 2022
28e0314
added explicit country-code option
nikhil153 Mar 22, 2022
e4ccf41
Merge branch 'carbon-trackers' of https://github.com/nikhil153/fmripr…
nikhil153 Mar 22, 2022
b96d20b
Update carbon_trackers_readme.md
nikhil153 Mar 22, 2022
8c29941
added arg flags and completed test with CodeCarbon v2.1.2
nikhil153 Jun 12, 2022
65236bc
updated README for BrainHack
nikhil153 Jun 12, 2022
3916810
Update carbon_trackers_readme.md
nikhil153 Jun 12, 2022
a9e98f4
fixed typos and added visual notebook
nikhil153 Jun 13, 2022
c0a84d6
added sample freesurfer emissions file
nikhil153 Jun 13, 2022
44aaa5e
Update carbon_trackers_readme.md
nikhil153 Jun 15, 2022
8c30d87
tried interactive widget for the demo
nikhil153 Jun 15, 2022
53f2f14
completed basic ohbm demo
nikhil153 Jun 19, 2022
f1baeaa
upgraded fmriprep version
nikhil153 Aug 9, 2022
afc4dad
completed test with fmriprepv22.0.0
nikhil153 Aug 9, 2022
f7bb04b
completed test with fmriprepv22.0.0
nikhil153 Aug 9, 2022
4a293de
Update README.md
nikhil153 Aug 9, 2022
553bc78
fixed pep errors
nikhil153 Aug 9, 2022
bb7b633
fixed pep errors
nikhil153 Aug 9, 2022
c8b9d8c
Update setup.cfg
nikhil153 Aug 9, 2022
23e3666
Merge branch 'master' into carbon-trackers
nikhil153 Aug 9, 2022
78d2a28
Update confounds.py
nikhil153 Aug 9, 2022
0cb9b14
updated contributors.json
nikhil153 Aug 10, 2022
99fb0bb
Merge branch 'carbon-trackers' of https://github.com/nikhil153/fmripr…
nikhil153 Aug 10, 2022
e954d63
Merge branch 'master' into carbon-trackers
effigies Oct 13, 2022
c27b6e1
Update fmriprep/cli/run.py
nikhil153 Nov 2, 2022
2301b75
Update fmriprep/cli/run.py
nikhil153 Nov 2, 2022
1b32e26
STY: black
effigies Nov 7, 2022
7a959f7
MNT: Ignore restyling commit
effigies Nov 7, 2022
178a263
Merge remote-tracking branch 'upstream/master' into carbon-trackers
effigies Nov 7, 2022
6f7f356
Merge remote-tracking branch 'upstream/master' into carbon-trackers
effigies Dec 3, 2022
f415af7
MNT: Remove working files
effigies Dec 3, 2022
142eee8
Revert .git-blame-ignore-revs
effigies Dec 3, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .maint/contributors.json
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@
"name": "Benson, Noah C.",
"orcid": "0000-0002-2365-8265"
},
{
"affiliation": "Montreal Neurological Institute, McGill University",
"name": "Bhagwat, Nikhil",
"orcid": "0000-0001-6073-7141"
},
{
"affiliation": "Perelman School of Medicine, University of Pennsylvania, PA, USA",
"name": "Cieslak, Matthew",
Expand Down
14 changes: 14 additions & 0 deletions fmriprep/cli/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -629,6 +629,20 @@ def _slice_time_ref(value, parser):
help="Use low-quality tools for speed - TESTING ONLY",
)

# Carbon tracker options
g_other.add_argument(
"--track-carbon",
action="store_true",
help="Tracks power draws using CodeCarbon package",
)
g_other.add_argument(
"--country-code",
action="store",
default="CAN",
type=str,
help="Country ISO code used by carbon trackers",
)

latest = check_latest()
if latest is not None and currentv < latest:
print(
Expand Down
21 changes: 21 additions & 0 deletions fmriprep/cli/run.py
100755 → 100644
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,20 @@ def main():

parse_args()

# Code Carbon
if config.execution.track_carbon:
from codecarbon import OfflineEmissionsTracker

country_iso_code = config.execution.country_code
config.loggers.workflow.log(25, "CodeCarbon tracker started ...")
config.loggers.workflow.log(25, f"Using country_iso_code: {country_iso_code}")
config.loggers.workflow.log(25, f"Saving logs at: {config.execution.log_dir}")

tracker = OfflineEmissionsTracker(
output_dir=config.execution.log_dir, country_iso_code=country_iso_code
)
tracker.start()

if "pdb" in config.execution.debug:
from fmriprep.utils.debug import setup_exceptionhook

Expand Down Expand Up @@ -179,6 +193,13 @@ def main():
finally:
from pkg_resources import resource_filename as pkgrf

# Code Carbon
if config.execution.track_carbon:
emissions: float = tracker.stop()
config.loggers.workflow.log(25, "CodeCarbon tracker has stopped.")
config.loggers.workflow.log(25, f"Saving logs at: {config.execution.log_dir}")
config.loggers.workflow.log(25, f"Carbon emissions: {emissions} kg")

from fmriprep.reports.core import generate_reports

# Generate reports phase
Expand Down
4 changes: 4 additions & 0 deletions fmriprep/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,10 @@ class execution(_Config):
"""Do not convert boilerplate from MarkDown to LaTex and HTML."""
notrack = False
"""Do not monitor *fMRIPrep* using Sentry.io."""
track_carbon = False
"""Tracks power draws using CodeCarbon package."""
country_code = "CAN"
"""Country ISO code used by carbon trackers."""
output_dir = None
"""Folder where derivatives will be stored."""
me_output_echos = False
Expand Down
3 changes: 3 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ install_requires =
tedana ~= 0.0.9
templateflow >= 0.6
toml
codecarbon
APScheduler

test_requires =
coverage
codecov
Expand Down