clpipe 1.9.1

1.9.1 (Dec 7, 2023)

Enhancements

• convert2bids - This command will now detect whether dcm2bids or heudiconv is being used based on the extension type of the conversion config file. dcm2bids is used when a .json file is given, and heudiconv is used when .py. The -heudiconv
• clpipe - When creating a new project, a .clpipe folder is now created which houses
  a batch_config.json file. This allows users to easily update batch config defaults
  without having to create their own batch_config.json file. This file is referenced
  from the users standard clpipe_config.json file. The .clpipe folder will
  be used in the future to store any additional files that apply to an entire project.
• clpipe - Stack trace is now hidden again when -debug is not being used, like in
  older versions of clpipe (< v1.6)
• postprocess - Indexing / query time slightly improved through use of pattern-based indexing exceptions

Bug Fixes

• postprocess - Image scrubbing node now properly persists header information
• bids_validate - Fixed issue where SLURM logs did not save to log folder after 1.9
• preprocess - Fixed issue where SLURM logs did not save to log folder
  after 1.9
• N_threads not being passed in from the configuration file when running preprocessing.
• Fix a bug where email addresses were not being properly formatted in slurm command.
• Update mem_use in ROI extract to use correct config value.
• Add transform method to convert between snake_case and camelCase to support camelCase legacy. Supposed to be slowly deprecated and replaced with snake_case eventually, but this acts as a bridge in the meanwhile.
• Fix mem/threads/time usage in job manager.
• Update custom atlas help documentation to say "sphere" instead of "spheres".

Development

• job_manager - Introduced JobManagerFactory for streamlined job manager selection and instantiation to replace the old BatchManager with BatchJobManager and LocalJobManager, enhancing job submission flexibility and configuration options.
• job_manager - Upgraded job submission mechanism from os.system to subprocess.run for better process handling and output capture, especially when jobs are run locally.

Documentation

• Updated flywheel_sync.rst to include links and a section to use flywheel_sync with an example script.
• Added an image and more text to bids_convert.rst.
• Updated dicom2bids_tutorial.md to add more helpful information and hyperlinks.

clpipe 1.9.0

1.9.0 (Sep 15, 2023)

Enhancements

• postprocess - postprocess2 is now named postprocess, replacing the original postprocessing functionality
• postprocess - Wildcards can now be used in the target_variables of the scrub configuration to select multiple columns with similar names, such as non_steady_state_outliers*
• postprocess - BIDS index now saves to the user's working directory
• postprocess - Logs now saved in folders according to stream, like the output and working directories
• postprocess - Distributor-level slurm jobs removed, simplifying job structure and removing the distributor log folder
• postprocess - Individual images now get their own rolling log files
• postprocess - Slurm output files are now saved to a separate slurm_out folder
• postprocess - There are now a default processing streams setup and named specifically for the GLM and functional connectivity analyses
• glm - GLM setup command now completely removed, in favor of using the GLM postprocessing stream
• documentation - Expanded documentation for Postprocessing section
• documentation - Sections reorganized and made more consistent with each other.
• project_setup - Now prompts for name of project if not given.
• config update - Converts config to new format and offers to backup old version

Bug Fixes

• postprocess - Fixed an issue where postprocess took excessively long to index large datasets due to a bug in pybids
• postprocess - Issue where streams did not properly update postprocessing config fixed
• documentation - path to HNG modules directory corrected - missing the 's' at the end

Deprecations & Removals

• postprocess - Removed original postprocessing command
• postprocess - Removed original susan command; now a step of postprocess
• postprocess - Disabled fmri-process-check report for now, pending rework
• postprocess - The stream file "processing_description.json" has been moved to the stream working directory and is now called "run_config.json"

Dependencies

• convert2bids - dcm2bids v2.1.9 now enforced to prevent update to v3, which uses a newer version of BIDS incompatible with fMRIPrep <= v23
• convert2bids - heudiconv v12.2 now enforced to prevent update to v13, which for mysterious reasons does not work in our environment
• other - all other top-level dependencies locked in for now to prevent breaking updates

Development

• configuration - Dataclass-based configuration has been applied to all major clpipe commands
• configuration - Configuration is now serialized/deserialized with Marshmallow, which allows both JSON and YAML file types to be used
• postprocess - Global workflow now constructs image and confounds workflows on its own

clpipe 1.8.1

1.8.1 (Jun 28, 2023)

Enhancements

• postproc2 - Added new step ScrubTimepoints, allowing timepoints to be removed from the image and confounds where a set threshold is exceeded for a chosen confound variable.
• postproc2 - Wildcard statements such as t_comp_cor* can now be used in the Columns section of ConfoundsOptions to select multiple columns at once.
• postproc2 - Added special case to Temporal Filtering, which will interpolate over any values removed in the ScrubTimepoints step. See the documentation at Postprocessing/postprocess2/Processing Step Options/Temporal Filtering

Development

• clpipe - Contribution guide is now in its own markdown file separate from the setup guide. More details added to guide
• tests - New helper added for plotting timeseries data
• tests - Longer 100 timepoint sample image added for steps like Temporal Filtering that need more data for effective testing

clpipe 1.8.0

GLM Setup Command Deprecation

• glm setup: command deprecated, to be replaced by postproc2
• glm setup: config file no longer contains GLMSetupOptions. TaskName, ReferenceImage, and ParentClpipeConfig options previously in GLMSetupOptions have been moved to the top level of the configuration file, as they are still used by Level 1 & 2 setups.
• glm setup: Will still run "classic" glm setup pipeline when using a clpipe < 1.8 style glm config file, but prints a warning. Will print deprecation error and exit if using new-style glm setup config file
• project_setup: The default TargetDirectory and ConfoundDirectory of the default glm config file now point to postproc2/default
• project_setup: The default folder data_GLMPrep is no longer created
• project_setup: Log folders logs/glm_logs/L1_launch and logs/glm_logs/L2_launch are created instead of glm_setup_logs, and these path are auto-populated in the LogDir fields of the default glm config file

Enhancements

• preprocess: /work added to the list of UNC bind paths, as /pine is being deprecated by Longleaf
• preprocess: when templateflow toggle is on, this step now automatically creates a .cache/templateflow folder for you in your home directory if it doesn't already exist
• glm prepare: Changed some message exclusive to the -debug flag to be viewable without the flag to improve verbosity of the command. Command also now gives a message when completed, won't run if no images are found, and won't run if the EV target folder doesn't exist
• clpipe: The bids_validate command was moved out of the bids sub-command and moved to the top level to make it easier to find/use and more closely resemble the original clpipe command. The bids sub-command is still useable, but has been hidden, and will be removed in a future update
• clpipe: Similar to above, the dicom command has been hidden, and its sub-commands convert2bids and flywheel_sync have been moved to the top level of the clpipe menu. The dicom command is still accessible, but hidden.
• clpipe: The setup command has been renamed project_setup for consistency with the original command, and to avoid a conflict where a general setup command for clpipe might be necessary (as opposed to setting up a project)
• clpipe: The double-dash form --help and --version options have been removed for consistency with the other commands. The short hand of help, -h, is removed for simplicity. The forms -help and -version, -v remain.
• project_setup: Several fields of config and glm_config files that need to be set by user, but appear blank in the starter config files, are now populated like "SET THIS FIELD" to make it clearer that they must be set
• clpipe: New documentation page "Overview" added to house top-level cli command info and future location for clpipe overview diagrams

Bug Fixes

• clpipe: Fixed issue where username for clpipe.log file was not obtainable from a desktop node, raising an exception
• glm report_outliers: Fixed issue where outputs were doubled
• project_setup: Fixed bug where the L2 fsfs dir was not auto-populated in the default glm_config file

Development

• setup: Decoupled creation of files at project_setup from the path setup of user's config file, paving the way to update the configuration system and making it easier to control when directory setup occurs.
• tests: Added fixture to allow testing for backwards compatibility with fmriprep < v21 style directory structure
• CI/CD: Added a modulefile generation script to support deploying modules automatically
• CI/CD: Updated build and deploy scripts to support automatic deployment of modules

clpipe v1.7.3

Enhancements

• setup: the "SourceOptions" block for dicom flywheel_sync is now included in the default configuration file
• setup: more modality examples added to conversion_config.json starter file (T1w and fmap)
• setup: conversion_config.json starter file now includes dcm2niix customization line which sets its search depth to its maximum value, allowing dcm2bids to work with flywheel's DICOM sync directory
• setup: the default .bidsignore file now includes scans.json, a file generated by heudiconv which would cause validation to fail otherwise
• bids validate: In clpipe_config.json, Moved BIDSValidatorImage from PostprocessingOptions block to BIDSValidationOptions block. The command will still look for the image in its old location if it can't find it in BIDSValidationOptions, maintaining backwards compatibility with < 1.7.3 config files
• preprocess: Leaving the [TemplateFlowPath] empty no longer results an unhelpful message
• glm prepare: improved logging messages
• glm prepare : changed how file copying works so only file contents are copied, not their permissions, making the command easier to run in a shared environment
• clpipe: -config_file/-c is now a required argument in most commands to prevent unhelpful error output if no config file given, and to steer clpipe more towards being configuration file driven
• clpipe: ignore writing to clpipe.log if the user does not have permission to do so
• clpipe: clpipe.log file now includes usernames
• clpipe: clpipe.log file is written with group write permission by default

Bug Fixes

• setup: generated glm_config.json file's [GLMSetupOptions][LogDirectory] field is now automatically filled out. Previously it was left blank, despite the appropriate log folder being automatically created, and would throw an error in the batch manager if not filled out manually.
• reports fmriprep: fixed issue where the main .html report files were not being bundled into the output zip
• glm prepare: fixed issue where FileNotFoundErrors were not caught correctly, causing the program to exit earlier than intended

clpipe v1.7.2

Flywheel Sync

clpipe can now be used to sync DICOM data from Flywheel. Using Flywheel through clpipe allows the user to store their project's remote Flywheel path and download destination in their clpipe configuration file, and to automatically submit Flywheel sync commands to a cluster as batch jobs. clpipe also cleans up an oddly-named, empty temporary directory that is left behind when running Flywheel's sync command.

This feature is accessed with the command clpipe dicom flywheel_sync. The clpipe subcommand clpipe dicom was added to give this command a "home" and to house future dicom-specific commands.

clpipe dicom flywheel_sync -c /path/to/your/clpipe_config.json -submit

GLM

• combined l1_prepare_fsf and l2_prepare_fsf into the clpipe glm prepare command to match how the launch command works:

> clpipe glm prepare
Usage: clpipe glm prepare [OPTIONS] LEVEL MODEL

  Propagate an .fsf file template for L1 or L2 GLM analysis.

  LEVEL is the level of anlaysis, L1 or L2

  MODEL must be a a corresponding L1 or L2 model from your GLM configuration
  file.

Options:
  -glm_config_file FILE  The path to your clpipe configuration file.
                         [required]
  -debug                 Flag to enable detailed error messages and traceback.
  -help, -h, --help      Show this message and exit.

• l1_prepare_fsf no longer leaves an underscore at the end of the .fsf file names or the feat output directories
• L1 and L2 config files now support LogDirectory options for specifying where to send the output of the launch command. The output folder is used by default if no LogDirectory is specified.
• Improved error handling for clpipe glm prepare command

clpipe bids convert

• Command has been moved from clpipe bids to the new clpipe dicom, which better reflects the BIDS conversion operation (a command acting on DICOM data to convert it to BIDS format). It has also been renamed from from convert back to its original name, convert2bids, to improve the clarity of this command's name.

Usage now looks like this:

clpipe dicom convert2bids -c path/to/my/clpipe_config.json 256 -submit

postproc2

• now supports either fmriprep v20 or v21's directory structure
• no longer deletes "processing_graph.dot" (source of "processing_graph.png" to avoid a race condition issues sometimes causing jobs to fail

get_reports

Command get_reports, used for combining fMRIPrep QC report outputs into a ZIP archive:

• has been added to the main clpipe menu as clpipe reports fmriprep
• now prints more log messages
• creates an archive with a shorter directory path
• archive is now named "fMRIPrep_Archive" by default

Other Updates

• Default version of fMRIPrep referenced in config updated to v21.0.2
• Shorthand commands (e.g. -c for config_file, -s for submit) have been made consistent across all commands under the clpipe command
• clpipe postproc no longer fails silently when no subjects are found - error is now raised

clpipe v1.7.1

CLI Updates

• Dramatically improved clpipe CLI speed by optimizing import order
• All commands now print a help message with no arguments are given
• ROI commands added to the clpipe CLI

> clpipe roi
Usage: clpipe roi [OPTIONS] COMMAND [ARGS]...

  Region of Interest (ROI) Commands.

  Please choose one of the commands below for more information.

Options:
  -help, -h, --help  Show this message and exit.

Commands:
  atlases  Display all available atlases.
  extract  Extract ROIs with a given atlas.

glm apply_mumford

• added an option for deleting reg_standard folders, set to False by default
• fixed a bug that was not allowing the GLM config file to be used over a direct path to the .feat folders

CLI demo

cli-demo.mov

clpipe v1.7.0

CLI Updates

clpipe's subcommands have been reorganized / renamed to make them (hopefully) more intuitive. Please note that the original commands (e.g. project_setup, convert2bids, etc.) still remain and probably wouldn't be removed without several versions containing deprecation warnings, perhaps in prelude to clpipe 2.0.0

• ordering of main clpipe menu now reflects order of processing steps (e.g. setup, bids, preprocess, postprocess)
• ordering of glm sub-menu now reflects order of processing steps (e.g. setup, fsl_onset_extract, l1_prepare_fsf, etc.)
• clpipe sub-commands "convert2bids" and "bids_validate" are now part of the clpipe sub-command "bids" as "convert" and "validate," respectively
• clpipe sub-command "fmriprep_process" renamed to "preprocess" to generalize the step
• clpipe sub-command "project_setup" shortened to "setup"
• clpipe sub-command "apply_mumford_workaround" shortened to "apply_mumford"

Usage: clpipe [OPTIONS] COMMAND [ARGS]...

  Welcome to clpipe. Please choose a processing command.

Options:
  -v, --version  Display clpipe's version.
  --help         Show this message and exit.

Commands:
  setup         Set up a clpipe project
  bids          BIDS Commands
  preprocess    Submit BIDS-formatted images to fMRIPrep
  postprocess   Additional preprocessing for connectivity analysis
  postprocess2  Perform additional processing on fMRIPrepped data
  glm           GLM Commands
  status        Check the status of your project

Usage: clpipe glm [OPTIONS] COMMAND [ARGS]...

  GLM Commands

Options:
  --help  Show this message and exit.

Commands:
  setup              Prepare task images and confound files for GLM analysis
  fsl_onset_extract  Convert onset files to FSL's 3 column format
  l1_prepare_fsf     Propagate an .fsf file template for L1 GLM analysis
  launch             Launch all prepared .fsf files for L1 or L2 GLM...
  apply_mumford      Apply the Mumford registration workaround to L1 FEAT...
  l2_prepare_fsf     Propagate an .fsf file template for L2 GLM analysis
  report_outliers    Generate a confound outliers report.

Return of Heudiconv

Heudiconv has returned to clpipe as an alternative to dcm2bids for handling more complicated BIDS conversions that cannot be adequately handled with a conversion_config.json file.

It is accessed through the same command as dcm2bids, clpipe bids convert

Example (note the abbreviate forms of -config_file and submit to -c and -s, respectively):

clpipe bids convert -heudiconv -c path/to/my/config.json -s

The heuristic.py file is given in the same place that your conversion_config file would go in the DICOMToBIDSOptions:

"DICOMToBIDSOptions": {
		...
		"ConversionConfig": "/path/to/project/heuristic.py",
		...

Update to Python 3.7

To support heudiconv v11, an update to Python > 3.7 was necessary. This is also the recommended minimum version for nypipe, a large dependency of clpipe.

Misc Updates

bids convert

• Fix a bug where values in the configuration file were being updated with values given in the CLI options
• Command now takes multiple subjects as arguments instead of the previous -subject option. The -subject option now warns users that they can use subject arguments. This brings its functionality in line with how the postprocessing scripts handle subjects.

clpipe bids convert 111 222 -config_file /path/to/project/clpipe_config.json -submit

glm apply_mumford

• Automatically create reg folder if it doesn't exist (such as when you don't use FEAT for preprocessing)
• Added debug option
• Increased verbosity of outputs

glm l1_prepare_fsf

• Added additional outputs
• No longer eats uncaught errors when not in debug mode

postproc2

• Cleaned up folder organization of log files for distributed tasks
• Defer creation of folders until necessary, to prevent empty folders from being created when an error occurs before processing starts
• "Debug" option now propagates down to distributed jobs
• ApplyAROMA implementation "fsl_regfilt_R" is now platform independent (previously the R package installation path was hard-coded to work with UNC longleaf)

Other

• Fix a bug in bids validate where the submission message was double printed
• Added output messages / logging to project setup command
• Updated documentation to use auto-generated click CLI documentation
• Added documentation page for unified CLI
• Add a simple bash build script

clpipe v1.6.0

CLI Update

A subset of commands have been gathered under one umbrella command, clpipe. Running the clpipe command with no arguments will show the user the various processing steps provided by clpipe:

Usage: clpipe [OPTIONS] COMMAND [ARGS]...

  Welcome to clpipe. Please choose a processing command.

Options:
  -v, --version  Display clpipe's version.
  --help         Show this message and exit.

Commands:
  bids_validate      Check that the given directory conforms to the BIDS...
  convert2bids       Convert DICOM files to BIDS format
  fmri_postprocess   Additional preprocessing for connectivity analysis
  fmri_postprocess2  Perform additional processing on fMRIPrepped data
  fmriprep_process   Submit BIDS-formatted images to fMRIPrep
  glm                GLM Commands
  project_setup      Set up a clpipe project

The command bids_validate, for example, can be accessed like this:

clpipe bids_validate --help

Usage: clpipe bids_validate [OPTIONS] [BIDS_DIR]

  Check that the given directory conforms to the BIDS standard

Options:
  -config_file FILE  Uses a given configuration file
  -log_dir FILE      Where to put HPC output files (such as SLURM output
                     files)
  -verbose           Creates verbose validator output. Use if you want to see
                     ALL files with errors/warnings.
  -submit            Flag to submit commands to the HPC
  -interactive       Run in an interactive session. Only use in an interactive
                     compute session.
  -debug             Flag to enable detailed error messages and traceback
  --help             Show this message and exit.

The GLM commands have been grouped into the sub-command glm, and can be accessed like this:
clpipe glm

Usage: clpipe glm [OPTIONS] COMMAND [ARGS]...

  GLM Commands

Options:
  --help  Show this message and exit.

Commands:
  apply_mumford_workaround  Apply the Mumford registration workaround to...
  fsl_onset_extract         Convert onset files to FSL's 3 column format
  l1_prepare_fsf            Propagate an .fsf file template for L1 GLM...
  l2_prepare_fsf            Propagate an .fsf file template for L2 GLM...
  launch                    Launch all prepared .fsf files for L1 or L2...
  report_outliers           Generate a confound outliers report.
  setup                     Prepare task images and confound files for...

Further clpipe commands will be wrapped into this interface with future updates.
The interface currently does not support auto-complete for sub commands - this is possible, and can be included in a future release.
The stand-alone commands such as bids_validate remain available for use outside of the clpipe command like in previous versions of clpipe.

GLM Updates

GLM Launchers

Launchers have been added clpipe's GLM line, allowing .fsf files generated with l1_prepare_fsfs and l2_prepare_fsfs to be launched automatically using your GLM configuration.

Usage: glm_l1_launch [OPTIONS]

  Launch all prepared .fsf files for L1 GLM analysis

Options:
  -glm_config_file FILE  Use a given GLM configuration file.  [required]
  -l1_name TEXT          Name of your L1 model  [required]
  -test_one              Only submit one job for testing purposes.
  -submit                Flag to submit commands to the HPC
  -debug                 Flag to enable detailed error messages and traceback
  --help                 Show this message and exit.

Usage: glm_l2_launch [OPTIONS]

  Launch all prepared .fsf files for L2 GLM analysis

Options:
  -glm_config_file FILE  Use a given GLM configuration file.  [required]
  -l2_name TEXT          Name of your L2 model  [required]
  -test_one              Only submit one job for testing purposes.
  -submit                Flag to submit commands to the HPC
  -debug                 Flag to enable detailed error messages and traceback
  --help                 Show this message and exit.

Stand-alone Mumford Workaround Command

The logic of the Mumford Workaround (see here), normally applied by clpipe at L2, can now be applied to an L1 folder manually. This helps with L3 issues in the case where some subjects have L2, and some don't, causing the subjects with only L1s to not have the Mumford Workaround applied.

clpipe glm apply_mumford_workaround --help

Usage: clpipe glm apply_mumford_workaround [OPTIONS]

  Apply the Mumford registration workaround to L1 FEAT folders.  Applied by
  default in glm-l2-preparefsf.

Options:
  -glm_config_file FILE           Location of your GLM config file.
  -l1_feat_folders_path DIRECTORY
                                  Location of your L1 FEAT folders.
  --help                          Show this message and exit.

Outlier Reporting

A new command has also been provided to report on the number of outliers each subject has after running through glm_setup or fmri_postprocess2. The logic of this command will likely be integrated into fmri_postprocess2 in a future update, but it is available as a stand-alone command for now as:

clpipe glm report_outliers --help

Usage: clpipe glm report_outliers [OPTIONS]

  Generate a confound outliers report.

Options:
  --confounds_dir DIRECTORY  Path to a directory containing subjects and
                             confounds files.
  --confounds_file FILE      Path to confounds file
  --output_file FILE         Path to save outlier count results.
  --confound_suffix TEXT     Confound file to search for, like 'confounds.tsv'
  --help                     Show this message and exit.

Logging & Display Updates

New Combined-view Log File

A new log file, logs/clpipe.log, will now be generated that shows the outputs of main line (setup, convert2bids, validate, fmriprep, postproc) commands all in one place. This makes it easier to help track what commands have been previously run for your project.

General Verbosity Improvement

The amount of information displayed by all of the main line commands has been expanded, both in and out of debug mode, making it easier to tell where your outputs are going and what particular settings have been used.

Here is an example from the L2 launcher:

Setting up L2 .fsf launch using model: emot
Targeting .fsfs in dir: ...projects/MAP/data_GLM/data_GLM_emot/level2_emot/design_files
Output dir: .../projects/MAP/data_GLM/data_GLM_emot/level2_emot/gfeat
Using log dir: .../MAP/data_GLM/data_GLM_emot/level2_emot/gfeat
Batch job output path: .../MAP/data_GLM/data_GLM_emot/level2_emot/gfeat

These log lines are also printed to the logs/clpipe.log file

Improved Submission Preview

The submission preview display, shown when running most commands without submit, has been improved to be more readable. Tabs and new lines space out the individual commands, and additional commands beyond the first 5 are folded up as a simple tally. You can still see all commands by including -debug.

Jobs to run:

        sbatch --no-requeue -n 1 --mem=5G --time=5:00:00 --cpus-per-task=4 --job-name="sub-0081" --output=.../data_GLM/data_GLM_emot/level2_emot/gfeat/Output-sub-0081-jobid-%j.out --wrap="feat .../data_GLM/data_GLM_emot/level2_emot/design_files/sub-0081.fsf"

        sbatch --no-requeue -n 1 --mem=5G --time=5:00:00 --cpus-per-task=4 --job-name="sub-0132" --output=.../data_GLM/data_GLM_emot/level2_emot/gfeat/Output-sub-0132-jobid-%j.out --wrap="feat .../data_GLM/data_GLM_emot/level2_emot/design_files/sub-0132.fsf"

        sbatch --no-requeue -n 1 --mem=5G --time=5:00:00 --cpus-per-task=4 --job-name="sub-1011" --output=.../data_GLM/data_GLM_emot/level2_emot/gfeat/Output-sub-1011-jobid-%j.out --wrap="feat .../data_GLM/data_GLM_emot/level2_emot/design_files/sub-1011.fsf"

        sbatch --no-requeue -n 1 --mem=5G --time=5:00:00 --cpus-per-task=4 --job-name="sub-0231" --output=.../data_GLM/data_GLM_emot/level2_emot/gfeat/Output-sub-0231-jobid-%j.out --wrap="feat .../data_GLM/data_GLM_emot/level2_emot/design_files/sub-0231.fsf"

        sbatch --no-requeue -n 1 --mem=5G --time=5:00:00 --cpus-per-task=4 --job-name="sub-0201" --output=.../data_GLM/data_GLM_emot/level2_emot/gfeat/Output-sub-0201-jobid-%j.out --wrap="feat .../data_GLM/data_GLM_emot/level2_emot/design_files/sub-0201.fsf"

        ...and 3 more job(s).
Re-run with the '-submit' flag to launch these jobs.

Under the Hood Changes

• Main line commands no longer eat uncaught exceptions when not in debug mode
• Logging is now initialized at the top init.py file
• Each main line command is now a sub-logger of the main clpipe logger, including batch-manager
• Each of the main line commands have been refactored for readability, including shortening the line lengths and moving constants to the top of the module.
• Created centralized configuration module

clpipe 1.5.0 Release: New Command fmri_postprocess2

This update combines the functionalities of glm_setup and fmri_postprocess into one command, fmri_postprocess2*

fmri_postprocess2 also provides the ability to customize which processing steps to use, and in which order. These configurations and their outputs can be handled as independent pipelines using the processing streams feature.

Image postprocessing steps provided by fmri_postprocess2:

• Temporal Filtering
• Intensity Normalization
• Spatial Smoothing
• AROMA Regression
• Confound Regression
• Timepoint Trimming
• Masking

In addition to processing images, fmri_postprocess2 provides a processing pipeline for each image's corresponding confound file. When enabled, fmri_postprocess2 will apply any temporally relevant postprocessing steps to the confounds that were applied to the images. It also can automatically provide spike regressors to replicate glm_setup's functionality.

fmri_postprocess2 is implemented completely with nipype, which, among other features, provides a framework for automatic parallelization of processing steps and caching for quick-reruns of pipelines. Nipype also makes it easier to implement additional processing steps. Additionally, nipype allows for visualization of processing streams, and some examples are provided below:

An example of a pipeline which performs smoothing, AROMA regression, temporal filtering, and normalization. Note how the temporally relevant processing steps are also applied to the confounds, in the same order:

An example of a pipeline providing only normalization and masking, as well as providing spike regressors:

• fmri_postprocess2 will replace fmri_postprocess and deprecate glm_setup in v2.0.0