Scripts: Help pages and algoritms
Hi all,
I’ve just pushed some more changes to the Python scripts and scripting library provided with MRtrix3. I thought that in addition to highlighting those changes, I’d also take the chance to draw attention to some of the work I’ve done in the past related to script algorithms, which may interest some users looking to implement their own processing scripts or integrate their own personalized approaches into MRtrix3 workflows.
Script help pages
Apart from fixing a few bugs here and there, the main change I’ve made to the scripts that users will notice is the inline documentation: the help page that is provided by the script within the terminal, whenever you either type the name of the script without any command-line arguments, or use the -help option. Previously, this help page was provided by the argparse library, which is also responsible for parsing the arguments and options that you provide at the command-line when running these scripts. Unfortunately these help pages could be quite confusing. So as of today, the scripts now provide help pages that are very similar to those provided by the MRtrix3 binary commands; they are also piped to less by default, which allows you to navigate up and down the help page contents more easily.
Contrast this old help page for dwipreproc:
Error: the following arguments are required: pe_dir, input, output
usage: dwipreproc [-continue <TempDir> <LastFile>] [-force] [-help]
[-nocleanup] [-nthreads number] [-tempdir /path/to/tmp/]
[-quiet | -verbose] [-cuda]
(-rpe_none | -rpe_pair forward reverse | -rpe_all input_revpe)
[-grad GRAD | -fslgrad bvecs bvals]
[-export_grad_mrtrix grad | -export_grad_fsl bvecs bvals]
pe_dir input output
Perform diffusion image pre-processing using FSL's eddy tool; including inhomogeneity distortion correction using FSL's topup tool if possible
positional arguments:
pe_dir The phase encode direction; can be a signed axis
number (e.g. -0, 1, +2) or a code (e.g. AP, LR, IS)
input The input DWI series to be corrected
output The output corrected image series
optional arguments:
-cuda Use the CUDA version of eddy
-grad GRAD Provide a gradient table in MRtrix format
-fslgrad bvecs bvals Provide a gradient table in FSL bvecs/bvals format
-export_grad_mrtrix grad
Export the final gradient table in MRtrix format
-export_grad_fsl bvecs bvals
Export the final gradient table in FSL bvecs/bvals
format
standard options:
-continue <TempDir> <LastFile>
Continue the script from a previous execution; must
provide the temporary directory path, and the name of
the last successfully-generated file
-force Force overwrite of output files if pre-existing
-help Display help information for the script
-nocleanup Do not delete temporary directory at script completion
-nthreads number Use this number of threads in MRtrix multi-threaded
applications (0 disables multi-threading)
-tempdir /path/to/tmp/
Manually specify the path in which to generate the
temporary directory
-quiet Suppress all console output during script execution
-verbose Display additional information for every command
invoked
Options for passing reversed phase-encode data:
-rpe_none Specify explicitly that no reversed phase-encoding
image data is provided; eddy will perform eddy current
and motion correction only
-rpe_pair forward reverse
Provide a pair of images to use for inhomogeneity
field estimation; note that the FIRST of these two
images must have the same phase-encode direction as
the input DWIs
-rpe_all input_revpe Provide a second DWI series identical to the input
series, that has the opposite phase encoding; these
will be combined in the output image
Relevant citations for tools / algorithms used in this script:
eddy:
Andersson, J. L. & Sotiropoulos, S. N. An integrated approach to correction for off-resonance effects and subject movement in diffusion MR imaging. NeuroImage, 2015, 125, 1063-1078
FSL:
Smith, S. M.; Jenkinson, M.; Woolrich, M. W.; Beckmann, C. F.; Behrens, T. E.; Johansen-Berg, H.; Bannister, P. R.; De Luca, M.; Drobnjak, I.; Flitney, D. E.; Niazy, R. K.; Saunders, J.; Vickers, J.; Zhang, Y.; De Stefano, N.; Brady, J. M. & Matthews, P. M. Advances in functional and structural MR image analysis and implementation as FSL. NeuroImage, 2004, 23, S208-S219
Skare2010:
Skare, S. & Bammer, R. Jacobian weighting of distortion corrected EPI data. Proceedings of the International Society for Magnetic Resonance in Medicine, 2010, 5063
topup:
Andersson, J. L.; Skare, S. & Ashburner, J. How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging. NeuroImage, 2003, 20, 870-888
Author:
Robert E. Smith (robert.smith@florey.edu.au)
Copyright (C) 2008-2016 The MRtrix3 contributors. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
With the new help page (which also includes bold and underlined characters when viewed in the terminal):
dwipreproc: Script using the MRtrix3 Python libraries
SYNOPSIS
dwipreproc [ options ] pe_dir input output
pe_dir The phase encode direction; can be a signed axis number
(e.g. -0, 1, +2) or a code (e.g. AP, LR, IS)
input The input DWI series to be corrected
output The output corrected image series
DESCRIPTION
Perform diffusion image pre-processing using FSL's eddy tool; including
inhomogeneity distortion correction using FSL's topup tool if possible
Options for passing reversed phase-encode data; one of these options MUST be provided
-rpe_none
Specify explicitly that no reversed phase-encoding image data is provided;
eddy will perform eddy current and motion correction only
-rpe_pair forward reverse
Provide a pair of images to use for inhomogeneity field estimation; note
that the FIRST of these two images must have the same phase-encode
direction as the input DWIs
-rpe_all input_revpe
Provide a second DWI series identical to the input series, that has the
opposite phase encoding; these will be combined in the output image
Options for the dwipreproc script
-cuda
Use the CUDA version of eddy
-grad GRAD
Provide a gradient table in MRtrix format
-fslgrad bvecs bvals
Provide a gradient table in FSL bvecs/bvals format
-export_grad_mrtrix grad
Export the final gradient table in MRtrix format
-export_grad_fsl bvecs bvals
Export the final gradient table in FSL bvecs/bvals format
Standard options
-continue <TempDir> <LastFile>
Continue the script from a previous execution; must provide the temporary
directory path, and the name of the last successfully-generated file
-force
Force overwrite of output files if pre-existing
-help
Display help information for the script
-nocleanup
Do not delete temporary files during script, or temporary directory at
script completion
-nthreads number
Use this number of threads in MRtrix multi-threaded applications (0
disables multi-threading)
-tempdir /path/to/tmp/
Manually specify the path in which to generate the temporary directory
-quiet
Suppress all console output during script execution
-verbose
Display additional information for every command invoked
AUTHOR
Robert E. Smith (robert.smith@florey.edu.au)
COPYRIGHT
Copyright (c) 2008-2016 the MRtrix3 contributors This Source Code Form is
subject to the terms of the Mozilla Public License, v. 2.0. If a copy of
the MPL was not distributed with this file, You can obtain one at
http://mozilla.org/MPL/2.0/ MRtrix is distributed in the hope that it will
be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. For more details, see
www.mrtrix.org
REFERENCES
Andersson, J. L. & Sotiropoulos, S. N. An integrated approach to correction
for off-resonance effects and subject movement in diffusion MR imaging.
NeuroImage, 2015, 125, 1063-1078
Smith, S. M.; Jenkinson, M.; Woolrich, M. W.; Beckmann, C. F.; Behrens, T.
E.; Johansen-Berg, H.; Bannister, P. R.; De Luca, M.; Drobnjak, I.;
Flitney, D. E.; Niazy, R. K.; Saunders, J.; Vickers, J.; Zhang, Y.; De
Stefano, N.; Brady, J. M. & Matthews, P. M. Advances in functional and
structural MR image analysis and implementation as FSL. NeuroImage, 2004,
23, S208-S219
* If using -rpe_all option:
Skare, S. & Bammer, R. Jacobian weighting of distortion corrected EPI data.
Proceedings of the International Society for Magnetic Resonance in
Medicine, 2010, 5063
* If using -rpe_pair or -rpe_all options:
Andersson, J. L.; Skare, S. & Ashburner, J. How to correct susceptibility
distortions in spin-echo echo-planar images: application to diffusion
tensor imaging. NeuroImage, 2003, 20, 870-888
I think the biggest issue with the default argparse help page is that the ‘usage’ field becomes entirely congested with command-line options, and fails to adequately highlight the command-line arguments that are actually required. So hopefully these new help pages are much easier when first learning how to use a new script, or looking for available command-line options.
Some other little changes that will hopefully make script usage more seamless:
-
Less picky about the order in which you provide options (e.g. previously to use
-forceit had to appear before the algorithm name; that’s no longer the case) -
Scripts will now print the terminal output of any command that fails, and also write that output to a text file in the temporary directory.
Script algorithms
For scripts 5ttgen and dwi2response, the first compulsory command-line argument is the name of the algorithm to be used: e.g. 5ttgen fsl to use the ‘default’ approach for generating the ACT 5TT image using FSL tools BET / FAST / FIRST, and dwi2response tournier to use Donald’s response function estimation algorithm from this paper.
These two scripts are unique in that they provide solutions to a particular computational task (i.e. generating the 5TT image / estimating the response function), but there is no single unique way of achieving those tasks; therefore a range of algorithms are provided, which can be selected at the command line. Furthermore, each individual algorithm may have its own set of different command-line arguments or options associated with them.
What many may not have realised is that the algorithm choices made available by these scripts are not actually hard-coded into the main script itself. Rather, the master script (i.e. 5ttgen / dwi2response) looks for appropriate source files within the relevant scripts/src/ sub-directory, and makes those algorithms available for selection by the user at the command-line:
rob@Three MINGW64 /c/Users/rob/mrtrix3
$ ls scripts/src/dwi2response/ | grep -v "__"
fa.py
manual.py
msmt_5tt.py
tax.py
tournier.py
rob@Three MINGW64 /c/Users/rob/mrtrix3
$ dwi2response
dwi2response: Script using the MRtrix3 Python libraries
SYNOPSIS
dwi2response [ options ] algorithm ...
algorithm Select the algorithm to be used to derive the response function;
additional details and options become available once an
algorithm is nominated. Options are: fa, manual, msmt_5tt,
tax, tournier
Note how the list of algorithms made available by dwi2response matches the contents of the scripts/src/dwi2response directory. Furthermore, if a particular algorithm is selected when invoking the help page, the script will additionally present any arguments and options specific to the chosen algorithm within the generated help page.
This therefore presents an opportunity for anybody interested in, for instance, using an alternative software package for performing anatomical image segmentation and providing the 5TT image, or an alternative sequence of processing steps for generating a response function for spherical deconvolution. It is possible to provide a new source file within the relevant scripts/src/ sub-directory, and the main script will automatically detect the presence of that file, and make that algorithm available at the command-line. This doesn’t require any modification to MRtrix3 files, yet allows you to take full advantage of the command-line parsing and provided library functions. So people are free to experiment with new algorithm designs for these tasks, by using one of the existing algorithms as a starting point and modifying from there.
I hope somebody out there other than myself manages to have a bit of fun mucking about with these 
Cheers Rob