We provide Python tools for seamless integration of MEG data from the Human Connectome Project into the Python ecosystem. In only a few lines of code, complex data retrieval requests can be readily executed on the resources from this neuroimaging reference dataset. Raw HCP data are translated into actionable MNE objects that we know and love. MNE-HCP abstracts away difficulties due to diverging coordinate systems, distributed information, and file format conventions. Providing a simple and consistent access to HCP MEG data will facilitate emergence of standardized data analysis practices. By building on the MNE software package, MNE-HCP naturally supplements a fast growing stack of Python data science toolkits.
Allow us to give you a flavor by a few example queries of MEG HCP data from subject 105923:
# Get all entries from the MEG data header
info = hcp.read_info('105923', 'task_motor')
# Get continuous MEG time series
raw = hcp.read_raw('105923', 'task_motor')
# Get segmented MEG time series
epochs = hcp.read_epochs('105923', 'task_motor')
# Get all MEG time series averaged across events
list_of_evoked = hcp.read_evokeds('105923', 'task_motor')
# Get details on contamination and noise sources
annotations_dict = hcp.read_annot('105923', 'task_motor')
# Get precomputed independent components that compose the signal time series
ica_mat = hcp.read_ica('105923', 'task_motor')This code is under active research-driven development. The API is still changing, but is getting closer to a stable release.
Note
For now please consider the following caveats:
- We only intend to support a subset of the files shipped with HCP.
- Specifically, for now it is not planned to support io and processing for any outputs of the HCP source space pipelines.
- This library breaks with some of MNE conventions in order to make the HCP outputs compatible with MNE.
We recommend the Anaconda Python distribution, which comes with the necessary dependencies. Alternatively, to install mne-hcp, you first need to install its dependencies:
$ pip install numpy matplotlib scipy scikit-learn mne joblib pandas
Then clone the repository:
$ git clone http://github.com/mne-tools/mne-hcp
and finally run setup.py to install the package:
$ cd mne-hcp/ $ python setup.py install
If you do not have admin privileges on the computer, use the --user flag
with setup.py.
Alternatively, for a devoloper install based on symbolic links (which simplifies keeping up with code changes), do:
$ cd mne-hcp/ $ python setup.py develop
To check if everything worked fine, you can do:
$ python -c 'import hcp'
and it should not give any error messages.
The following main and additional dependencies are required to use MNE-HCP:
- MNE-Python master branch
- the MNE-Python dependencies, specifically
- scipy
- numpy
- matplotlib
- scikit-learn (optional)
The following data layout is expected: a folder that contains the HCP data as they are unpacked by a zip, subject wise. When data were downloaded via the Aspera connect client, the following command should produce the expected layout:
$ for fname in $(ls *zip); do
$ echo unpacking $fname;
$ unzip -o $fname; rm $fname;
$ doneWhen files are downloaded using the Amazon webservice tools, e.g. s3rcmd, all should be fine.
The code is organized by different modules. The io module includes readers for sensor space data at different processing stages and annotations for bad data.
MNE-HCP uses custom names for values that are more MNE-pythonic, the following table gives an overview:
| name | readers | HCP jargon |
| 'rest' | raw, epochs, info, annotations, ica | 'Restin' |
| 'task_working_memory' | raw, epochs, info, annotations, ica | 'Wrkmem' |
| 'task_story_math' | raw, epochs, info, annotations, ica | 'StoryM' |
| 'task_motor' | raw, epochs, info, annotations, ica | 'Motor' |
| 'noise_subject' | raw, info | 'Pnoise' |
| 'noise_empty_room' | raw, info | 'Rnoise' |
MNE HCP comes with convenience functions such as hcp.make_mne_anatomy. This one will create an MNE friendly anatomy directories and extracts the head model and coregistration MEG to MRI coregistration. (Yes, it maps to MRI, not to the helmet -- a peculiarity of the HCP data.) It can be used as follows:
>>> import os.path as op
>>> import hcp
>>> storage_dir = op.expanduser('~/data/MNE-HCP')
>>> hcp.make_mne_anatomy(
>>> '100307', subjects_dir=storage_dir + '/subjects',
>>> hcp_path=storage_dir + '/HCP',
>>> recordings_path=storage_dir + '/hcp-meg')
reading extended structural processing ...
reading RAS freesurfer transform
Combining RAS transform and coregistration
extracting head model
coregistring head model to MNE-HCP coordinates
extracting coregistrationMNE-HCP supports a low level file mapping that allows for quick compilations of sets of files for a given subejct and data context. This is done in :func:`hcp.io.file_mapping.get_file_paths`, think of it as a file name synthesizer that takes certain data description parameters as inputs and lists all corresponding files.
Example usage:
>>> import hcp
>>> files = hcp.file_mapping.et_file_paths(
>>> subject='123455', data_type='task_motor', output='raw',
>>> hcp_path='/media/storage/HCP')
['/media/storage/HCP/123455/unprocessed/MEG/10-Motor/4D/c,rfDC',
'/media/storage/HCP/123455/unprocessed/MEG/10-Motor/4D/config']Why we are not globbing files? Because the HCP-MEG data are fixed, all file patterns are known and access via Amazon web services easier if the files to be accessed are known in advance.
The HCP for MEG provides coregistration information for native BTI/4D setting. MNE-Python expects coordinates in meters and the Neuromag right anterior superior (RAS) coordinates. However, essential information is missing to compute all transforms needed to easily perform the conversions.
For now, the way things work, all processing is performed in native BTI/4D coordinates with the device-to-head transform skipped (set to identity matrix), such that the coregistration directly maps from the native 4D sensors, represented in head coordinates, to the freesurfer space. This has a few minor consequences that may be confusing to MNE-Python users.
1. In the reader code you will see many flags set to `convert=False`, etc.
This is not a bug.
2. All channel names and positions are native. Topographic plotting might not work as as expected. First of all, the layout file is not recognized. Second, the coordinates are not regonized as native ones, eventually rotating and distorting the graphical display. To fix this, either a proper layout can be computed with :func:`hcp.viz.make_hcp_bti_layout`. Or the conversion to MNE can also be performed using :func:`hcp.preprocessing.map_ch_coords_to_mne`. But note that source localization will be wrong when computed on data in Neuromag coordinates. As things are, coordinates have to be kept in the native space to be aligned with the HCP outputs.
A couple of steps are necessary to reproduce the original sensor space outputs.
Reference channels should be regressed out. Checkout :func:`hcp.preprocessing.apply_ref_correction`.
The trial info structure gives the correct latencies of the events The latencies in the trigger channel are shifted by around 18 ms. For now we'd recommend using the events from the function :func:`hcp.read_trial_info`.
The default filters in MNE and FieldTrip are different. FieldTrip uses a 4th order butterworth filter. In MNE you might need to adjust the *_trans_bandwidth parameter to avoid numerical errors. In the HCP outputs, evoked responses were filtered between 0.5 and 30Hz prior to baseline correction.
Annotations need to be loaded and registered. The HCP consortium ships annotations of bad segments and bad channels. These have to be read and used. Check out hcp.read_annot and add bad channel names to raw.info['bads'] and create and set an mne.Annotations object as attribute to raw, see below.
annots = hcp.read_annot(subject, data_type, hcp_path=hcp_path, run_index=run_index) bad_segments = annots['segments']['all'] / raw.info['sfreq'] raw.annotations = mne.Annotations( bad_segments[:, 0], (bad_segments[:, 1] - bad_segments[:, 0]), description='bad')
ICA components related to eye blinks and heart beats need to be removed from the data. Checkout the ICA slot in the output of hcp.read_annot to get the HCP ICA components.
NNE-HCP includes convenience functions that help setting up directory and file layouts expected by MNE-Python.
:func:`hcp.make_mne_anatomy` will produce an MNE and Freesurfer compatible directory layout and will create the following outputs by default, mostly using sympbolic links:
$subjects_dir/$subject/bem/inner_skull.surf
$subjects_dir/$subject/label/*
$subjects_dir/$subject/mri/*
$subjects_dir/$subject/surf/*
$recordings_path/$subject/$subject-head_mri-trans.fifThese can then be set as $SUBJECTS_DIR and as MEG directory, consistent with MNE examples. Here, inner_skull.surf and $subject-head_mri-trans.fif are written by the function such that they can be used by MNE. The latter is the coregistration matrix.
MNE-HCP corrects on reading the indices it finds for data segments, events, or components. The indices it reads from the files will already be mapped to Python conventions by subtracting 1.
- currently @dengemann is pushing frequently to master, if you plan to contribute, open issues and pull requests, or contact @dengemann directly. Discussions are welcomed.
This project is supported by the Amazon Webservices Research grant issued to Denis A. Engemann and by the ERC starting grant ERC StG 263584 issued to Virginie van Wassenhove.
I acknowledge support by Alex Gramfort, Mainak Jas, Jona Sassenhagen, Giorgos Michalareas, Eric Larson, Danilo Bzdok, and Jan-Mathijs Schoffelen for discussions, inputs and help with finding the best way to map HCP data to the MNE world.