/Computational-CryoEM

A curated list of awesome computational cryo-EM methods.

Primary LanguageJupyter NotebookCreative Commons Zero v1.0 UniversalCC0-1.0

Computational CryoEM Methods

Awesome

This repository is a curated list for computational cryo-EM methods mainly target for single particle analysis!

You will find the paper and associated software for the popular algorithms used in the cryo-EM field.

The format for the item is [software link] - [Paper link] (Available in which package) and the headings may contain a ink to a review paper.

Resources

Introduction

Courses

Resources

  • 3DEM Methods - A great wiki that collects papers or books for computational methods.
  • A collective resource - A great repository that covers single particle analysis, model building and tomography.
  • Math behind CryoEM - A great repository that collects the materials which elaborate the math behind single-particle analysis.
  • Chimera Tutorial

Events

Software

  • Scipion - An integrated platform that allows users to use a variety of methods in the same framework.
  • Relion - A comprehensive package that utilizes Bayesian approach for 2D classification and 3D refinement.
  • CryoSparc 2 - A general package that employs stochastic gradient descent, branch and bound as well as GPU acceleration for rapid reconstructions.
  • Sphire - A general package that contains neural network methods for particle picking, denoising and classification.
  • CisTEM - An easy-to-use framework that implements a complete pipeline for single-particle analysis.
  • EMAN2 - A comprehensive package that contains a python interface and handy scripts for common tasks.
  • SPIDER - A well-known package that implements image processing methods for electron microscopy.

Workflow

Import Data

  1. From EMPIAR or EMDB.
  2. From tutorial data set.
DataSet (Molecule) File Size Movie (#frame) Micrograph Size (Pixel Size) Picked Particles (Size) Final Resolution(A)(Symmetry)
Scipion
(Beta-gal)		 4.0GB 15(16) 1950x1950 (3.54) (100x100) 7.3 (D2)
Relion 3
(Beta-gal)		 3.1GB 24(24) 3710x3838 (0.885) (256x256) 2.9 (D2)
CryoSparc 2 (T20S) 8.1GB 20(38) 7420x7676 (0.6575) (440x440) 2.93 (D7)
CisTEM (ApoFerritin) 5.6GB 20(50) 1240x1200 (1.5) 3.0 (O)
  1. Generate simulation data

Motion correction

  • Whole frame alignment

  1. Unblur - Measuring the optimal exposure for single particle cryo-EM using a 2.6 Å reconstruction of rotavirus VP6 ( Use in CisTEM)
  2. Full-frame motion correction - (Use in CryoSparc)
  3. Optical Flow - Alignment of direct detection device micrographs using a robust Optical Flow approach (Use in Xmipp)

  • Patch-based/Per-particle based alignment

  1. MotionCorr2 - MotionCor2: anisotropic correction of beam-induced motion for improved cryo-electron microscopy (Use in Relion and CryoSparc)
  2. Alignparts - Alignment of cryo-EM Movies of Individual Particles by Optimization of Image Translations (Use in CryoSparc(local, patch))
  3. Warp - Contains Patch-based motion correction.
  4. FlexAlign - FlexAlign: An Accurate and Fast Algorithm for Movie Alignment in Cryo-Electron Microscopy - (Use in Scipion)

CTF estimation

  • Whole frame

  1. CTFFIND4 - CTFFIND4: Fast and accurate defocus estimation from electron micrographs ( Use in CisTEM)
  2. gCTF - Gctf: Real-time CTF determination and correction

  • Patch-based/tilt data

  1. Patch-Based CTF Estimation - Real-time cryo-electron microscopy data preprocessing with Warp ( Use in CryoSparc, Warp)
  2. goCTF - goCTF: Geometrically optimized CTF determination for single-particle cryo-EM
  3. Warp - Contains Patch-based CTF estimation.

  • Denoising micrograph

  1. Restore - Enhancing SNR and generating contrast for cryo-EM images with convolutional neural networks
  2. Topaz - Topaz-Denoise: general deep denoising models for cryoEM
  3. JANNI - Just Another Noise 2 Noise Implementation
  4. Warp - Contains methods base on noise2noise and deconvolution filter

Particle picking

  • Semi-supervised picking

  1. Topaz - Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs. [Video]
  2. Cryolo - SPHIRE-crYOLO is a fast and accurate fully automated particle picker for cryo-EM. [Video]
  3. Xmipp - A pattern matching approach to the automatic selection of particles from low-contrast electron micrographs

  • Template-based picking

  1. Relion, CryoSparc - Use 2D class averages or 3D projection for more accurate particle picking

  • Automatic picking

  1. DoG - DoG Picker and TiltPicker: software tools to facilitate particle selection in single particle electron microscopy
  2. LoG - The Laplacian of Gaussian and Arbitrary Z-Crossings Approach Applied to Automated Single Particle Reconstruction (Use in Relion auto-picking)
  3. APPLE - APPLE picker: Automatic particle picking, a low-effort cryo-EM framework

  • Denoising particle and dimenison reduction

  1. 2SDR - Two-stage dimension reduction for noisy high-dimensional images and application to Cryogenic Electron Microscopy
  2. CWF - Denoising and covariance estimation of single particle cryo-EM images
  3. GAN - Generative adversarial networks as a tool to recover structural information from cryo-electron microscopy data

2D classification

  • Multirefence alignment based classification

  1. ISAC - Iterative Stable Alignment and Clustering of 2D Transmission Electron Microscope Images. [GPU version]
  2. CL2D - A clustering approach to multireference alignment of single-particle projections in electron microscopy
  1. Relion - Bayesian (Empirical Bayes) approach
  2. CryoSparc - Bayesian with Branch and bound method
  3. CisTEM - Maximum likelihood method

  • Mixed approach classification

  1. ROME - Massively parallel unsupervised single-particle cryo-EM data clustering via statistical manifold learning

  • Automatic selection of 2D classes

  1. Cryoassess -High-Throughput Cryo-EM Enabled by User-Free Preprocessing Routines
  2. Cinderella - Cinderella: Deep learning based binary classification tool

  • New clustering methods

  1. γ-SUP - γ-SUP: A clustering algorithm for cryo-electron microscopy images of asymmetric particles

Ab-initial model

  • Class-averages based method

  1. Simple-Single-particle cryo-EM-Improved Ab Initio 3D Reconstruction With SIMPLE/PRIME

  • Particles based method

  1. CryoSparc - cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination. [Slides]

3D refinement

  • 3D Homogeneous Refinemnet

  1. Relion - RELION: Implementation of a Bayesian approach to cryo-EM structure determination. [Video]
  2. CryoSparc - Use Expectation-Maximization with branch and bound method for higher resolution
  3. OPUS-SSRI - Sparseness and Smoothness Regularized Imaging for improving the resolution of Cryo-EM single-particle reconstruction

  • 3D classification

  1. Relion, CryoSparc - Perturb the initial model and use projection matching with weighted assignment

  • 3D non-uniform Refinemnet

  1. CryoSparc - Non-uniform refinement: Adaptive regularization improves single particle cryo-EM reconstruction
  2. SideSplitter - Mitigating Local Over-fitting During Single Particle Reconstruction with SIDESPLITTER. [Video]
  1. 3DVA - 3D Variability Analysis: Directly resolving continuous flexibility and discrete heterogeneity from single particle cryo-EM images ( Use in CryoSparc)
  2. CryoDRGN - CryoDRGN: Reconstruction of heterogeneous structures from cryo-electron micrographs using neural networks
  3. ManifoldEM - Retrieving functional pathways of biomolecules from single-particle snapshots
  4. AlphaCryo4D - Deep manifold learning reveals hidden dynamics of proteasome autoregulation
  1. localrec - Localized reconstruction of subunits from electron cryomicroscopy images of macromolecular complexes (Use in Scipion)
  2. Multi-body refinement - Characterisation of molecular motions in cryo-EM single-particle data by multi-body refinement in RELION (Use in Relion)

Postprocessing

  • Per-particle based motion and ctf refinement

  1. CTF refinement - Relion3/CisTEM/CryoSparc, 3D Reference required
  2. Ewald sphere correction - Relion3 (Use relion_reconstruct --reverse_curvature)
  3. High-order aberrations - Estimation of high-order aberrations and anisotropic magnification from cryo-EM data sets in RELION-3.1
  4. Bayesian Polishing - A Bayesian approach to beam-induced motion correction in cryo-EM single-particle analysis (Use in Relion, 3D Reference required)
  5. M - Multi-particle cryo-EM refinement with M visualizes ribosome-antibiotic complex at 3.5 Å inside cells. [Video]
  1. blocres - One number does not fit all: Mapping local variations in resolution in cryo-EM reconstructions
  2. ResMap - Quantifying the Local Resolution of Cryo-EM Density Maps
  3. MonoRes - MonoRes: Automatic and Accurate Estimation of Local Resolution for Electron Microscopy Maps (Use in Scipion)
  4. 3DFSC - Addressing preferred specimen orientation in single-particle cryo-EM through tilting
  5. MonoDir - Measuring local-directional resolution and local anisotropy in cryo-EM maps (Use in Scipion)
  1. Confidence Maps - Thresholding of cryo-EM density maps by false discovery rate control (Use in ccpem)
  2. DeepEnhancer-DeepEMhancer: a deep learning solution for cryo-EM volume post-processing
  3. LocScaleModel-based local density sharpening of cryo-EM maps (Use in ccpem)
  4. LocalDeblur - Automatic local resolution-based sharpening of cryo-EM maps

  • Denoising 3D volume

  1. Warp - Based on Noise2Noise
  2. Topaz - Contains 3D denoise functionality
  3. Relion - Exploiting prior knowledge about biological macromolecules in cryo-EM structure determination

  • Ewald Sphere Correction

  1. Relion - New tools for automated high-resolution cryo-EM structure determination in RELION-3

Visualization

  1. Chimera - UCSF Chimera--a Visualization System for Exploratory Research and Analysis
  2. ChimeraX - UCSF ChimeraX: Meeting modern challenges in visualization and analysis

Conventions

3DEM Convention

Image contrast

White on Black - Relion, Xmipp, EMAN2, CryoSparc (Cryo-EM data is typically recorded as Black on White but will invert during processing)

Black on White - Frealign (CisTEM)

Pixel size

For the format defined in RELION, the actual pixel size is calculated as rlnDetectorPixelSize * 10000 / rlnMagnification

FSC calculation

Using e2proc3d

Mask generation in CryoSparc

No Mask:

This is the raw FSC calculated between two independent half-maps reconstructed from the data. There is no masking applied, so both the structure and solvent are included in this FSC.

Spherical:

This is the FSC calculated after applying a soft spherical mask to both half maps. The outer radius of the soft sphere is equal to half the volume box-size (i.e. the sphere extends to the faces of the box in all directions). The inner radius is 85 percent of the outer radius. Between inner and outer radii, a soft cosine edge transitions from a mask value of one to a value of zero.

Loose:

This is the FSC calculated after applying a soft solvent mask to both half maps. The loose mask is calculated as follows. First, the density map is thresholded at 50% of the maximum density value. The resulting volume is dilated to create a soft mask. Voxels in the mask that are within 25 angstroms of the thresholded region receive a mask value of 1.0. Voxels between 25 and 40 angstroms fall off with a soft cosine edge, and voxels outside 40 angstroms receive a value of 0.0.

Tight:

This is the same as the loose mask, except the dilation distances are 6 angstroms for the value 1.0 distance and 12 angstroms for the value 0.0 distance.

Corrected:

This is the FSC curve calculated using the tight mask with correction by noise substitution [1]. The two half maps have their phases randomized beyond a certain resolution, then the tight mask is applied to both, and an FSC is calculated. This FSC is used along with the original FSC before phase randomization to compute the corrected FSC as in [1]. This accounts for correlation effects induced by masking. The resolution at which phase randomization begins is the resolution at which the no-mask FSC drops below the FSC = 0.143 criteria.

Chen, S. et al. High-resolution noise substitution to measure overfitting and validate resolution in 3D structure determination by single particle electron cryomicroscopy. Ultramicroscopy 135, 24–35 (2013).

Sharpen and filtering in CryoSparc

The map_filtered output in non-uniform refinement is generated as follows, after refinement has converged:

  1. both raw, unfiltered halfmaps are averaged together
  2. the raw map is filtered using the Gold-Standard FSC curve
  3. the filtered map is sharpened using the Guinier-plot estimated b-factor
  4. the sharpened map is outputted as _map_sharp_local.mrc (confusing filename… sorry)
  5. the sharpened map is locally filtered using a local resolution estimate computed from the half-maps and the locally filtered map is outputted as _map_filtered.mrc

Tips

Format Conversion

  1. Using PyRelion Operate Star files

  2. Using PyEM Convert metadata

  3. Using EMAN2 Convert binary data

  4. Using SPHIRE Convert binary data

Parse Star file

  1. Using StarTool
  2. Using Custom function like here

Parse CryoSparc file

  1. Using PyEM
  2. Using Custom function like here. See full tutorial here

Performing Focus Classification

Using Chimera

CisTEM

A focus mask is defined as a sphere specified by radius and x,y,z coordinates of the sphere center, all given in Å. In order to find x,y,z, in CisTEM

  1. Open the 3D map in Chimera. The map should be assigned model #0.
  2. Display the command line by selecting Tools > General Controls > Command Line.
  3. Execute vop threshold #0 maximum -1000 setMaximum 1.0 (this assumes that the minimum voxel value in the 3D map is larger than 1000). This will create a new model #1.
  4. Execute shape sphere radius 30 color red mesh true on the Command Line. This will generate a sphere with a 30 Å radius, numbered model #2 (change the radius as needed).
  5. Open Tools > General Controls > Model Panel and deactivate models #0 and #1. Then move the sphere (model #2) into model #1 until it is fully contained in that volume.
  6. Execute mask #1 #2 on the Command Line. This will generate a new model #3 of a solid sphere with a 30 Å radius.
  7. Close models #1 and #2 in Model Panel and make sure model #0 is visible but deactive.
  8. Select model #3 (the solid sphere) in Volume Viewer and change its color to red (or something easily distiguishable from the original 3D map) and make it transparent.
  9. Move model #3 to the desired location of the focus mask in the 3D map.
  10. Execute vop resample #3 onGrid #0 on the Command Line. This will generate a copy of the solid sphere as a new model #1, now resampled in the same coordinate system as the original 3D map.
  11. Execute measure center #1 on the Command Line. This will display the x,y,z coordinates of the mask in pixel coordinates below the command line. These coordinates have to be converted to Å by multiplying them with the pixel size of the 3D map before they can be used in cisTEM's Manual Refine panel.

Local classification in CryoSparc

Display images

Micrographs

In Matlab use grayImage = uint8(255 * mat2gray(originalImage)); imshow(grayImage);

Chimera

Dispaly local resolution

Flip handedness of model and map

Slice view

volume #0 planes z,220 step 1 level -1 style surface

Contributing

Your contributions are highly appreciated! Please take a look at the contribution guidelines first.