Motion Capture and Two-photon dataset

A tethered Drosophila melanogaster behaving freely while neural and behavior activity is recorded using multi-view infrared cameras and two-photon microscopy imaging. Please check the following publications for more information.

Introduction

Download the files using the following link. We give a dataset of 8 of the animals.
The file format will look like this:

+-- 201008_G23xU1_Fly1_001
|   +-- 2p_dff.mm
|   +-- 2p_dff_resized.mm
|   +-- behData_images_camera_1.mp4
|   +-- sync_indices.pkl
|   +-- pose_result.pkl
|   +-- pose_result_inverse_kinematics.pkl
|   +-- rest.npy
+-- 201008_G23xU1_Fly1_002
+-- 201008_G23xU1_Fly1_003
+-- 201008_G23xU1_Fly1_004
+-- 201008_G23xU1_Fly1_005 
+-- 201008_G23xU1_Fly1_006
...
+-- Readme.md

2p_dff files includes microscopy two-photon imaging.
BehData camera videos include raw behavioral videos.
pose_result files include 2D and 3D pose.

How to Use the Dataset

Install the dependencies using

conda env create -f environment.yml
conda activate mc2p

You can directly read raw files and visualize:

import functional as NF
dff = NF.read_dff('/data/MM/data/201014_G23xU1_Fly1_005')
dff = NF.normalize_video(dff)
plt.imshow(dff[0], cmap='jet')

or to load the 2D pose and visualize them as a time sequence, use

import functional as NF
import augmentation as NA

pr = NF.read_pose_result_mm('/data/MM/data/201014_G23xU1_Fly1_005')
pr = pr + 0 # make it writable
pr = NA.PRunNorm()(pr.reshape(-1, 38, 2)) # preprocess
pr = pr.cpu().data.numpy().reshape(-1, 38, 2) # convert to numpy to visualize
media.show_video(NP.plot_pts2d_video(pr[:100]), height=200)

You can use pytorch dataloaders to train a model. To get a single modality:

import dataset as ND
dat = ND.DatasetUM(
    path_list=['./201008_G23xU1_Fly1_001'],
    n_frames=32,
    modal='dff',
    stride=1
)

dff, meta = dat[0]

You can choose from modalities, dff, resized_dff, pr and angles. To get a multiple synchronized modalities:

import dataset as ND
dat = ND.DatasetMM(
    path_list=['./201008_G23xU1_Fly1_001'],
    modal1='dff',
    modal2='pr',
    n_frames1=32,
    n_frames2=8,
    aug1=nn.Identity(),
    aug2=nn.Identity(),
    stride=1,
)
((dff, _),  (pr, _)) = dat[0]

semihgunel/MC2P

Motion Capture and Two-photon dataset

Introduction

How to Use the Dataset