https://chalearnlap.cvc.uab.cat/dataset/24/description/
- train-1~6.zip
- val-1~2.zip
- test-1~2.zip
- gt : 'extraversion', 'neuroticism', 'agreeableness', 'conscientiousness', 'openness'
each zip folder contains about 960 clips, clips length = 15s
- Preprocessed data
- data_v1: 6 random frame per video, frame size 128
- trian_set.dat (train-1.zip)(734)
- valid_set.dat (val-1.zip)(764)
- test_set.dat (test-1~2.zip)(1845)
- data_v2: 6 random frame per video, frame size 128
- train_set.dat(train-2~6.zip)(4300)
- data_v3: 6 random frame per video, frame size 128, try not drop any videos
- train_set.dat(train-2~6.zip)(5034)
- valid_set.dat (val-1.zip)(960)
- valid_set_2.dat (val-2.zip)(960)
- test_set.dat (test-1~2.zip)(1998)
- latest_data
- 30_256px: train, valid data are saved individually using video name. 30 frames (等距離的切), frame size 256.
- data_v1: 6 random frame per video, frame size 128
- Train
- train: all videos in train-1.zip (960)
- train13: all videos in train-2~6.zip (5040)
- train_6000: all videos in trainset (6000)
- Valid
- valid: all videos in val-1.zip (960)
- valid_2: all videos in val-2.zip (960)
- valid_small: few vidoes from val-1.zip (960)
- Test
- test: all videos in testset (2000)
- big5.ipynb
- regression
- criterion: accuracy=1-MAE
- big5_classification.ipynb
- classification
- criterion: accuracy, precision, recall
- extract folder
- only for extracting folders in Drive
- Packages*
- remember to rerun if colab shut down
- Configuration*
- settings for model, training, preprocess data...
- define root directory
- define checkpoint & logs name: date, note
- Helper
- functions to detect face, extract audio and frames
- import dlib
- Build New Dataset
- use functions in helper to build traian, valid, test dataset
- list of tuples save as .dat
- never run it if you are using .dat
- Dataloader*
- pytorch Dataset for latter use
- dim: input channel, default = 1
- return
sample = {'images': images, # size: N frames, H, W 'label': float(class_label), # 0. or 1. 'audio': audio, # None now 'uid': uid, # video name 'value': org_value} # origin annotation value # in big5.ipynb, 'label' is original 5 value
- augmentation (horizon flip)
- pytorch Dataset for latter use
- Load data
- read .dat
- data structure in .dat
# update 11/1 # new structure: only record one preprocessed data sample1 tuple(array[(30*256*256)] ########## # old structure: each video and true labels in a list list[ # np arrays: preprocessed images, ground truth big 5 sample1 tuple(array[(6*128*128)], array[(5)]), sample2 tuple(array[(6*128*128)], array[(5)]), sample3 ... ... ]
- in classification.ipynb, find Q40 & Q60 to filter 20% data in the center
- no need to filter them in regression
- Model*
- model classes and helper function
- CNN block
- resnet
- it is OK to run all of them
- model input: Batch size, Channel(1 for gray scale), Depth(N frame), H, W
- Build model
- only choose one of the model to build
- CNN Block
- Resnet
- Pretrain
- Then run Optimizer & Checkpoint
- only choose one of the model to build
- Train Helper*
- loss function
- train & valid module: input batches into model
- Train
- Tensorboard for monitoring the training process, may need to wait a bit for it to show up
- main: do serveral time of training & validing (epoch)
- Test
- Run cells with * before testing: Packages, Config, Dataloader, Model, Train Helper
- Then build test dataloader and model to inference
Code Source: https://github.com/grimmdaniel/personality-trait-prediction
Competition: Chalearn 2017 Looking at People CVPR/IJCNN Competition
Leaderboard(Results), Evaluation Criterion: Platform