Lucas-Kanade-tracking-and-Correlation-Filters

This repository contains implementation of Lucas-Kanade algorithm proposed by Lucas and Kanade. Lucas-Kanade algorithm can be used for sparse optical flow (associate feature points across frames) and tracking (associate image patch cross frames). This repo implements the algorithm for tracking a single template across 400 frames video.
Please unzip data.zip and then follow the instructions below.

Lucas Kanade Tracking with one single template

The "vanilla" algorithm for tracking. Detailed derivation can be referred to Lucas-Kanade 20 Years On: A Unifying Framework. This tracker runs around 36 Hz on my local machine.
Files included:
/data/carseq.npy
/src/LucasKanade.py
/src/testCarSequence.py
Run

python testCarSequence.py

Sample results

/src/carseqrects.npy stores the vertices coordinates of bounding box in each frame.

Lucas Kanade Tracking with template correction

The upgraded version of the first algorithm, which mitigates template drifting problem. The template can be updated every frame, but it must be re-aligned to the original template to remove drift. Detailed derivation can be referred to The Template Update Problem. This tracker runs around 18 Hz on my local machine.
FIles included:
/data/carseq.npy
/src/LucasKanade.py
/src/TemplateCorrection.py
/src/testCarSequenceWithTemplateCorrection.py
Run

python testCarSequenceWithTemplateCorrection.py

Sample results
Blue bbox: with template correction | Red bbox: without template correction

Lucas Kanade Tracking with appearance basis

The former 2 algorithms may not suffice for real life challenges with drastic appearance variance. Through analyzing historical data collected, we can use an eigen-space approach to produce a principal template at each frame. This tracker runs around 39 Hz on my local machine.
Files included:
/data/sylvseq.npy
/data/sylvbbases.npy
/src/LucasKanade.py
/src/TemplateCorrection.py
/src/LucasKanadeBasis.py
/src/testSylvSequence.py
Run

python testSylvSequence.py

Sample results
Blue bbox: with appearance bases | Red bbox: without appearance bases

Lucas Kanade Tracking with dominant affine motion

This algorithm works on non-stationary camera video. This tracker runs around 23 Hz on my local machine.
Files included
/data/aerialseq.npy
/src/LucasKanadeAffine.py
/src/SubtractDominantMotion.py
/src/InverseCompositionAffine.py
/src/testAerialSequence.py
Run

python testAerialSequence.py

Sample results

Correlation Filters

The paper of correlation filters can be found hereVisual Object Tracking using Adaptive Correlation Filters
Files included
/src/Corr-Filters/lena.npy
/src/Corr-FIlters/example.py
Run

python example.py