The main goal of this repo is to learn the Optical Flow task. It includes:
- task overview
- research of existing approaches
- results of reprodusing
- practice in Attacking Optical Flow
Optical Flow Estimation is the problem of finding pixel-wise motions between consecutive images.
Sparse optical flow of traffic | Optical flow problem |
---|---|
[ images source, paperswithcode page ]
- FlowNet2: paper, source code
- RAFT: paper, source code
- GMA: paper, source code
Implementation of a patch with noise or other content into frames can break the process of optical flow estimation.
This approach includes patches with a default value or the content of the patch is determined by the optimization problem. This customizes the patch and improves the results.
Example of attacks from Attacking Optical Flow paper:
Unattacked Frames | Attacked Frames | Unattacked Flow | Attacked Flow |
---|---|---|---|
Model | MSE | Noise Type |
---|---|---|
FlowNet2 | 56.9827 | Patch |
GMA | 2.5502 | Patch |
For more effective patch attacks, more complex approaches are needed. Gaussian noise and even patches optimized for testing networks will not add a significant amount of noise.