This repo contains the code for the ACCV paper on Compressed Vision. The paper describes how we can first compress videos to a smaller representation and then train a neural network directly on this compressed representation for various downstream tasks. We can also apply augmentations directly in this compressed space, thereby replicating the full standard video pipeline. By operating directly in the compressed representation, we can save both memory and speed.
In particular, we provide pre-trained models that can be used to compress and decompress your own data. We provide both models that can be used to perform augmentation or just compression.
First, install dependencies following these instructions:
- Create a virtual env:
python3 -m venv ~/.venv/compressed_vision(we used python==3.7.14) - Switch to the virtual env:
source ~/.venv/compressed_vision/bin/activate - Follow instructions for installing JAX on your platform: https://github.com/google/jax#installation if you want to run on device
- Install other dependencies:
pip install -r requirements.txt
After installing dependencies, you can open the notebooks in the colabs directory
using Jupyter or Colab.
Our colabs assume that you are running from the
compressed_vision directory if running locally.
A simple CoLAB showing how to load our models at
different compression levels can be found in
./colabs/demo.ipynb. It also includes an example
showing how augmentation is applied to a compressed
video.
Another simple CoLAB in ./colabs/demo_data.ipynb
shows how to load compressed versions
and visualises the PCA of these representations as videos.
@inproceedings{wiles22,
title={Compressed Vision for Efficient Video Understanding},
authors={Wiles, Olivia and Carreira, Joao and Barr, Iain and Zisserman, Andrew and Malinowski, Mateusz},
conference={ACCV},
year=2022
}
Copyright 2022 DeepMind Technologies Limited
All software is licensed under the Apache License, Version 2.0 (Apache 2.0); you may not use this file except in compliance with the Apache 2.0 license. You may obtain a copy of the Apache 2.0 license at: https://www.apache.org/licenses/LICENSE-2.0
All other materials are licensed under the Creative Commons Attribution 4.0 International License (CC-BY). You may obtain a copy of the CC-BY license at: https://creativecommons.org/licenses/by/4.0/legalcode
Unless required by applicable law or agreed to in writing, all software and materials distributed here under the Apache 2.0 or CC-BY licenses are distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the licenses for the specific language governing permissions and limitations under those licenses.
This is not an official Google product.