/chainer-gqn

Neural scene representation and rendering (GQN)

Primary LanguagePythonMIT LicenseMIT

🚧 Work in Progress 🚧

Neural scene representation and rendering

https://deepmind.com/blog/neural-scene-representation-and-rendering/

Requirements

  • Python 3
  • h5py
  • Chainer
    • pip3 install chainer
  • CuPy
    • pip3 install cupy-cuda100 for CUDA 10.0
    • pip3 install cupy-cuda91 for CUDA 9.1

Also you need the followings for visualization:

  • ffmpeg
    • sudo apt install ffmpeg
  • imagemagick
    • sudo apt install imagemagick

Current training progress

figure

figure

Network Architecture

gqn_conv_draw

gqn_representation

Dataset

deepmind/gqn-datasets

Datasets used to train GQN in the paper are available to download.

https://github.com/deepmind/gqn-datasets

You need to convert .tfrecord files to HDF5 .h5 format before starting training.

https://github.com/musyoku/gqn-datasets-translator

gqn-dataset-renderer

I am working on a OpenGL/CUDA renderer for rendering GQN dataset.

https://github.com/musyoku/gqn-dataset-renderer

  • Shepard-Metzler

shepard_matzler

  • Rooms

rooms_rotate_object

  • MNIST Dice

mnist_dice