Weighted-Voxel

This repository contains the source codes for the paper Weighted Voxel: a novel voxel representation for 3D reconstruction (Xie et al. 2018).

Abstract

3D reconstruction has been attracting increasing attention in the past few years. With the surge of deep neural networks, the performance of 3D reconstruction has been improved significantly. However, the voxel reconstructed by extant approaches usually contains lots of noise and leads to heavy computation. In this paper, we define a new voxel representation, named Weighted Voxel. It provides more abundant information, facilitating the subsequent learning and generalization steps. Unlike regular voxel which consists of zero-one, the proposed Weighted Voxel makes full use of the structure information of voxels. Experimental results demonstrate that Weighted Voxel not only performs better in reconstruction but also takes less time in training.

Cite this work

If you find this work useful in your research, please consider citing:

@inproceedings{xie2018weighted,
    title={Weighted Voxel: a novel voxel representation for 3D reconstruction},
    author={Xie, Haozhe and Yao, Hongxun and Sun, Xiaoshuai and Zhou, Shangchen and Tong, Xiaojun},
    booktitle={International Conference on Internet Multimedia Computing and Service {ICIMCS} 2018},
    year={2018},
    organization={ACM}
}

Project Page

The project page is available at https://haozhexie.com/project/weighted-voxel.

Overview

Overview of Weighted Voxels

The generation of Weighted Voxels can be regarded as applying a convolutional kernel on regular voxels. The value of each voxel in the Weighted Voxel is weighted summed over voxel values of its immediate neighbors. More formally, the value in Weighted Voxel can be calculated as

where denotes the value in the regular voxel, and is set to 26. Specially, we define when , or .

Methods Illustration

The network architecture of 3D-R2N2 and Weighted Voxel. Both of them consist of an encoder, a 3D convolutional LSTM, and a decoder. In 3D-R2N2, the reconstructed voxels are composed of zeros and ones, while in Weighted Voxel, the voxel values are filled with integers.

Comparison of Reconstruction Results with 3D-R2N2

Reconstruction samples of (a) cars (b) cabinets (c) speakers (d) sofas on the ShapeNet testing dataset. The Weighted Voxel preserves more structural details of 3D objects

Prerequisites

Install `pygpu`

Please follow the instruction on the homepage of gpuarray.

Install Other Python Denpendencies

pip3 install -r requirements.txt

Create `.theanorc`

Please paste following lines to ~/.theanorc:

[cuda]
root = /opt/cuda    # Please change it with your CUDA installation path

[global]
device = cuda0      # Please change it with your GPU device ID
floatX = float32

Download Dataset

Use following commands to download the ShapeNet dataset:

cd /path/to/the/repository
mkdir -p datasets/ShapeNet
cd datasets/ShapeNet

wget http://cvgl.stanford.edu/data2/ShapeNetRendering.tgz
tar -xf ShapeNetRendering.tgz

wget http://cvgl.stanford.edu/data2/ShapeNetVox32.tgz
tar -xf ShapeNetVox32.tgz

Use following command to generate Weighted Voxel dataset:

cd /path/to/the/repository
python utils/binvox_weighting.py datasets/ShapeNet/ShapeNetVox32 datasets/ShapeNet/WeightedShapeNetVox32

Get Started

Use following command to train the neural network:

python3 runner.py

Use following command to test the neural network:

python3 runner.py \
      --test \
      --weights output/weights.npy

The pretrained model can be downloaded from here (206 MB).

License

This project is open sourced under MIT license.

hzxie/Weighted-Voxel