CS499R - Machine Learning for Computer Graphics
This is a seminar course with a majority of the lecture time devoted to student-led presentations. In each session, a student will present and critique one or two assigned papers related to the term's topic. They will discuss the paper or papers, and offer their own thoughts about what could have been done differently and how the work could be extended. The rest of the students are expected to read the papers, attend the presentations, and participate in the following discussion. Students who take the course for credit are also expected to complete a small project at the end of the term.
1. Topics covered:
- Differentiable Rendering
- Shape Representations
- Inverse Rendering
- Systems for Differentiable Graphics
- Learning Physical Phenomenons
- Reinforcement Learning for Animation
- Motion Capture
2. Schedule:
- Differentiable Rendering
- OpenDR: http://files.is.tue.mpg.de/black/papers/OpenDR.pdf
- Neural 3D Mesh Renderer: https://arxiv.org/abs/1711.07566
- Differentiable Surface Splatting: https://igl.ethz.ch/projects/differentiable-surface-splatting/DSS-2019-SA-Yifan-etal.pdf
- Differentiable Monte-Carlo Light Transport: https://people.csail.mit.edu/tzumao/diffrt/
- Neural Volumes: https://research.fb.com/publications/neural-volumes-learning-dynamic-renderable-volumes-from-images/
- Geometry 1:
- Deep SDF: https://arxiv.org/abs/1901.05103
- Cubic Stylization: http://www.dgp.toronto.edu/projects/cubic-stylization/
- Deformed Distance Fields: https://cs.dartmouth.edu/~wjarosz/publications/seyb19nonlinear.html
- Deep Cuboids: https://isunchy.github.io/projects/cuboid_abstraction.html
- StructureNet: https://cs.stanford.edu/~kaichun/structurenet/
- Geometry 2:
- SDM-Net: http://geometrylearning.com/sdm-net/
- AtlasNet: https://arxiv.org/abs/1802.05384
- Pixel2Mesh: https://arxiv.org/abs/1804.01654
- MeshCNN: https://arxiv.org/abs/1809.05910
- Sketch CNN: https://haopan.github.io/sketchCNN.html
- http://geometrylearning.com/index.html
- Scene Represntation
- Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
- Image Generation from Scene Graphs: https://arxiv.org/abs/1804.01622
- Neural Procedural Reconstruction for Residential Buildings
- Neural Turtle Graphics for Modeling City Road Layouts
- https://towardsdatascience.com/neural-networks-and-the-future-of-3d-procedural-content-generation-a2132487d44a
- Materials
- Deep Inverse Rendering: https://gao-duan.github.io/publications/mvsvbrdf/mvsvbrdf_low_resolution.pdf
- TileGAN: https://github.com/afruehstueck/tileGAN
- Single Image SVBRDF: https://www-sop.inria.fr/reves/Basilic/2018/DADDB18/
- Reflectance Capture: http://www.cad.zju.edu.cn/home/hwu/publications/autoenc/project.html
- Systems
- Physics Simulation
- tempoGAN: https://ge.in.tum.de/publications/tempogan/
- DRL Fluids: http://gamma.cs.unc.edu/DRL_FluidRigid/
- Data-Driven Synthesis of Smoke Flows with CNN-based Feature Descriptors
- Deep Fluids: A Generative Network for Parameterized Fluid Simulation
- Subspace Neural Physics: Fast Data-Driven Interactive Simulation: http://www.cim.mcgill.ca/~derek/publication.html?id=89
- Animation 1
- Michiel van de Panne: https://www.cs.ubc.ca/~van/
- KangKang Yin: https://www2.cs.sfu.ca/~kkyin/
- Learning to Move: SIGGRAPH ASIA 2019, SIGGRAPH 2019, SIGGRAPH 2017,
- Deep Thoughts on How THings Move: SIGGRAPH 2018
- Motion Control: SIGGRAPH 2016
- Animation 2
- Plants: SIGGRAPH 2016
- Data-Driven Physics for Human Soft Tissue Animation
- Taking Control: SIGGRAPH 2015
- Deep Incremental Learning for Efficient High-Fidelity Face Tracking
- Character Animation: SIGGRAPH ASIA 2018
- Learning to Animate: Eurographics 2019
- Computational Display
- DeepFocus: Learned Image Synthesis for Computational Displays
- DeepFovea: Neural Reconstruction for Foveated Rendering and Video Compression using Learned Statistics of Natural Videos
- Learned Large Field-of-View Imaging With Thin-Plate Optics
- Wildcard
- Final Project Presentations