/panoramic_material

awesome indoor panoramic materials
awesome indoor lighting papers

Some indoor panoramic datasets and related indoor lighting papers.

Table of Contents

Panoramic Indoor Datasets

360Roam
360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360ᵒ Radiance Fields
Huajian Huang, Yingshu Chen, Tianjia Zhang and Sai-Kit Yeung
[To be released]

Laval HDR database.
Learning to Predict Indoor Illumination from a Single Image
Gardner, M.-A., Sunkavalli, K., Yumer, E., Shen, X., Gambaretto, E., Gagné, C., and Lalonde, J.F.
ACM Transactions on Graphics (SIGGRAPH Asia), 9(4), 2017.
[Paper] [Project] [HDR Data] [SUN360 Data Info]

Features:

  • Real indoor scenes.
  • 2100+ high resolution and fully HDR indoor panoramas, captured using a Canon 5D Mark III and a robotic panoramic tripod head.
  • Diverse lighting conditions.
  • Around 400 light sources hand-labeled LDR panoramas (SUN360).

360-Indoor.
360-Indoor: Towards Learning Real-World Objects in 360° Indoor Equirectangular Images
Shih-Han Chou, Cheng Sun, Wen-Yen Chang, Wan-Ting Hsu, Min Sun, Jianlong Fu
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2020.
[Project] [Data]

Features:

  • Real indoor scenes
  • Information:
    • Object annotations (37 cateogories, incld. light)

Structured3D (panoramas).
Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling
Jia Zheng*, Junfei Zhang*, Jing Li, Rui Tang, Shenghua Gao, Zihan Zhou
European Conference on Computer Vision (ECCV), 2020
[Code] [Paper] [Supplementary Material] [Benchmark]

Features:

  • Synthetic indoor scenes
  • 3,500 scenes, 21,835 rooms
  • Rendering contains empty/unfurnished (w/ all lights), full(w/ all lights), simple versions(w/ all lights)
  • Information:
    • 2d rendering (raw, cold, wram lighting)
    • normal
    • depth
    • albedo
    • semantic (also annotate light fixtures)
    • structure/layout annotation (wall, ceiling, floor).

Matterport3D.
Matterport3D: Learning from RGB-D Data in Indoor Environments
A. Chang, A. Dai, T. Funkhouser, M. Halber, M. Niessner, M. Savva, S. Song, A. Zeng, Y. Zhang
International Conference on 3D Vision (3DV 2017) [Project] [Code] [Paper]

Features:

  • A large-scale RGB-D dataset containing 10,800 panoramic views from 194,400 RGB-D images of 90 building-scale scenes.
  • Information:
    • RGB
    • HDR
    • Depth
    • Normal
    • Semantic
    • Camera poses
    • Textured 3D mesh

Matterport3D Variants - Im2Pano3D (RGB-D Panorama).
Im2Pano3D: Extrapolating 360° Structure and Semantics Beyond the Field of View
Shuran Song, Andy Zeng, Angel X. Chang, Manolis Savva, Silvio Savarese, Thomas Funkhouser
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2018.
[Project] [Code] [Paper]

Features:

  • Generate RGB-D Panorama imaes from SUNCG dataset(Not available now) and Matterport dataset.
  • Information:
    • color image
    • semantic segmentation
    • depth
    • plane encoding (plane distance and surface normal)

ZInD.
Zillow Indoor Dataset: Annotated Floor Plans With 360º Panoramas and 3D Room Layouts
Steve Cruz*, Will Hutchcroft*, Yuguang Li, Naji Khosravan, Ivaylo Boyadzhiev, Sing Bing Kang
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021
[Paper] [Supplementary Material]

Features:

  • Real residential rooms
  • 1575 scenes (71,474 panoramas in 1524 unfurnished rediential homes)
  • Unfurnished rooms (decorated with basic lights, e.g. ceilling lamps)
  • Information:
    • Panoramas of daytime captures with lights on
    • Layout annotation (wall, ceilling, floor, window, door)
    • Floor plans (raster, vector)

Other Indoor Datasets

InteriorNet (LARGE)
InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset
Wenbin Li, Sajad Saeedi, John McCormac, Ronald Clark, Dimos Tzoumanikas, Qing Ye, Yuzhong Huang, Rui Tang and Stefan Leutenegger
British Machine Vision Conference (BMVC), 2018.
[Project] [Paper]

Features:

  • Professional designed indoor CAD models
  • 10,000 scenes
  • 1.7 million rooms
  • Released dataset (consisting of the rendered sequences and images) as well as ExaRenderer, ViSim, and a subset of the 3D models and layouts used for evaluations.
  • Information:
    • rendered perpective sequences and panoramas (pano: except for HD7)
    • rearrangement
    • random lighting
    • depth
    • semantic instance
    • semantic class segmentation.

The InteriorNet dataset is split into the following: (ref: InteriorNet2ROSBag)

  1. 705 scenes with 3 ground truth trajectories and IMU data, color and depth images, ground truth instance images, with 1000 frames each, at 25 frames per second. Each trajectory is available in regular lighting and random lighting. [HD1-HD6]

  2. 20000 scenes with 20 random views containing ground truth camera pose, color and depth images and ground truth instance and object class images. These views are also available in both regular lighting and random lighting. [HD7]

Replica.
The Replica dataset: A digital replica of indoor spaces
Julian Straub and Thomas Whelan and Lingni Ma and Yufan Chen and Erik Wijmans and Simon Green and Jakob J. Engel and Raul Mur-Artal and Carl Ren and Shobhit Verma and Anton Clarkson and Mingfei Yan and Brian Budge and Yajie Yan and Xiaqing Pan and June Yon and Yuyang Zou and Kimberly Leon and Nigel Carter and Jesus Briales and Tyler Gillingham and Elias Mueggler and Luis Pesqueira and Manolis Savva and Dhruv Batra and Hauke M. Strasdat and Renzo De Nardi and Michael Goesele and Steven Lovegrove and Richard Newcombe
arXiv, 2019.
[Code] [Paper]

Features:

  • 18 3D reconstructed scenes
  • provide own renderer: ReplicaRenderer
  • Information:
    • 3D mesh
    • Some surface materials such as glass, mirrors
    • Texture
    • 3D segmentation
    • ReplicaRenderer

Hypersim.
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, Joshua M. Susskind
ICCV, 2021.
[Project] [Apple News] [Code] [Paper]

Features:

  • 461 scenes
  • 77,400 images
  • 3D meshes
  • HRD images
  • Semantic segmentation
  • Instance segmentation
  • Diffuse + residule for images intrinsics
  • rendered in 3DMax, VRay renderer
  • Information of data:
    • Lossy preview of prespective images
    • Lossless prespective images data (hd5f)
    • GT camera information
    • Rendering settings
    • 3D meshes (hd5f)
    • 2D and 3D segmentations

Papers

Lighting related papers using panoramas

Learning to Estimate Indoor Lighting from 3D Objects
Weber, Henrique, Donald Prévost, and Jean-François Lalonde.
International Conference on 3D Vision (3DV). IEEE, 2018.
[Code] [Project] [Paper]

Deep Parametric Indoor Lighting Estimation
Marc-Andre Gardner, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Christian Gagne, Jean-Francois Lalonde.
In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2019.
[Reproduce Code] [Paper]

Deep Lighting Environment Map Estimation from Spherical Panoramas
Gkitsas, V., Zioulis, N., Alvarez, F., Zarpalas, D., & Daras, P.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020.
[Code] [Project] [Paper]

[hot:fire:] Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination
Pratul P. Srinivasan, Ben Mildenhall, Matthew Tancik, Jonathan T. Barron, Richard Tucker, Noah Snavely.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[Code] [Paper]

  • I/O: panoramas, panoramic lighting map
  • Input: stereo pair of RGB images
  • Outpu: a multiscale RGBA lighting volume
  • Data: InteriorNet

[hot:fire:] Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, Manmohan Chandraker.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2020.
[Code and Data] [Project]

  • Data: OpenRooms (CVPR2021)

HDR Environment Map Estimation for Real-Time Augmented Reality
Gowri Somanath, Daniel Kurz. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[Code] [Project] [Paper]

  • I/O: LDR normal single image; HDR panoramic environmental map
  • Data: Laval HDR Database

Lighting, Reflectance and Geometry Estimation from 360° Panoramic Stereo
Li, Junxuan, Hongdong Li, and Yasuyuki Matsushita.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. [Code] [Paper]

  • I/O: two (top+bottom) stereo panoramas (and predicted depth); panoramic lighting, reflectance and geometry estimation
  • Dataset: Structured3D panoramas (training), 360SD-Net (stereo input, depth estimation)