/ar-storytelling

Project for SIGGRAPH 2022 paper "Interactive Augmented Reality Storytelling Guided by Scene Semantics"

Primary LanguageC#

Interactive Augmented Reality Storytelling Guided by Scene Semantics


This repository is the implementation for our SIGGRAPH 2022 paper: "Interactive Augmented Reality Storytelling Guided by Scene Semantics"

[Paper] [Video] [Project page]
Changyang Li, Wanwan Li, Haikun Huang, Lap-Fai Yu

Introduction

We present a novel interactive augmented reality (AR) storytelling approach guided by indoor scene semantics. Our approach automatically populates virtual contents in real-world environments to deliver AR stories, which match both the story plots and scene semantics. During the storytelling process, a player can participate as a character in the story. Meanwhile, the behaviors of the virtual characters and the placement of the virtual items adapt to the player's actions.

Setup

  • Unity3D 2020.1.16f1
  • MRTK 2.7.2.0 + Microsoft Hololens 2

To deliver full AR experiences, our framework relies on some commercial Unity3D assets for virtual models, animations, navigation, etc. For example:

Citation

@article{arstorytelling,
    title={Interactive Augmented Reality Storytelling Guided by Scene Semantics},
    author = {Changyang Li and Wanwan Li and Haikun Huang and Lap-Fai Yu},
    journal = {ACM Transactions on Graphics (TOG)},
    volume = {41},
    number = {4},
    year = {2022},
    publisher={ACM New York, NY, USA}
}