/what-you-dont-know

An interactive music video made for VR on the web in partnership with Ghostly International.

Primary LanguageJavaScriptMIT LicenseMIT

What You Don't Know

An interactive music video made for VR only on WITHIN in partnership with Ghostly International.

With music by Matthew Dear.

Experience it for yourself here: http://jonobr1.github.io/what-you-dont-know/release/

Project Environment

There are three main resources for this project:

  • /inspiration: Visual inspiration links for the project
  • /release: The current stable build for the project
  • /staging: The latest build for the project
  • /editor: A WYSIWYG timeline editor for the project
  • /prototypes: All interaction prototypes

These folders all require you run a local server on your computer. A good cross-platform way to get that started is by installing Node.js, opening up a command line prompt, and typing npm install -g http-server. Once you have that, point your command line to this project's root directory and type http-server. This will create a local server at http://localhost:8080. The following instructions will use this local server's details as the example.

Below are more details for each of the pages and how to use them.

/release

is the most current place to see our current build of the project.

/staging

is the build used for WITHIN's staging website.

/editor

gives you an empty viewport to see and manipulate how we're choreographing What You Don't Know. Once this is open in your web browser go to File > Reset to insure the a clean version of the project is loaded. When doing this, the editor loads /release/experience.json into the editor, insuring that what you're working with is the published version of the project. Similar to video editing software you can hit the spacebar to play and pause the video with some additional controls to see how animations are laid out.

/prototypes

is a directory of one off HTML5 webpages exploring different interactions, graphics, interfaces, and previous builds.

Roadmap

  • Exiting VR Bug in Oculus Browser
  • Fix Animation Skipping on Sped up Tracks
  • Choreograph Multiple Camera Angles for non VR Contexts
  • Account for Portrait Screen Sizes on Page Load (Design)
  • Make UI Text All Same Size
  • Update to r107
  • Add Polar Spheres (Reference 1 • Reference 2)
  • Add Title Page (UX)
  • Make Playhead First Read (UX)
  • Add Arrows to Compass to Guide Dragging
  • Add Global Reset Button
  • Add Zen Mode (Hide UI)
  • Add Random Parameterizer
  • Add Custom Sound / Noise When Scrubbing
  • Create Clear UI for EQ Changes
  • Spatialize Audio Tracks
  • Choreograph Polar Spheres
    • Separate Blob (vertex) Animation and Liquid (fragment) Animation
    • Rotate and Position in Relation to the Ring
    • Place Initial Position at Normal Viewing Position
    • Set Lyrics and Credits via the polarSpheres.userData.setTexture method
  • Choreograph Lyrics
    • Opacity Animation Based on Vocals
  • Choreograph Debris
    • Experiment with amount of visible Debris
    • Experiment with regions of Debris
    • Experiment with size through FFT
    • Experiment with Field Movement
    • Experiment with Turbulence
  • Choreograph Ring
    • Experiment with Changing x / z Rotation
    • Experiment with x / z Position
    • Choreograph Alpha Map Texture Repeat on "Oh" Vocals
    • Opacity Inversely Tied to SFX - Alpha Map Intensity
    • Y Position Oscillation Tied to Instrumental
  • Choreograph Backdrop
  • Choreograph Boids
    • Diversify Geometries
    • Animate / Oscillate Distance from Ring
    • Three Part Melody Should Trigger Boids
    • Tie TWEEN of Boids to player.currentTime not requestAnimationFrame
  • Add Custom THREE.LightProbe
  • Remove playbackRate Change on Playhead (keep it simple)
  • Make Text Geometry Based Instead of Based out of a Texture
  • Add Lyrics
  • Add Meta-Lines around Lyrics
  • Add Child Shapes to Ring
  • Add Lyric Interaction
  • Hide Controller on Leaving VR
  • Add Ghostly Logo
  • Instance Green Buttons
  • Add renderer.initMaterial( material ) to All Materials on Setup
  • Add Reset Button to Each filter
  • Add Label to Each Filter
  • Add Compass to Orient the World Based on Audio Time
  • Fix Orientation Offset by Compass on Playhead + Lights Animations
  • Remove Twist CatmullRomCurve3 Calculations on Boids
  • Match Lighting on Boids
  • Create Secondary Instance of TWEEN Bound to Frame.Player.currentTime
  • playbackRate Persists on loop, but Colors Reset to Gray (default)
  • Improve Scheduler on Audio Sequencing
  • Increase Height of Playhead.Cylinder
  • Make Debris a Sphere
  • Make Biquad Filter More Apparent (audio & visual)
  • Add Color Palette Changes on Speed Modification
  • Modify OrbitControls to Ease Rotation
  • Modify Materials on Buttons to be from Pixar 130
  • Fix Multiple VR Controllers
  • Fix Safari Playback Issues
  • Fix touch events for Paddles
  • Remove THREE.VRController Dependency
  • Update to r101