Overview

Inside Music YouTube Edition is a web app that uses Artificial Intelligence to separate the instruments of a YouTube song, and places them in a virtual room to hear the songs in a totally inmersive environment. The original edition was created by Song Exploder and Google Creative Lab as a WebVR Experiment that lets you step inside a song, giving you a closer look at how music is made.

The audio of the available songs is not included in this repo. To add your own YouTube audio and play it, use the interface to convert it.

This is an Experiment, not a Google product.

Demo

Try it now! https://3dyt.hcilab.ml

Technology

Inside Music Youtube Edition is built with aframe, THREE.JS, Tone.js, and Spleeter.

Contributors

Interaction

Select a song from the menu. The stems of the song will appear in a circle around you, each represented by a sphere. In Browser Mode, tap the spheres to turn them on or off. In VR Mode, you can use your controller to toggle their state. On Google Cardboard, you will have a retical (a small circle in front of you eye) which can be used to turn the stems on and off.

Build

To build a local version of Inside Music, you will need to have Node.js and webpack installed (<4.0).

In the terminal, inside the project's directory, install all of the project's dependencies:

npm install

Then compile the source code with webpack:

webpack -p

You can now run a local server such as http-server. To install http-server, run npm install -g http-server. You can now view the site in a browser.

You can easily build the tool and expose the server by running:

npm run restart

Server side (music processing)

The provided code reads the songs list from a public URL. If you want to test the tool in your local environment, you have to set the audioDirectory variable to 'audio' in the src/Config.js file.

You have to install the requirements before executing the python script:

pip install -r requirements.txt

The script can be executed via command line arguments:

python app.py --url [YOUTUBE_VIDEO_URL]

or, via web:

python app.py --web

curl http://localhost:4000/?url=[YOUTUBE_VIDEO_URL]

Examples:

python app.py --url https://www.youtube.com/watch?v=cNAdtkSjSps

curl http://localhost:4000/?url=https://www.youtube.com/watch?v=cNAdtkSjSps

After processing a song, you can play it in the web app by sending the video ID and the duration in seconds separated by comman in the URL hash as follows:

open http://localhost:8080/#[VIDEO_ID],[DURATION_IN_SECONDS]

Example:

open http://localhost:8080/#cNAdtkSjSps,351

LICENSE

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

This project has been modified by Carlos Toxtli to add support for YouTube songs.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

toxtli/webvr-360-music