/OculusGoStreamer

Stream your desktop to the Oculus Go / GearVR browser

Primary LanguageJavaGNU General Public License v3.0GPL-3.0

OculusGoStreamer

A desktop streaming application for the OculusGo and GearVR (although untested). Basically works in any browser. Tested on Android phone. Reported working with Nintendo Switch as well.

------ Made by Kaj Toet ---

  • Cross platform using Java.

  • Captures desktop with VLC using VLCj library.

  • Compresses video using JPEG, PNG or GIF.

  • Uses custom interlacing and interframe compression.

  • Streams over websockets.

  • Supports virtual keyboard. Input is copied to clipboard on PC and then copy-pasted, so it can handle all characters.

  • Supports touch for mouse.

  • Supports native keyboard events.

  • Test results: ~22mbit/s at 60fps for a Call of Duty Youtube video @ 1920x1080. Will be lower if the client browser has a lower resolution as the video is downscaled before compression. The OculusGo browser has a resolution of 800x480 when not in fullscreen.

  • Stereo sound support implemented! Still some more latency inconsistencies, but works pretty well overall.

  • WebVR support: visit "http://address:port/webvr.html"

Instructions:

  • Download the Jar: https://github.com/Kjos/OculusGoStreamer/raw/master/out/artifacts/OculusGoStreamer_jar/OculusGoStreamer.jar
  • Run from commandline: java -jar OculusGoStreamer.jar
  • Or any other way you run Java programs.
  • The console will give details about Java and VLC architectures. On Windows, a separate copy of VLC that matches Java architecture will be downloaded and put in a new directory 'vlc-override'.
  • Visit the address printed in the terminal in your PCs browser first The address will have port 7578 by default. Note: The address printed may not be the correct address for your network architecture. If you get a 404, you need to use another address.
  • It will print "Polling minimum achievable latency" for a few seconds, while the browser shows a black screen.
  • And then should begin to stream.
  • Visit the address in your browser (close the other stream, only 1 stream at a time can be open at one time).
  • Click the keyboard icon to open the virtual keyboard.
  • Reloading the browser page will also reload config.json from file.
  • Change raise resolution by lowering 'initial-scale=1.0' in index.html. 0.5 is resolution multiplier of two. Same for lowering resolution, but vice versa. The console output will print your resolution, for example: 'Recorder set up, width: 1096, height: 617'.

Notes

  • Requires Java 1.6 or higher. VLCj for Mac OSX can only handle JVM 1.6 (? not sure).
  • The vlcj included library requires version 2.1.0+ of VLC to be present.
  • VLC 3 reported not working for Mac. Possibly Windows and Linux too (unverified), however Windows downloads VLC 2.2.6 from start.
  • VLC 2.2.6 confirmed working. Any 2.x should probably work fine.
  • VLC needs to be same architecture as JVM. The console will print your Java and VLC architectures.
  • If not working from start, you can download VLC here: http://download.videolan.org/pub/videolan/vlc/
  • You can put the contents of VLC in a directory called 'vlc-override' next to the Jar, and it will use that instead of the default VLC.
  • More information about vlcj: http://capricasoftware.co.uk/#/projects/vlcj/tutorial/prerequisites

For OculusGo specifically:

  • At 10% battery the connection or browser will be limited and streaming will stutter.
  • When you have the headset off, the browser will keep running in the background, so close the app when you're done.
  • If you're experiencing issues connecting to the webserver, try rebooting the Oculus Go.

config.json example

See: https://github.com/Kjos/OculusGoStreamer/blob/master/config.json

Todo

  • Check SBS 3D support.
  • Blit cursor without VLC for cross platform and dynamically toggle on/off on movement pointer.
  • Test sound different browsers.
  • Improve latency. TCP makes it so packets can get stacked. Needs to send and receive frame timestamps to be able to read the actual latency and account for it. (Done)
  • Browser side pull up menu so more options can be accessible. (Done)
  • Such as fullscreen support. (Done)
  • Or switching displays. (Done)
  • Improve input latency for mouse cursor. Don't know why or if it can be improved much, but I think it can. (Mostly done by fixing the video latency. Don't think it can be improved anymore.)
  • Support bluetooth keyboard and other controllers.
  • Video compression needs overall improvement. Perhaps the interframe method can be removed, but I think the bandwidth usage might otherwise become too high at cost of quality. Overall interframe compression is GPU-costly for browserside. (Mostly done)
  • Sound is missing. (Done, needs some improvement though)
  • Bundle everything as single executable. (Website dir now extracts from Jar, for Windows VLC is downloaded)
  • Config file support or commandline parameters if sufficient. (Done, config.json)
  • Maintain aspect ratio. Will also lessen bandwidth usage a tiny bit. (Done)