Reality Mixer
Oculus Quest Mixed Reality For iOS
This is a standalone Oculus Quest Mixed Reality app for iOS. It is able to generate VR gameplay videos in Mixed Reality without a PC and without a green screen.
Click on this link to join the TestFlight beta, or follow the instructions below to install it using AltStore.
Follow us on Twitter for more updates!
Examples
| Game | YouTube video | Photos |
|---|---|---|
| Beat Saber | Link |   |
| SUPERHOT | Link |   |
| The Thrill of the Fight | Link |   |
| Space Pirate Trainer | Link |   |
Requirements
- Oculus Quest 1 or 2 with the Oculus MRC Calibration app version 1.7 installed.
- iPhone or iPad with an A12 chip or newer, running iOS 14. The LiDAR sensor is optional but recommended for better results.
- 5 Ghz WiFi network.
- A compatible Quest VR application/game (check this page for a list of the officially supported games).
Note that this app is still just a prototype and it's still being developed, use at your own risk.
Installation
AltStore
-
Follow these instructions to install and configure AltServer on your PC or Mac, then install the AltStore app on your iPhone/iPad.
-
Open the camera app on your iPhone/iPad and scan this QR code:
OR
- Open this URL on your iPhone/iPad:
altstore://install?url=https://github.com/fabio914/OculusQuestMixedRealityForiOS/releases/download/0.1.3/RealityMixer.ipa
TestFlight
Use this link to join the beta. Limited to the first 10 thousand users.
Instructions
TO-DOs
-
Optimize the video stream (adopt a lower frame rate if the iPhone/iPad can't keep up with the stream);
-
Investigate the audio delay;
-
Improve the calibration screen and fix the positioning of the controller models;
-
Add a "green screen mode" that will allow the user to use the app with a regular green screen, allowing the app to run on devices without an A12 chip;
Wishlist
These are features that could be implemented in the future:
-
Create a custom calibration app for the Quest to replace the Oculus MRC app, allowing the user to calibrate from VR itself;
-
Use ARKit's body tracking to replace the image from the camera with an avatar;
-
Allow the user to adjust the FOV of the mixed reality capture;
-
Allow the app to cast the video output (although this should already be possible if you long press the "Screen recording" button on "Control Center" and then select a streaming app);
-
Android version;
Credits
This project is based on the Oculus MRC plugin for OBS.
It uses SwiftSocket to handle the TCP connection with the Oculus Quest, FFMPEG with Apple's VideoToolbox to decode the stream, and ARKit for the "virtual green screen".
The right and left controller models are modified versions of the oculus-touch-v3 models from the WebXR Input Profiles.
It also requires Carthage to download its dependencies.