alvr-org/alvr-visionos

Support for Quest Controllers w/ Headset as Base Station?

zaptrem opened this issue · 23 comments

zaptrem commented

Would it be possible to/does ALVR support using Quest controllers with Vision Pro by placing the Quest headset on a nearby table/vantage point?

zarik5 commented

ALVR supports "tracking reference only" mode for the headset, which means you can connect controllers and use another headset for video. The problem is we don't support running multiple ALVR instances at the same time. To properly support this we need to connect multiple headsets to the same ALVR streamer and select only one to stream video to.

akre54 commented

Does it support any controllers? I've gotten as far as the lobby.

I have a DualSense that I've tried pairing to both the PC and the Vision Pro, and in neither case does Steam VR register the button inputs. It does, however, recognize that a standard controller is present (it changes the graphic to an Xbox controller) but none of the button inputs seem to register. Do I need to do anything to enable this?

Fox-McCloud commented

@akre54 not paired with the headset itself.

You can use external controllers, however.

Predominant method you're likely to run into is:

  • 2 SteamVR basestations
  • 2 Basestation tracked controllers (Vive wands or Index controllers---I do not recommend the wands)
  • 2 Dongles for each of the controllers
  • A piece of software called OVR Space Calibrator

The above setup is essentially its own 6DOF tracked space; OVR Space Calibrator then brings the AVP's playspace and the lighthouse tracked playspace into alignment with each other.

  • 👍1
akre54 commented

Okay, good to know it works with the Index setup - let me see if I can borrow a friend's. Would be great if it could work with the quest's inside-out setup. Thanks!

pwhizzard commented

I wonder if anything could be done with the controller data forwarding server 🤷 https://x.com/studio_vron/status/1759358156927365483?s=61

akre54 commented

This guy has managed to make it work with ODTKRA, Oculus Link, OpenVR-SpaceCalibrator, and some tape over the Pro's proximity sensor. 

This does require a spare Quest headset just sitting around to act as man-in-the middle, and at a grand total of an apple vision pro, a mac with apple silicon for xcode, a pc, quest controllers, a quest headset, and about 6 separate pieces of software, I may've just hit my limit for the amount of expensive tech Rube Goldberg I want to dedicate to this project unfortunately. Hopefully someone comes out with inside-out controllers that can be paired to a PC with no headset in between

pwhizzard commented

Yeah that is incredibly fair. Thanks for the share though I may actually try that since I have a Quest Pro.

akre54 commented

Awesome. Looking forward to seeing the results!

pwhizzard commented

Well I got it running. It actually wasn't that hard with the tutorial video you shared from Mysticle. It feels really good while it works, but there are frequent disconnects. Super interesting as a proof of concept if you ask me. I'll try it a few more times and see if I can figure out the interference and disconnects. https://x.com/Pwhizzard/status/1764804389284581839?s=20

pwhizzard commented

OK, this may actually be more viable than I thought. After updating quest firmware and downloading Meta Quest developer hub, starting everything up after initial set up is much more simple and can all be done from the computer. I also just played for an hour without it disconnecting. If anyone is interested in chatting more about this, definitely let me know.

joshuadmatthews commented

I’d love to discuss @pwhizzard , I just got a quest 3 so I think I now have all the necessary gear. Could definitely use your knowledge of how to get it all working!

pwhizzard commented

Sure thing, @joshuadmatthews. I’ll note that Quest 3 included Touch Plus controllers will require line of sight from the Quest but it should work. Quest Touch Pro controllers do their own SLAM/track themselves so don't need line of sight from a Quest headset.

I originally followed this video https://www.youtube.com/watch?v=1zkp2fXzP4k which is mostly still up-to-date.

I will list the steps and leave some insight below:

Initial setup:

Running:

  • Disable proximity sensor and initiate Link or AirLink - either on Quest headset or through Quest developer hub
  • Run ODTKRA - give it some time to run and don’t touch anything until it’s done. It needs to open oculus debug console and make changes without app focus being lost.
  • Oculus Touch Link folder from SteamVR > Drivers
    • Run ovr_dummy.exe
    • Run ovr_test.exe
    • Note: A window labelled “title” should open - make sure “Enable External Tracking” is checked.
  • Run ALVR server on PC & launch SteamVR from it
  • Run ALVR client from Vision
  • Open OpenVR-SpaceCalibrator
    • Reference space: use either your Vision HMD or a hand being tracking
    • Target space: select oculus link and one of the controllers.
    • Start calibration: I select fast then move in a figure 8 very slowly one time and that seems to work for a while.

Random bug - For Touch Pro controllers I found a couple times where I initially needed to let the Quest headset physically “see” the controllers themselves otherwise I only got 3DoF tracking data. I wouldn’t have to do this again but just at the beginning of the session.

Let me know if there are any questions you have. Also, free free to let me know if you want to chat in some other form.

wendal2841 commented

Is it possible to connect Quest Touch Pro controllers without the glasses themselves, buy only the controllers?

  • 👍1
pwhizzard commented

I haven't found a way to connect the controllers directly to PC (granted I've only poked around that a little). You can use a Quest 2 and pair that with the Touch Pro controllers though. And you can find a relatively cheap, used Q2.

joshuadmatthews commented

@pwhizzard I am having trouble towards the end, if I run ALVR on my Vision Pro and then take it off to run the OpenVr Space Calibrator, the VP goes to sleep as soon as I take it off and disconnects from ALVR. How do you work around that?

joshuadmatthews commented

Also not sure how to disable proximity sensor, I can't get my quest to peer with the develper hub unfortunately.

pwhizzard commented

I never had to pair it with the developer hub but I have previously paired it with the Oculus pc app. I wonder if doing that first would suffice. https://www.meta.com/help/quest/articles/headsets-and-accessories/oculus-rift-s/install-app-for-link/. Either way it'll have to be paired there anyway because you'll need to kick off Link/AirLink.

And we're chatting on Discord but for posterity re- Vision pro disconnecting when taking off to run OpenVR SC:
"If you can manage to point the unsynced controllers, then you can actually bring it up from in the headset SteamVR menu
Otherwise you peak without removing Vision or walk over to your PC if it's far enough from the 1.5m center area on Vision and it will fade in your monitor view so you can see it from headset."

And if you don't want to use developer hub you can literally just tape something over the proximity sensor on the Quest headset.

kensuddaby commented

I’m not sure if this is the right thread so let me know. I have used ALVR on AVP, but my computer right now is too slow to support decent resolution with Microsoft Flight Simulator or DCS. (NVIDIA 3070ti in a laptop).

So what I’m doing for now is streaming fairly high res 2D using Moonlight/Sunshine.

However, I need head tracking data for MSFS. I am using Aruco with OpenTrack and it works great, but requires a webcam and also a cardboard on my head.

What would be great for guys like me would be a windowed (not volume) AVP ALVR app that just passes head tracking data to steam VR or even OpenTrak.

I’m guessing this wouldn’t be too hard to implement, but the use case may be niche.

shinyquagsire23 commented

@kensuddaby not possible, visionOS only gives head tracking data to immersive applications, so Moonlight would have to implement an immersive space itself (which would shut out all other windows except its own)

kensuddaby commented

Good to know. Thanks! ALVR can be used in 2-D and it does do head tracking. I’m wondering if there will be away to adjust the position and size of the screen in 2-D ALVR. @shinyquagsire23

shinyquagsire23 commented

not sure what you mean tbh? ALVR itself doesn't do 2D afaik, SteamVR has a 2D game launcher thing I guess but passing rotations to the 2D games would be outside of ALVR's scope tbh

kensuddaby commented

@shinyquagsire23 I'm running ALVR on the computer and steam VR. With MSFS, you launch in 2D and then only initiate 3d within the MSFS app. In ALVR, I'm presented a room with a screen with the 2D version of MSFS (haven't triggered VR yet). At that point Steam VR is passing through the head tracking to MSFS and I can play it in 2D. The screen is about 30degrees up from eye level so it's not practical.

dantesvictus commented

I haven't found a way to connect the controllers directly to PC (granted I've only poked around that a little). You can use a Quest 2 and pair that with the Touch Pro controllers though. And you can find a relatively cheap, used Q2.

can you please try to play Eleven tennis game if you have it. and share your experience if its playable.