Image is upside-down when rendering with Oculus OpenXR runtime
Closed this issue · 10 comments
While testing Servo on my Win10 machine I wanted to track down the cause of the in-headset image being flipped despite my earlier patch updating the projection matrix logic. What I found was that SteamVR via Virtual Desktop displayed the image rightside-up, but with the Oculus runtime it renders upside down. I'm not sure what the specific cause of this is yet.
Only other instance of a similar-sounding issue I could find, unfortunately the XR_FB_composition_layer_image_layout extension is only available on headsets, not the PC Link runtime.
https://github.com/servo/webxr/blob/main/webxr/openxr/mod.rs#L635 might be related. Not sure how to determine if it's needed for a particular openxr implementation though.
Oh wow I didn't even see that, that definitely seems like the most likely culprit though. We can get runtime name once the XrInstance is created, so it should just be a matter of keeping an internal flag for it. I'll test and make sure that's the actual fix first though.
Hmm, commenting those two lines out didn't seem to have any effect. I'll keep digging, maybe see what the actual FOV values are that we're getting
Update: Sent a message in the Khronos Discord and got relevant info
This would explain why commenting out those lines did nothing, since they were never actually swapping in the first place
Matthieu suggested inverting the projection matrix being calculated in fov_to_projection_matrix, so I'll try that
So by swapping fov up/down in fov_to_projection_matrix instead we can fix the texture coord mismatch, but it seems like now there's a culling issue (planets look odd and the spherical skybox is no longer visible). I'm unsure what the best way would be to remedy that here, I think I can dip into the d3d11 device but that requires some unsafe code I think and I'm not super familiar with the APIs.
@jdm Do you know if there's a way to conditionally adjust face culling for the rendered image? I tried dipping into the d3d11 device and adjusting rasterizer state, adding a pixel shader, but to no effect (I assume because most stuff is happening in surfman and modifying the device doesn't affect that).
I do not. The extent of my d3d experience is the code in this repository.
Finally figured out the culling issue, turns out all I needed to do was call gl.front_face(gl::CW) in GlClearer::fbo() to change the winding order and it now renders correctly (when paired with the angle swap in the projection matrix calculation). I'll still need to figure out a good way to determine whether or not the current OpenXR runtime supports fovMutable, but I at least have a path to a solution now.