opengl-tutorials/ogl

TestRayOBBIntersection issue

Opened this issue · 0 comments

I've incorporated the TestRayOBBIntersection function from misc05_picking_custom.cpp into my own program and it was working great when I was in a perspective projection matrix but whenever I was in an ortho projection it was failing. I've done a lot of debugging and I believe there is a coding error in that function for the else part of each axis where it labels it as a "rare case". For ortho projections the ray_direction vector is always (0, 0, -1) which causes the dot product against every axis to be zero (i.e. the variable f = 0.0) causing it to take the "rare case" path for every axis. The problem is the variable "e" is being negated in the rare case if statement which causes a hit to fail.

This code snippet I attached below hard codes all the inputs to the functions to demonstrate the error. This should result in a direct hit to the center of the object. I had to modify 2 things in the TestRayOBBIntersection function to get this to result in a hit. 1) Modify the rare cases to change all occurrences of "-e" to "e". 2) Add a Boolean input to the function to indicate you are doing a 2D hit detection so it will skip the Z axis.

I've got it working in my program now but wanted to provide this feedback to help others.

int screenWidth = 1200;
int screenHeight = 700;
glm::vec3 ray_origin;
glm::vec3 ray_direction;

ProjectionMatrix = glm::ortho(0.0f, (float)screenWidth, 0.0f, (float)screenHeight, 1.0f, 100.0f);
ViewMatrix = glm::mat4(1.0f);

int mousex = 200;
int mousey = 100;

ScreenPosToWorldRay(
	mousex, screenHeight - mousey,
	screenWidth, screenHeight,
	ViewMatrix,
	ProjectionMatrix,
	ray_origin,
	ray_direction
);

glm::vec3 aabb_min = glm::vec3(0.0f, 0.0f, 0.0f);
glm::vec3 aabb_max = glm::vec3(5.0f, 5.0f, 0.0f);
glm::mat4 ModelMatrix = glm::translate(glm::mat4(1.0f), glm::vec3(197.5f, 602.5f, 0.0f));

float distance = 1000000000.0f;
float intersection_distance;
bool hit = false;

if (TestRayOBBIntersection(
	ray_origin,
	ray_direction,
	aabb_min,
	aabb_max,
	ModelMatrix,
	intersection_distance,
	true)
	) {
	if (intersection_distance < distance) {
		distance = intersection_distance;
		hit = true;
	}
}