Doubt about cam1_points initialization

Hi!
I have been seeing the code and got a doubt about this if. As far as I understand, trying to initialize the features of the right camera in that way would only make sense if the 3D points belonged to the same plane (in space) which should not be the case in general. Is it just an approximated guess or I am missing something?
Thanks in advance.

Not sure about the "the same plane" way of thinking. Probably it's easier to think in the following way.

What's in the condition is a way to general initial guesses for matches (which will be later refined through tracking).
The guess is generated by simply projecting the pixel coordinates from the left frame to the right frame.
If the rotation between left and right cameras is identity, and the calibrations parameters of both cameras are the same, cam1_points would be just a copy cam0_points.
It may not be the best way to generate initial guess, but should be sufficient most times.

msckf_vio/src/image_processor.cpp

Lines 616 to 626 in e3a39a9

    
           if(cam1_points.size() == 0) { 
        
             // Initialize cam1_points by projecting cam0_points to cam1 using the 
        
             // rotation from stereo extrinsics 
        
             const cv::Matx33d R_cam0_cam1 = R_cam1_imu.t() * R_cam0_imu; 
        
             vector<cv::Point2f> cam0_points_undistorted; 
        
             undistortPoints(cam0_points, cam0_intrinsics, cam0_distortion_model, 
        
                             cam0_distortion_coeffs, cam0_points_undistorted, 
        
                             R_cam0_cam1); 
        
             cam1_points = distortPoints(cam0_points_undistorted, cam1_intrinsics, 
        
                                         cam1_distortion_model, cam1_distortion_coeffs); 
        
           }

Thank you for the answer. I understand. When I asked I was thinking about a case like this:

For P, it would not be a good initial guess, right?

Anyway, the condition is not related to belonging to the same plane in space or not, actually. I was wrong about that. It is a matter of stereo disparity I guess.

I see. In that sense, the above initialization assumes all 3-d points are infinitely far away from the image plane.

	if(cam1_points.size() == 0) {
	// Initialize cam1_points by projecting cam0_points to cam1 using the
	// rotation from stereo extrinsics
	const cv::Matx33d R_cam0_cam1 = R_cam1_imu.t() * R_cam0_imu;
	vector<cv::Point2f> cam0_points_undistorted;
	undistortPoints(cam0_points, cam0_intrinsics, cam0_distortion_model,
	cam0_distortion_coeffs, cam0_points_undistorted,
	R_cam0_cam1);
	cam1_points = distortPoints(cam0_points_undistorted, cam1_intrinsics,
	cam1_distortion_model, cam1_distortion_coeffs);
	}