照片合成一張
shawn846520 opened this issue · 1 comments
shawn846520 commented
您好，想詢問您知不知道如何使用halcon將兩張照片合成一張，不是普通的A圖在左B圖在右的那種。
實驗概念如下：
我是用雙目相機的概念，左右相機傾斜一個角度往下拍，所以左相機會拍到一張照片，右相機會拍到一張照片，我是想將照片重疊合成一張，並且保留其拍攝物的特徵，例如一個10cm*10cm的矩形，且矩形面凹凸不平，然後左右相機由上往下傾斜一個角度後進行拍攝那個矩形凹凸不平的面，然後將左右相機所拍攝到的照片進行合成為一張照片，以達到3D的效果。
不知道您是否知道相關的程式碼之類的
lovelyyoshino commented
这一种SFM方式，这里我提供一种简单的方法供你参考
<?xml version="1.0" encoding="UTF-8"?>
<hdevelop file_version="1.1" halcon_version="17.12">
<procedure name="main">
<interface/>
<body>
<c>* 此程序参照例程calibration_sheet_of_light_calplate.hdev</c>
<c>*分为两部分</c>
<c>*激光三角搭建1：各种位姿标定</c>
<c>*激光三角搭建2：点云合成</c>
<c></c>
<l>dev_update_off ()</l>
<l>dev_close_window ()</l>
<l>read_image (ProfileImage, './ProfileImage_L.bmp')</l>
<l>get_image_size (ProfileImage, Width, Height)</l>
<l>dev_open_window (0, 0, Width, Height, 'black', WindowHandle)</l>
<l>dev_set_draw ('margin')</l>
<l>dev_set_line_width (3)</l>
<l>dev_set_color ('lime green')</l>
<l>dev_set_lut ('default')</l>
<l>set_display_font (WindowHandle, 14, 'mono', 'true', 'false')</l>
<c></c>
<c>* 第一次标定数据</c>
<c>* 第一次标定的数据有内参和外参。其中内参是畸变，外参是相对位姿，该位姿是整个个系统的标准。</c>
<l>CameraParameters := ['area_scan_division',0.0122183,-1106.37,3.42648e-06,3.45e-06,1265.86,1033.39,2448,2048]</l>
<l>CameraPose := [-0.0128994,-0.0281569,0.234878,325.441,0.177233,179.927,0]</l>
<l>stop ()</l>
<c></c>
<c>*第二次标定</c>
<c>*第二次标定是光平面标定所需要的参数。PoseL是低平面放置一块标定版，标定版标定一次，再拍次光条的图片</c>
<l>PoseL :=[-0.0312937,-0.0427627,0.269893,321.431,358.718,175.582,0]</l>
<c></c>
<c>*第三次标定</c>
<c>*第三次标定是光平面标定所需要的参数。PoseH是高平面放置一块标定版，标定版标定一次，再拍次光条的图片</c>
<l>PoseH := [-0.0241666,-0.010946,0.200405,326.692,359.457,178.002,0]</l>
<c></c>
<c>*光平面标定</c>
<c></c>
<c>*很关键的标定，高度数据的精度将会是这次标定成功与否的判定标准</c>
<c>*具体是将光条（曝光值调低）在低平面拍次照，曝光值恢复正常标定次位姿。然后在平面上放置一个高一点的物件，再次重复标定高处</c>
<c></c>
<c>*经过二、三次的标定之后。分别将标定版数据和图片代入以下函数。</c>
<c>*既 标定数据 PoseL\PoseH 两次所得的光条图片（这里有个小坑，光条图需要处理下哦，具体看文件）</c>
<l>read_image (ProfileImage1, './ProfileImage_L.bmp')//低处的光条图</l>
<l>compute_3d_coordinates_of_light_line (ProfileImage1, 80, CameraParameters, [], PoseL, X19, Y19, Z19) //这里需要注意第一个Pose为空值</l>
<c></c>
<l>read_image (ProfileImage2, './ProfileImage_H.bmp')//高处的光条图</l>
<l>compute_3d_coordinates_of_light_line (ProfileImage2, 80, CameraParameters, PoseH, PoseL, X20, Y20, Z20) //这里是第一个高位，第二个低位</l>
<c></c>
<l>fit_3d_plane_xyz ([X19,X20], [Y19,Y20], [Z19,Z20], Ox, Oy, Oz, Nx, Ny, Nz, MeanResidual)</l>
<l>get_light_plane_pose (Ox, Oy, Oz, Nx, Ny, Nz, LightPlanePose) //输出光平面位姿</l>
<c></c>
<c>*移动位姿标定</c>
<c>*标定板起始位标定一次，然后移动一段距离后再标定一次（其中，移动距离根据 相机的Fps/时间差 = 距离）(存疑）</c>
<c>*这里我觉得我的理论很可能不对，而且我是用手推的产品，所以出来的点云长度不忍直视，希望有大佬指点</c>
<l>MovePoseStart := [-0.0236051,0.00302755,0.200835,324.381,357.449,355.023,0]</l>
<l>MovePoseEnd := [-0.0253978,-0.0205769,0.210492,325.67,357.288,354.81,0]</l>
<l>StepNumber := 40 //移动步长</l>
<l>CalTabThickness:=0.0032 //标定版厚度</l>
<l>pose_to_hom_mat3d (MovePoseStart, HomMat3DPos1ToCamera)</l>
<l>pose_to_hom_mat3d (MovePoseEnd, HomMat3DPos20ToCamera)</l>
<l>pose_to_hom_mat3d (CameraPose, HomMat3DWorldToCamera)</l>
<l>hom_mat3d_invert (HomMat3DWorldToCamera, HomMat3DCameraToWorld)</l>
<l>hom_mat3d_compose (HomMat3DCameraToWorld, HomMat3DPos1ToCamera, HomMat3DPos1ToWorld)</l>
<l>hom_mat3d_compose (HomMat3DCameraToWorld, HomMat3DPos20ToCamera, HomMat3DPos20ToWorld)</l>
<l>affine_trans_point_3d (HomMat3DPos1ToWorld, 0, 0, 0, StartX, StartY, StartZ)</l>
<l>affine_trans_point_3d (HomMat3DPos20ToWorld, 0, 0, 0, EndX, EndY, EndZ)</l>
<l>create_pose (EndX - StartX, EndY - StartY, EndZ - StartZ, 0, 0, 0, 'Rp+T', 'gba', 'point', MovementPoseNSteps)</l>
<l>MovementPose := MovementPoseNSteps / StepNumber //移动位姿</l>
<c></c>
<c></c>
<c></c>
<c></c>
<c></c>
<c></c>
<c></c>
<c></c>
</body>
<docu id="main">
<parameters/>
</docu>
</procedure>
<procedure name="compute_3d_coordinates_of_light_line">
<interface>
<io>
<par name="ProfileImage" base_type="iconic" dimension="0"/>
</io>
<ic>
<par name="MinGray" base_type="ctrl" dimension="0"/>
<par name="CameraParameters" base_type="ctrl" dimension="0"/>
<par name="LocalCameraPose" base_type="ctrl" dimension="0"/>
<par name="ReferenceCameraPose" base_type="ctrl" dimension="0"/>
</ic>
<oc>
<par name="X" base_type="ctrl" dimension="0"/>
<par name="Y" base_type="ctrl" dimension="0"/>
<par name="Z" base_type="ctrl" dimension="0"/>
</oc>
</interface>
<body>
<c>* This procedure computes the 3D-coordinates of the light</c>
<c>* line points</c>
<c></c>
<c>* The extraction of the coordinates of light line points in</c>
<c>* the image coordinate system is performed by using the</c>
<c>* operator measure_profile_sheet_of_light.</c>
<c>* The 3D-coordinates are computed by projecting the extracted</c>
<c>* coordinates to the plane z=0 of the local coordinate system.</c>
<c>* Then those coordinates are transformed to the reference pose.</c>
<c>* Note that if no local coordinate system is explicitly</c>
<c>* provided, the reference coordinate system is used instead.</c>
<c></c>
<c>* WARNING: if the profile is not oriented roughly horizontal</c>
<c>* empty tuples are returned for X, Y and Z.</c>
<c></c>
<c>* Initialize the output controls</c>
<l>X := []</l>
<l>Y := []</l>
<l>Z := []</l>
<c></c>
<c>* Check LocalCameraPose in order to determine if a transform</c>
<c>* has to be applied after the projection from the camera to</c>
<c>* the z=0 plane</c>
<l>if (LocalCameraPose == [])</l>
<l> DoTransform := 0</l>
<l> LocalCameraPose := ReferenceCameraPose</l>
<l>elseif (LocalCameraPose == ReferenceCameraPose)</l>
<l> DoTransform := 0</l>
<c></c>
<l>else</l>
<l> DoTransform := 1</l>
<l>endif</l>
<c></c>
<c>* Compute the pose for the projection of the</c>
<c>* local coordinate system to the plane z=0</c>
<l>pose_to_hom_mat3d (LocalCameraPose, HomMat3D_LocalToCam)</l>
<c></c>
<c>* Compute the homography which transform the 3D-coordinates</c>
<c>* of points from the local coordinate system to the reference</c>
<c>* coordinate system</c>
<l>if (DoTransform)</l>
<l> pose_to_hom_mat3d (ReferenceCameraPose, HomMat3D_ReferenceToCam)</l>
<l> hom_mat3d_invert (HomMat3D_ReferenceToCam, HomMat3D_CamToReference)</l>
<l> hom_mat3d_compose (HomMat3D_CamToReference, HomMat3D_LocalToCam, HomMat3D_LocalToReference)</l>
<l>endif</l>
<c></c>
<c>* Determine the profile region and test if the profile is</c>
<c>* oriented roughly horizontal</c>
<l>threshold (ProfileImage, Region, MinGray, 999999999)</l>
<l>orientation_region (Region, Phi)</l>
<l>if (cos(Phi) > cos(rad(135)) and cos(Phi) < cos(rad(45)))</l>
<c> * The detected profile is NOT oriented roughly horizontal,</c>
<c> * therefore return empty tuples X, Y and Z.</c>
<l> return ()</l>
<l>endif</l>
<l>dilation_circle (Region, RegionDilation, 5.5)</l>
<l>smallest_rectangle1 (RegionDilation, Row1, Column1, Row2, Column2)</l>
<l>gen_rectangle1 (ProfileRegion, Row1, Column1, Row2, Column2)</l>
<l>get_domain (ProfileImage, Domain)</l>
<l>intersection (ProfileRegion, Domain, ProfileRegion)</l>
<c></c>
<c>* Get the 2D-coordinates of the points on the light line</c>
<l>create_sheet_of_light_model (ProfileRegion, ['min_gray','num_profiles'], [MinGray,1], SheetOfLightHandle)</l>
<l>measure_profile_sheet_of_light (ProfileImage, SheetOfLightHandle, [])</l>
<l>get_sheet_of_light_result (Disparity, SheetOfLightHandle, 'disparity')</l>
<c></c>
<c>* Get the 3D-coordinates of the points on the light line</c>
<c>* in the coordinate system defined by ReferenceCameraPose</c>
<l>get_domain (Disparity, DisparityDomain)</l>
<l>get_region_points (DisparityDomain, Rows, Columns)</l>
<l>get_grayval (Disparity, Rows, Columns, Disparities)</l>
<c></c>
<c>* Project those points to the z=0 plane of the WCS</c>
<l>image_points_to_world_plane (CameraParameters, LocalCameraPose, Disparities, Columns + Column1, 1.0, X, Y)</l>
<l>tuple_gen_const (|Columns|, 0.0, Z)</l>
<c></c>
<c>* If necessary, transform the 3D-coordinates from the</c>
<c>* local coordinate system to the reference coordinate</c>
<c>* system</c>
<l>if (DoTransform)</l>
<l> affine_trans_point_3d (HomMat3D_LocalToReference, X, Y, Z, X, Y, Z)</l>
<l>endif</l>
<l>return ()</l>
</body>
<docu id="compute_3d_coordinates_of_light_line">
<parameters>
<parameter id="CameraParameters"/>
<parameter id="LocalCameraPose"/>
<parameter id="MinGray"/>
<parameter id="ProfileImage"/>
<parameter id="ReferenceCameraPose"/>
<parameter id="X"/>
<parameter id="Y"/>
<parameter id="Z"/>
</parameters>
</docu>
</procedure>
<procedure name="fit_3d_plane_xyz">
<interface>
<ic>
<par name="X" base_type="ctrl" dimension="0"/>
<par name="Y" base_type="ctrl" dimension="0"/>
<par name="Z" base_type="ctrl" dimension="0"/>
</ic>
<oc>
<par name="Ox" base_type="ctrl" dimension="0"/>
<par name="Oy" base_type="ctrl" dimension="0"/>
<par name="Oz" base_type="ctrl" dimension="0"/>
<par name="Nx" base_type="ctrl" dimension="0"/>
<par name="Ny" base_type="ctrl" dimension="0"/>
<par name="Nz" base_type="ctrl" dimension="0"/>
<par name="MeanResidual" base_type="ctrl" dimension="0"/>
</oc>
</interface>
<body>
<c>* This procedure fits a 3D-plane into a set of 3D-points</c>
<c></c>
<c>* The procedure returns the coordinates [Ox, Oy,Oz] of</c>
<c>* the centroid of the provided input coordinates, and</c>
<c>* the coordinates [Nx, Ny,Nz] of the normal vector to</c>
<c>* the fitted plane.</c>
<c></c>
<c>* WARNING: If the system of equations is under-determined</c>
<c>* (i.e. if it has too few input coordinates in X, Y, Z),</c>
<c>* it cannot be solved and the procedure returns empty tuples</c>
<c>* for X, Y, and Z</c>
<c></c>
<c>* Perform some initializations</c>
<l>Ox := []</l>
<l>Oy := []</l>
<l>Oz := []</l>
<l>Nx := []</l>
<l>Ny := []</l>
<l>Nz := []</l>
<l>MeanResidual := []</l>
<c></c>
<c>* Test the size of X, Y and Z, and return if necessary</c>
<l>Size := |X|</l>
<l>if (Size < 3 or Size != |Y| or Size != |Z|)</l>
<l> return ()</l>
<l>endif</l>
<c></c>
<c>* Compute the coordinates of the centroid point.</c>
<l>tuple_mean (X, Ox)</l>
<l>tuple_mean (Y, Oy)</l>
<l>tuple_mean (Z, Oz)</l>
<c></c>
<c>* Setup the equation system as a matrix M and compute</c>
<c>* its singular value decomposition. The singular vector</c>
<c>* of M corresponding to its smallest singular value provides</c>
<c>* the coordinates of the normal vector of the fitted plane.</c>
<l>create_matrix (3, |X|, [X - Ox,Y - Oy,Z - Oz], MatrixID_Mt)</l>
<l>transpose_matrix (MatrixID_Mt, MatrixID_M)</l>
<l>svd_matrix (MatrixID_M, 'reduced', 'right', MatrixID_U, MatrixID_S, MatrixID_V)</l>
<l>get_value_matrix (MatrixID_S, [0,1,2], [0,1,2], SingularvaluesOfM)</l>
<l>tuple_sort_index (SingularvaluesOfM, Indices)</l>
<c></c>
<c>* Test if more than one singular value of M is (nearly) equal</c>
<c>* to zero. This indicates that the provided 3d points are</c>
<c>* inappropriate to fit the plane (e.g. they are nearly</c>
<c>* collinear or reduce to a single point).</c>
<l>if (SingularvaluesOfM[Indices[0]] < 1e-9 and SingularvaluesOfM[Indices[1]] < 1e-9)</l>
<l> return ()</l>
<l>endif</l>
<c></c>
<c>* Get coordinates of the normal vector to the fitted plane</c>
<l>get_value_matrix (MatrixID_V, [0,1,2], [Indices[0],Indices[0],Indices[0]], N)</l>
<l>create_matrix (3, 1, N, MatrixID_N)</l>
<l>Nx := N[0]</l>
<l>Ny := N[1]</l>
<l>Nz := N[2]</l>
<c></c>
<c>* Compute the mean residual distance between the 3d points</c>
<c>* and the fitted plane, in order to guess the quality of</c>
<c>* the fitted plane:</c>
<l>mult_matrix (MatrixID_M, MatrixID_N, 'AB', MatrixID_MN)</l>
<l>get_full_matrix (MatrixID_MN, Distances)</l>
<l>Distances := abs(Distances)</l>
<l>MeanResidual := sum(Distances) / Size</l>
<c></c>
<l>return ()</l>
</body>
<docu id="fit_3d_plane_xyz">
<parameters>
<parameter id="MeanResidual"/>
<parameter id="Nx"/>
<parameter id="Ny"/>
<parameter id="Nz"/>
<parameter id="Ox"/>
<parameter id="Oy"/>
<parameter id="Oz"/>
<parameter id="X"/>
<parameter id="Y"/>
<parameter id="Z"/>
</parameters>
</docu>
</procedure>
<procedure name="get_light_plane_pose">
<interface>
<ic>
<par name="OriginX" base_type="ctrl" dimension="0"/>
<par name="OriginY" base_type="ctrl" dimension="0"/>
<par name="OriginZ" base_type="ctrl" dimension="0"/>
<par name="NormalVectorX" base_type="ctrl" dimension="0"/>
<par name="NormalVectorY" base_type="ctrl" dimension="0"/>
<par name="NormalVectorZ" base_type="ctrl" dimension="0"/>
</ic>
<oc>
<par name="LightPlanePose" base_type="ctrl" dimension="0"/>
</oc>
</interface>
<body>
<c>* This procedure determines a lightplane pose,</c>
<c>* e.g. a pose whose plane defined by z=0 coincides</c>
<c>* with the physical light plane.</c>
<c></c>
<c>* Test that the vector passed at input is not null</c>
<l>LightPlanePose := []</l>
<l>Norm := NormalVectorX * NormalVectorX + NormalVectorY * NormalVectorY + NormalVectorZ * NormalVectorZ</l>
<l>if (abs(Norm) < 1e-8)</l>
<l> return ()</l>
<l>endif</l>
<c>* In order to compute a light-plane pose, we determine</c>
<c>* two rotations which align the unit vector of the z-axis</c>
<c>* to the normal vector of the light plane, when applied</c>
<c>* successively. For example, we can compute the angles</c>
<c>* Alpha (rotation around the x-axis) and Beta (subsequent</c>
<c>* rotation around the y-axis) in this successive order.</c>
<c>* (The rotation around the z-axis is arbitrarily set</c>
<c>* to zero).</c>
<c></c>
<c>* Determine the value of the angle Alpha and rotate the</c>
<c>* normal vector to the plane y=0. This provides the</c>
<c>* vector N1.</c>
<l>hom_mat3d_identity (HomMat3DIdentity)</l>
<l>tuple_atan2 (NormalVectorY, NormalVectorZ, Alpha)</l>
<l>hom_mat3d_rotate (HomMat3DIdentity, Alpha, 'x', 0, 0, 0, HomMat3DRotateAlpha)</l>
<l>affine_trans_point_3d (HomMat3DRotateAlpha, NormalVectorX, NormalVectorY, NormalVectorZ, N1x, N1y, N1z)</l>
<c></c>
<c>* Determine the value of the angle Beta by using the</c>
<c>* coordinates of N1. Note that the rotation around the</c>
<c>* y-axis with angle Beta is carried out in the counter</c>
<c>* trigonometric direction, therefore we apply -Beta.</c>
<l>tuple_atan2 (N1x, N1z, Beta)</l>
<l>hom_mat3d_rotate (HomMat3DIdentity, -Beta, 'y', 0, 0, 0, HomMat3DRotateBeta)</l>
<c></c>
<c>* Create the LightPlanePose</c>
<l>hom_mat3d_translate (HomMat3DIdentity, -OriginX, -OriginY, -OriginZ, HomMat3DTranslate)</l>
<l>hom_mat3d_compose (HomMat3DRotateAlpha, HomMat3DTranslate, HomMat3DTmp)</l>
<l>hom_mat3d_compose (HomMat3DRotateBeta, HomMat3DTmp, HomMat3DWorldToLightplane)</l>
<l>hom_mat3d_to_pose (HomMat3DWorldToLightplane, LightPlanePose)</l>
<l>return ()</l>
<c></c>
</body>
<docu id="get_light_plane_pose">
<parameters>
<parameter id="LightPlanePose"/>
<parameter id="NormalVectorX"/>
<parameter id="NormalVectorY"/>
<parameter id="NormalVectorZ"/>
<parameter id="OriginX"/>
<parameter id="OriginY"/>
<parameter id="OriginZ"/>
</parameters>
</docu>
</procedure>
</hdevelop>