C++ face alignment

[anguoyang]

is there a c++ version of face alignment? thank you

[nttstar]

Yes, check cpp-align dir

[anguoyang]

@nttstar Thank you for your response, I have check the dir of cpp-align, it seems that there is only a header file FacePreprocess.h, it implements similar transformation(without key facial points), but how could I used it face alignment?
I have searched for a long time, and implement the face alignment as follow:

cv::Mat align_face(const cv::Mat& src, const FaceLandmark& landmark, int width, int height) {

const int N = landmark.points_.size();
std::vector<cv::Point2f> detect_points;
if (N != 5) {
	detect_points = {
		landmark.points_[ReferenceID[0]],
		landmark.points_[ReferenceID[1]],
		landmark.points_[ReferenceID[2]],
		landmark.points_[ReferenceID[3]],
		landmark.points_[ReferenceID[4]],
	};
}
else {
	detect_points = landmark.points_;
}

const int ReferenceWidth = 112;
const int ReferenceHeight = 112;

int wex;
if (ReferenceHeight == 112)
	wex = 8;
std::vector<cv::Point2f> reference_points = {
	cv::Point2f{ 30.29459953f,  51.69630051f + wex },
	cv::Point2f{ 65.53179932f,  51.50139999f + wex },
	cv::Point2f{ 48.02519989f,  71.73660278f + wex },
	cv::Point2f{ 33.54930115f,  92.3655014f + wex },
	cv::Point2f{ 62.72990036f,  92.20410156f + wex }

};
for (auto& e : reference_points) {
	e.x += (width - ReferenceWidth) / 2.0f;
	e.y += (height - ReferenceHeight) / 2.0f;
}
cv::Mat tfm = get_similarity_transform(detect_points, reference_points);
cv::Mat aligned_face;
warpAffine(src, aligned_face, tfm, cv::Size(width, height));
return aligned_face;

}

it was taken out from sphereface, which the face image is 96x112, but insightface use 112x112, so I have to add a shift on it, it seems solve the problem, but the result of this alignment is not as good as your python code, so I have to use your cpp-align to keep the same accuracy, but I dont know how to use your ccp-align code because the facial landmark information is not used in your code.

[youthM]

I also don't know how to use ccp-align code, and do you know now? @anguoyang

[MirrorYuChen]

感谢感谢，解决了~

[BogdanovKirill]

@anguoyang

It seems that your solution is wrong, because you have to add 8 to X coordinates of reference points as mentioned here:
tpys/face-everthing#14 (comment)

[xiakj]

@anguoyang 你好。对于这个“cv::Mat align_face(const cv::Mat& src, const FaceLandmark& landmark, int width, int height)”我有几个问题：

1. src是原图像，还是抠出来的人脸图像？
2. FaceLandmark的数据结构是什么？
3. width和height分别代表什么？是检测到的人脸框大小，还是112*112？
4. sphereface的对齐代码链接在哪？
   谢谢！

[xiakj]

@anguoyang 你好。对于这个“cv::Mat align_face(const cv::Mat& src, const FaceLandmark& landmark, int width, int height)”我有几个问题：

1. src是原图像，还是抠出来的人脸图像？
2. FaceLandmark的数据结构是什么？
3. width和height分别代表什么？是检测到的人脸框大小，还是112*112？
4. sphereface的对齐代码链接在哪？
   谢谢！

@anguoyang 你好。对于这个“cv::Mat align_face(const cv::Mat& src, const FaceLandmark& landmark, int width, int height)”我有几个问题：

1. src是原图像，还是抠出来的人脸图像？
2. FaceLandmark的数据结构是什么？
3. width和height分别代表什么？是检测到的人脸框大小，还是112*112？
4. sphereface的对齐代码链接在哪？
   谢谢！

@MirrorYuChen 你解决了吗，能帮我回答以上问题吗？

[MirrorYuChen]

你可以参考我的ncnn-example项目里面，有个align模块的代码，觉得有用记得给star：
https://github.com/MirrorYuChen/ncnn_example

[MirrorYuChen]

我这里计算的时候，src是原图，如果想用截取出来的图，在计算关键点坐标时，需要减去人脸左上角点坐标~

[MirrorYuChen]

然后加不加8这个偏移量，取决于你人脸识112x112还是112x96上面已经将的很清楚了

[xiakj]

@MirrorYuChen 谢谢，我先试试，不懂的还要问您。

[xiakj]

@MirrorYuChen 解决了，谢谢老哥。方便加个微信吗13969756973

[MirrorYuChen]

已经添加好友~

[cannguyen275]

Hey guys, sorry for bothering you. But my issue might be related to this issue.

That is I sometimes received an up-side-down aligned face when I used the C++ version of the face align code. Especially, it's usually happened with high face's yaw value.

Debugging brings me to a conclusion that it's an error in calculating the rotation value in SimilarityTransform in C++, when its scikit-image version in Python works perfectly without any issues.

I've created an issue in #1559 but no one care, then I try to re-post this issue around to find if someone can help.

Thanks a lot for your attention.

[lucasjinreal]