Cannot reproduce the results on IJBB and IJBC

[johnnysclai]

I used the pretrained IR101 model and the IJB evaluation code from insightface. Following are the results I got:

IJB-B TAR@FAR:
1e-6 -> 41.76%
1e-5 -> 69.81%
1e-4 -> 87.14%
1e-3 -> 93.27%

IJB-C TAR@FAR:
1e-6 -> 62.64%
1e-5 -> 75.46%
1e-4 -> 87.53%
1e-3 -> 94.01%

There is no problem if I use the pretrained models from face.evoLVe.PyTorch. Would you please share the evaluation code on IJB-B and IJB-C?

[johnnysclai]

I found the problem is due the face alignment. Once I change to the same face alignment code as inisightface, I can reproduce the results.

from skimage import transform as trans
def alignment(img, landmark):
	image_size = (112, 112)
	src = np.array([
		[30.2946, 51.6963],
		[65.5318, 51.5014],
		[48.0252, 71.7366],
		[33.5493, 92.3655],
		[62.7299, 92.2041]], dtype=np.float32)
	src[:, 0] += 8.0
	assert landmark.shape[0] == 68 or landmark.shape[0] == 5
	assert landmark.shape[1] == 2
	if landmark.shape[0] == 68:
		landmark5 = np.zeros((5, 2), dtype=np.float32)
		landmark5[0] = (landmark[36] + landmark[39]) / 2
		landmark5[1] = (landmark[42] + landmark[45]) / 2
		landmark5[2] = landmark[30]
		landmark5[3] = landmark[48]
		landmark5[4] = landmark[54]
	else:
		landmark5 = landmark
	tform = trans.SimilarityTransform()
	tform.estimate(landmark5, src)
	M = tform.params[0:2, :]
	img = cv2.warpAffine(img, M, (image_size[1], image_size[0]), borderValue=0.0)
	return img

[wangs311]

hi, this is my first time doing an experiment about face recognition. Could you share the IJB evaluation code which you used before? It will be very helpful. Thanks a lot!

[CloudWalking0]

I found the problem is due the face alignment. Once I change to the same face alignment code as inisightface, I can reproduce the results.

from skimage import transform as trans
def alignment(img, landmark):
	image_size = (112, 112)
	src = np.array([
		[30.2946, 51.6963],
		[65.5318, 51.5014],
		[48.0252, 71.7366],
		[33.5493, 92.3655],
		[62.7299, 92.2041]], dtype=np.float32)
	src[:, 0] += 8.0
	assert landmark.shape[0] == 68 or landmark.shape[0] == 5
	assert landmark.shape[1] == 2
	if landmark.shape[0] == 68:
		landmark5 = np.zeros((5, 2), dtype=np.float32)
		landmark5[0] = (landmark[36] + landmark[39]) / 2
		landmark5[1] = (landmark[42] + landmark[45]) / 2
		landmark5[2] = landmark[30]
		landmark5[3] = landmark[48]
		landmark5[4] = landmark[54]
	else:
		landmark5 = landmark
	tform = trans.SimilarityTransform()
	tform.estimate(landmark5, src)
	M = tform.params[0:2, :]
	img = cv2.warpAffine(img, M, (image_size[1], image_size[0]), borderValue=0.0)
	return img

Hello, I was wondering how to test our pytorch model on IJB datasets. The code in insightface seems to use mxnet model to evaluate. Thank you very much!

[johnnysclai]

@wangs311 @CloudWalking0 I download the notebook and data from here: https://github.com/deepinsight/insightface/tree/8d971de36370320c286f8bf92829f2995234b386/recognition/_evaluation_/ijb , and then replace the mxnet model to a pytorch model

[CloudWalking0]

@wangs311 @CloudWalking0 I download the notebook and data from here: https://github.com/deepinsight/insightface/tree/8d971de36370320c286f8bf92829f2995234b386/recognition/_evaluation_/ijb , and then replace the mxnet model to a pytorch model

Thanks for your reply!