/jetson_faceTrack_pytorch

Comprehensive multifunctional AI project

Primary LanguagePythonMIT LicenseMIT

hardware1
hardware2

Overview

Lightweight face detection and recognition library, which can be recognized in real time on jetson/pc devices. The algorithm uses mature MTCNN/Ultra-Light-Fast-Generic-Face-Detector-1MB face detection+InsightFace(Arcface+cosface head Face recognition with +MobileFacenet, resnet backbone), the recognition range is limited to a closed set range. This project also adds Servo tracking face position function,opencv-cuda acceleration + voice recognition broadcast + Bluetooth control. Among them, "matlabPro/" is a functional module independent of other python/c++, equivalent to face detection + tracking + HC08 Bluetooth module transmit signal + voice broadcast and other functions.

轻量级的人脸检测和识别库，可在jetson/pc设备上实时识别，算法采用成熟的MTCNN/Ultra-Light-Fast-Generic-Face-Detector-1MB人脸检测+InsightFace(Arcface+cosface head和+MobileFacenet、resnet backbone)进行人脸识别，识别范围限定为闭集范围内。 本项目还增加舵机跟踪人脸位置功能，opencv-cuda加速+语音识别播报+蓝牙控制。其中，"matlabPro/"是独立与其他python/c++的功能模块，等价的人脸检测+追踪+HC08蓝牙模块发射信号+语音播报等功能。

software main requirements

for pc train and inference

for jetson inference

  • numpy
  • pre-build opencv-python or opencv with cuda support
  • pytorch
  • tensorRT
  • pyserial

hardware main requirements

datasets

train datasets （pc上完成）

参考其中项目InsightFace_Pytorch数据集,数据量太大，这里只给出了101人的人脸图像.
百度网盘，提取码：nfy3
google drive
训练数据集目录结构如下，命名无要求。

yourTrainDataFolder/
        ---> train_name1/
            ---> id1_1.jpg
			---> ...
        ---> train_name2/
            ---> id2_1.jpg
			---> ...
        ---> train_name3/
            ---> id3_1.jpg
           ---> id3_2.jpg
		   ---> ...

test datasets (pc或者jetson上完成)

在当前项目目录下的文件夹 ./dataSets/facebank/准备自己需要识别的人脸图像,目录结构也类似上面排布，命名无要求，文件夹名与姓名同名最佳。

./dataSets/facebank/
        ---> zhangsan/
            ---> id1_1.jpg
			---> ...
        ---> lisi/
            ---> id2_1.jpg
			---> ...
        ---> wangwu/
            ---> id3_1.jpg
           ---> id3_2.jpg
		   ---> ...
		---> ...
		...
		facebank.pth
		names.npy

提供已知的人脸图像还可以使用prepare_getFaces.py摄像头在线采集，图像会自动存储到上述路径。若有多幅图像，则特征取平均值。

How to recognize face (use camera to inference,for pc and jetson)

  1. Prepare the data set and model: The data set is the above test datasets and needs to recognize the known face images, and the model is the appropriate one under the folder ./models/;
  2. Prepare the face database: python prepare_faceDatabase.py stores the features and names of the face database, and will automatically generate facebank.pth and names.npy in the above directory structure;
  3. Online camera recognition: python camera_faceRec.py

1、准备数据集和模型:数据集为上述test datasets自己需要识别已知的人脸图像，模型为./models/文件夹下选择适合的;
2、准备人脸数据库：python prepare_faceDatabase.py进行人脸数据库特征和姓名存储，会自动生成上述目录结构中的facebank.pth和names.npy；
3、摄像头在线识别：python camera_faceRec.py

How to train your model (for pc)

准备好上述的train datasets，设置好一些arguments，执行train.py即可。

models

包含人脸检测和识别模型（IR-SE50，Mobilefacenet），caffemodel,onnx,pth类型.
百度网盘，提取码f3v6
google drive

performance

model LFW(%) CFP-FF(%) CFP-FP(%) AgeDB-30(%) calfw(%) cplfw(%) vgg2_fp(%)
ir_se50 0.9952 0.9962 0.9504 0.9622 0.9557 0.9107 0.9386
mobilenet-face 0.9918 0.9891 0.8986 0.9347 0.9402 0.866 0.9100

reference statistics

jetson-opencv-cuda编译

jetson-opencv-cuda编译

References