This project provides a high level wrapper for detecting faces in photos, generating facenet encodings and clustering faces belonging to recognized individuals, using tensorflow and keras.
- This project can be considered as Part I of a two-part series for Face Clustering on Android.
- Pre-trained models from facenet are converted to keras models and wrapper scripts are provided to cluster all faces in a collection of photos.
- tflite models are created using the keras models (This seemed like the most reliable way to port facenet to android).
- Also contains scripts for creating interactive visualizations of generated clusters.
tensorflow
keras
scikit-learn
cv2
imageio
dlib (for chinese whispers)
google_drive_downloader (for downloading models)
MulticoreTSNE (for optional speed-up)
1. Obtain pre-trained models.
$ python models/prepare_models.py -e 512
parameters:
-e/--emb_dim | size of the embedding generated by the model | one of [128, 512]
-lite/--tflite | create tflite model | one of [0, 1]
To possibly speed-up the process, models can be directly downloaded from 128, 512 and placed in the models folder, followed by script execution as above.
2. Detect faces and generate encodings for all photos in a folder.
$ python get_encodings.py -f ~/folder
paramters:
-f/--folder | folder containing photos to be processed
-e/--emb_dim | use the 128 or 512 dim model to generate encodings | one of [128, 512]
3. Cluster the encodings
$python clustering.py -f ~/folder -c kmeans -k 7 -v 1 -s 1
$python clustering.py -f ~/folder -c chinese_whispers -cw -v 1 -s 1
parameters:
-f/--folder | folder containing photos to be processed
-c/--cluster_method | method to be used for clustering | one of [kmeans, chinese_whispers, dbscan]
-k/--kmeans_k | no. of clusters for kmeans clustering
-eps/--dbscan_eps | epsilon value for dbscan clustering
-ms/--dbscan_min_samples | minimum sample count for a cluster in dbScan clustering
-cw/--chinese_whispers_eps | epsilon value or chinese whispers clustering
-v/--visualize | generate interactive vizualization for clusters | one of [0, 1]
-s/--save | save clusters as folders | one of [0, 1]
- Chinese whispers works best for large collections with large no. of individuals and is the overall winner.
- Kmeans works best for small datasets with equal photo count per class
- Kmeans with k = 2 can be used to identify an individual with relatively high no. of photos. For personal photo collections, this would mean identifying your own photos vs rest. For this specific usage, kmeans might beat chinese whispers.
- For Kmeans, set k to be 1 + expected no. of classes, to separarate out outliers.
- Parameters for dbscan and chinese whispers may require a lot of trial and error before decent results are obtained. I found it tricky to identify parameters that generalize well across different datasets. clustering.py prints out the no. of identified persons and the params can be tweaked till a satisfactory count is obtained.
- Will add scripts to identify best paramters for any given dataset.
- Will add Rank Order Clustering which is known to perform well for face datasets.