FaceRTC is a real-time remote facial recognition service implemented using WebRTC to effectively handle communication between a web browser client and the facial recognition backend server.
The idea is to make facial recognition accessible to any devices with connection to internet. (Behaves like a serverless application, doesn't require any installation on the client to perform facial recognition)
WebRTC is a free, open-source project that provides web browsers and mobile applications with real-time communication via simple application programming interfaces.
- Utilize WebRTC to establish a p2p connection between client web browser and server.
- Client sends stream of camera images to the backend service. Those images are later asynchronously processed.
- Detection&Recognition results are then sent back to the webbrowser for presentation accordingly.
-
Architect a peer to peer connection for client<->server -
Webcam video streaming from client -> server -
Performs Face Detection on the server then send the image result with bounded face region back to the client -
Performs Facial Recognition on detected faces -
Optimization:
-
Implement trackers for faces on frame to improve Facial Recognition Acurracy (recognize a face through series of frames, ignore motion blur, etc.):
Setup a working framework but not yet implemented a Tracking algorithm
-
Implement a more sophisticated
find_matchto find the best matching face_label for a given face feature (embeddings) in dataset.- Kd-tree search ( currently simple linear search )
-
Perform image processing at the frontend (Use canvas to draw an UI on top of
<video>). This would significantly reduce bandwith load, because we currently are sending the entire processed frame back to the browser. [Should be easy but I'm not very good at html :D].
-
The server below is hosted on my macbook (Performance is not great but acceptable).
The webclient could be ran on a seperate machine, no computation is heavy computation is done via the web client. Demo on Phone Browser: https://www.youtube.com/watch?v=oWQg18o9fic
Status: In-progress