Include orientation angles for detected faces

[cdata]

The Android face detection API includes orientation angles for detected faces. Similarly, the Google Cloud Vision API includes roll, pan and tilt angles for detected faces. It would be useful to have these details for detected faces if they are made available by the underlying system API. In cases where the author intends to decorate detected faces with art work, these angles can help the author to make the decorations more adaptive to the scene.

[jeffreytgilbert]

That's cool and all, but the math to estimate the distance from camera to face is also incredibly involved to crunch in js. It can be done (see head-coupled perspective), but you're really getting into the weeds with that. Would love it if we also had a way to estimate the distance from subject the camera was somehow and return that. That would unlock some really cool web apps where the 3D or image could be reframed or rotated dynamically giving a more engaging experience.

I created an example app here using GetUserMedia API, Web Workers, zero copy transfer of image data, tensor flow, a face detection library wrapping Tensor Flow, and Three.js that attempted to do this. In the end, the hardware and software were not fast enough to do this adequately because the models the face detect lib passed to tensor flow ran too slowly on the GPU and the GPU bottleneck on an integrated intel graphics chip was not sufficient to handle both the UI updates and the detection of a face. The screen would skip even though all CPU processes were not overwhelmed and were running in the background because Tensor Flow used the maximum power of the GPU and the UI didn't have anything which allocated dedicated processing power for its own use vs general tasks authored by the web dev.

Sample code here: https://github.com/jeffreytgilbert/goto-booth

Bottom line is, if this could also be provided in a speedy easy to implement way, that would really eliminate a ton of code I had to write just to find out this wasn't possible to do with the current tools available.