Presence
Presence is a kinetic sculpture that detects a viewer's gaze in real-time using a convolutional neural-network and moves in response to the gaze
It uses a webcam and the pre-trained model from Eye Tracking for Everyone to detect where users are gazing, and moves in the direction of the gaze.
Key files/documentation
Full documentation coming soon
- http://www.danioved.com/blog/posts/presence/ - blog posts which document the process
- The jupyter notebook that walks through how features are extracted in opencv and fed through the neural network to obtain a gaze prediction.
- features.py - the code to extract features from input images using opencv
- gaze.py - the code to feed the features through the pre-trained neural network and get a gaze prediction
- processing_sketches/GazeFromStream3d - processing sketch to render a simulation of the columns in 3d, and send servo positions over serial to the kinetic sculpture.
Requirements
For gaze detection:
- python
- caffe
- opencv
To render simulations of the servo positions:
- processing