/control-by-face

Control a robot by the shape of a face.

Primary LanguageC++MIT LicenseMIT

Control-By-Face

Control a robot by the shape of a face.

git clone https://github.com/sasekazu/control-by-face.git --recursive

Dependencies

  • dlib
  • cereal
  • OpenCV
  • nana
  • openFrameworks (for simulator)
  • xArm-CPLUS-SDK (for controller)

To install the dependencies using vcpkg, run this command.

vcpkg install dlib cereal opencv nana

Download shape_predictor_68_face_landmarks.dat from http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2 .

Apps

Collector

Collect face images from a webcam with labels. This exports collected data in out.cereal. You need to put shape_predictor_68_face_landmarks.dat in the same directory.

Trainer

Train a model to predict control signals from face images. This reads out.cereal and exports a trained model in decision_function.dat.

Predictor

Predict control signals from face images. This reads decision_function.dat and shape_predictor_68_face_landmarks.dat. You can check the predicted control signals on a console.

Simulator

Simulate a robot with the predicted control signals. A robot end-effector is modeled as a sphere. This requires openFrameworks. Include training_data_raw.h in a openFrameworks project. Replace ofApp.h and ofApp.cpp with these of this repository. This reads decision_function.dat and shape_predictor_68_face_landmarks.dat.

Controller

Control a robot (Ufactory Lite6) with the predicted control signals. This requires xArm-CPLUS-SDK. In an example lf xArm-CPLUS-SDK, such as 1009-cartesian_velocity_control.cc, add an include path to traning_data_raw.h and replace a source file with controller.cpp of this repository. This reads decision_function.dat and shape_predictor_68_face_landmarks.dat.