Research in object classification and novelty detection

This is a cutting-edge project in the field of machine learning, developed at the ISTC-CNR research centre. The system is designed to work on a robot and it allows to classify everyday objects which are in the environment using a convolutional neural network. Furthermore, I started a research activity to let the robot autonomously detect when the object is unknown to him, so that he can increase his knowledge capturing photos of it. To do this I developed a novelty detection system based on support vector machines.

Technologies involved:

Python
TensorFlow
Scikit-Learn
OpenCV
NumPy
Matplotlib

We got the real objects and the dataset here: http://www.ycbbenchmarks.com

Dataset sample

Prediction example

We got the real objects and the dataset here: http://www.ycbbenchmarks.com

Structure

Model built from scratch

cnn_ycb_v1.py: model definition from scratch and first version of training pipeline
cnn_ycb_v2.py: same model definition, but second version of training pipeline
predict_ycb_v1.py: simulation of the robot behaviour during prediction using the laptop webcam or smartphone camera
predict_ycb_for_results.py: script to get scientific results of the model on the test set

Model built using transfer learning and Support Vector Machine

retrain.py: model definition using transfer learning (CNN), SVM definition and training pipeline
prediction_mode: simulation of the robot behaviour during prediction using the laptop webcam or smartphone camera
prediction_mode_for_results.py: script to get scientific results of the model on the test set

Utilities

show_img_from_ip_webcam.py: script to use the smartphone camera connected in the network
crop_images.py: script to clean the dataset, which crops all images at the centre