What is hawkeye?
Hawkeye is an image search engine developed for the Multimedia Information Retrieval and Computer Vision course of the MSc Artificial Intelligence and Data Engineering at Università di Pisa.
It is a search engine based on the query-by-example paradigm: a user submit an image, the search engine returns the most similar ones belonging to the image collection. It uses as index a Vantage Point Tree and the MobileNetV2 for feature extraction. We extract two times the features: first, using the un-modified MobileNetV2, then with a fine-tuned version.
The fine-tuned model is in the \model directory. Each file contains a description of the steps performed.
Project files:
- Vantage Point Tree
- Model fine-tuning
- Feature Extraction
- Search Engine
- Performance Evaluation
- Web Interface
Query pre-processing pipeline issue
During search engine developing, we found that using the following preprocessing pipeline gives us strange results:
query = tf.keras.preprocessing.image.load_img(path_to_query, target_size=(224,224), interpolation='bilinear')
query = tf.keras.preprocessing.image.img_to_array(query)
query= tf.keras.applications.mobilenet_v2.preprocess_input(query)
query = np.array([query])
A careful investigation let us discover that loading the query using this pipeline gives us different results in terms of pixels, and so features, compared to what we obtain using the image_dataset_from_directory function. We decided to use the same pipeline used in the image_dataset_from_directory to avoid such discrepancy and have good results. Moreover, we investigate the performance of the two functions the code is available in this notebook: Pipelines performance comparison
Credits
@thorongil05, @matildao-pane, @ragnar1002, @nooshinnr, @seraogianluca