FreshEye

Fresh Eye: Eat Everything You Buy

Team Members: Rahul Shah, Trang Tran, Kira Sidhu, Carolyn Qi

Background

According to the USDA, 31% or 133 billion pound of food gets goes uneated every year. Fresh Eye is focused on combatting this food waste problem by using computer vision processing and Machine Learning Fine Tuning to provide addiitonal information to users about their groceries. Fresh Eyes will identify when a specific piece of fruit will go bad, acting as a fresh fruit expiration date for users to follow. When the user puts the item in the fridge, Fresh Eye will use OpenCV to identify the specific fruit or vegetable. It will compare it to other photos of the same fruit or vegetable to understand how ripe it is. Then, it will calculate how much longer before the item goes bad.

The Global Smart Fridge market has been growing at a 16% compound annual growth rate, and expected to hit $4.56B by 2025. By participating in this growth, Fresh Eye's functionality will become part of more homes and impact more households. Our functionality also rides on the backs of existing smart fridge functionality, which can identify the produce type with computer vision processing.

Purpose of Project

Fresh Eyes will combat the global food waste problem by using computer vision processing and Machine Learning Fine Tuning model to provide additional information to customers about their fresh produce and fruit. Provided extra information, customers will be better informed of the status of the produce in their fridge and will be less likely to let food rot.

Fresh eyes uses OpenCV, a computer vision library, to take a photo of the produce and identify the type of fruit that it is. In the future, we would like to integrate this with existing smart fridge technology, which includes cameras and food recognition capabilities. Then, we send the same photo, split up into the separate fruits, to ChatGPT4. Using OpenAI ChatGPT4's prompt engineering capabilities and our machine learning fine tuning algorithm, Fresh Eyes then identifies the estimated expiration date based on the current freshness of the produce. Our machine learning fine tuning algorithm is run on hundreds of photos of fruit, tagged with levels of freshness. We have also created a temporary User Experience Interface to display all the food that exists in the fridge. In the future, we would like to integrate this into the existing apps built for smart fridge technology.

Tech Stack: ML, AI, and IoTs

Python, Flask, Reactjs
Tensorflow
Google CoLab
OpenCV
YOLOv8
GPT-4 API

Additional Features:

Training our machine learning fine tuning algorithm with additional fruits and vegetables: we would like to upload more tagged photos of more fresh produce so that it can identify the expiration date for a larger variety of fresh produce.
Label reading capabilities for packaged foods: this will allow users to also be able to add packaged foods to their trackers, leading to even less food waste.
Notification features: notify the user when something is about to go bad, so that they can promptly use it.
Advanced food placement recommendations: provide recommendations of how to organize the fridge so that fruit and produce will last longer.
Syncing multiple cameras: intake information from multiple cameras to gather additional information about what is being stored in the fridge.
Integrate with existing smart fridge technology: smart fridge technology already has the camera and produce/fruit identification technology. By building upon this, we will be able to reach more customers and make a bigger impact.
End of year produce summary: inform customers of their produce consumption and waste so they can further improve their habits. It will also lead to more word of mouth customer acquisition.
Cellphone camera imaging: allow for our functionality to be used on the go. This will also be used for food that is stored outside of the fridge.

Getting Started: