With people paying increasing attention to tracking meal items and nutrition contents in daily life for weight loss or medical purposes, self-managed food monitoring has become a vital application for computer vision. The tradiation dietary assessment have multiple challenges to be accurately assessed. Most of the current methods for dietary assessment must rely on memory to recall foods eaten. Even we remember the food we ate, but we didn't know what it is and their ingredients.Furthermore, nutrition information is vital for fitness, where we need to control the number of calories, carbohydrates, fats, proteins. In order to improve accuracy and convenience in diet assessment, The project aim to build a system that can instantly detect food through the phone camera and return nutritional information of this food.
Build the mobile app that can help user can easily record their meals and keep track consumption of nutrition through their mobile phone.
- Detect food from a picture Build the mobile app that can take a picture and detect food in this picture.
- Detect 1 dish in a picture
- Detect multiple dishes in 1 picture
- Display a nutrition of food in a picture
- Record the meal nutrition
- Show ingredients and recipe of dish
- Daily nutriotion calculation
The project is mainly referenced from the two articles: YOLO for Real-Time Food Detection and A Real-time Food Detection Mobile Application by Deep Convolutional Neural Networks
- Food detection
Before we start to classify food in the picture, we have to understand how many objects appear on the image. The main idea here is to divide each input image in to (S,S) grid and predict N bouding boxes and confidence in each cell. The confidence reflects the accuracy of the bounding box and whether the bounding box actually contain an object. We will separate the input image to multipe input by the bounding box of YOLO.
Using the Open Images Dataset V6 + Extensions to train model that could clasify food in a picture.
- Food tracking
After we have the input images, we can use the CNN model to detect type of food. I believe we can train use some pretrain model such as mobilenet or EfficientNet. After detecting the food, we should use YOLO to display name of food.
-
Display nutrition of Food Using api of Nutritionix
-
Mobile application Our final application will detect the food through image which is input by user. Mobile app will send image to server and receive result of food from server. After that, app will display food detection and nutrition for user.
A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images Other food dataset
https://www.nutritionix.com/ https://github.com/ivanDonadello/Food-Categories-Classification https://github.com/SumithBaddam/NeuralCook http://pic2recipe.csail.mit.edu/ https://arxiv.org/ftp/arxiv/papers/1606/1606.05675.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6368251/