Using this app, users can choose to add a photo from their computer (we need users to take a photo of their clothes and store it on the computer) to our app. In the process of adding pictures, this picture will be automatically recognized by our pre-trained model at one time, by type (such as T-shirt), gender (such as women), color (such as navy blue), season (such as summer), usage (such as formal, casual, sport, etc.), and path of the photo. These six types of information will be stored in the app and displayed for the user to refer to. Considering that our model is sometimes inaccurate, we also provide users with editing and deleting buttons to facilitate user operations. Therefore, the user can edit the information we misidentified or delete a piece of clothing.
We have a "Generate Today's Outfit Recommendations" button, providing users with recommendations. The recommended result is composed of three pictures, namely top, bottom, and shoes. Of course, these three pictures are the user's clothes.
Below is a functional diagram of using our app:
Behind such an app, we have four neural network models and a recommendation system.
For our neural network models: The first model (with accuracy over 99%) is used to identify whether the clothes are tops, bottoms, or shoes. After this, the photo will be directed to one of the other three models based on the results of the first model (please see the picture below). These three similar but not exactly the same models (with overall accuracy around 80% ) will identify type(65), gender(5), color(46), season(4), and usage(8).
Below is a flow chart of our recognition process:
For the recommendation process: The program randomly selects a top, and we look for the same gender, the same season, and the same usage in the bottoms and shoes stored in the app. After this, we recommend outfits based on the color. Our recommended method for color is based on such a color wheel. We summarized the 46 colors in the data set into 12 colors in the figure, plus three colors of black, white, gray, and multi-color, recommending according to the angle between the colors.
First of all, although our data is huge and the accuracy of the test set is very high, we found that our program is not suitable for all the pictures we randomly searched from Google. Therefore, we think our data is with bias, which is discussed in "A special explanation for data cleaning and analysis(bias).ipynb". In addition, it seems partial to be due to changes in people's thinking about fashion, or perhaps because the dataset is relatively old.
To start the app:
a) First download ui_module.py, recognition_module.py, and models from this repo to the same folder.
Install all the packages:
pip install -r requirements.txt
b) Next, type the following code in any Python environment:
from ui_module import*
run_ui()
c) Then you will see the app!
.
├── A special explanation for data cleaning and analysis(bias).ipynb # A note that analyzes bias of the data set used in this project
├── LICENSE # The license
├── README.md # The readme file
├── models # This folder contains everything related to the models: data, models themselves, training code
│ ├── data # Data for training models from Kaggle
│ ├── models # A google drive folder which contains our trained models
│ │ ├── model_sub # A model that distinguishes tops, bottoms, and shoes
│ │ ├── model_top # A model that recognizes the type, color, gender, season, and usage of tops
│ │ ├── model_bottom # A model that recognizes the type, color, gender, season, and usage of bottoms
│ │ └── model_shoes # A model that recognizes the type, color, gender, season, and usage of shoes
│ ├── train_module.py # A module which contains functions we used to train the models
│ └── training.py # Steps that we train the models
├── pictures # Pictures used in this repo
│ ├── 51109bb074d95c059f716e48786568f.jpg
│ ├── IMG_0159.jpg
│ ├── top_question.png
│ └── tutorial.png
├── proposal.md # Our original proposal
├── py # All you need to run the app
│ ├── recognition_module.py # A module that contains functions and classes to generate the GUI
│ └── ui_module.py # A module that contains the function to run the app
└── requirements.txt # The packages