Fashion-MNIST

A dataset of Zalando's article images consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. Fashion-MNIST is intended to serve as a direct drop-in replacement of the original MNIST dataset for benchmarking machine learning algorithms.

Here is an example how the data looks like (each class takes three-rows):

Why?

The original MNIST dataset contains a lot of handwritten digits. People from AI/ML/Data Science community love this dataset and use it as a benchmark to validate their algorithms. In fact, MNIST is often the first dataset they would try on. "If it doesn't work on MNIST, it won't work at all", they said. "Well, if it does work on MNIST, it may still fail on others."

Fashion-MNIST is intended to serve as a direct drop-in replacement for the original MNIST dataset to benchmark machine learning algorithms, as it shares the same image size and the structure of training and testing splits.

To Serious Machine Learning Researchers

Seriously, we are talking about replacing MNIST. Here are some good reasons:

MNIST is too easy. Check out our side-by-side benchmark and "Most pairs of MNIST digits can be distinguished pretty well by just one pixel"
MNIST is overused. Check out "Ian Goodfellow wants people to move away from mnist"
MNIST can not represent modern CV tasks. Check out "Ideas on MNIST do not transfer to real CV"

Get the Data

You can use direct links to download the the dataset. The data is stored in the same format as the original MNIST data.

Name	Content	Examples	Size	Link
`train-images-idx3-ubyte.gz`	training set images	60,000	26 MBytes	Download
`train-labels-idx1-ubyte.gz`	training set labels	60,000	29 KBytes	Download
`t10k-images-idx3-ubyte.gz`	test set images	10,000	4.2 MBytes	Download
`t10k-labels-idx1-ubyte.gz`	test set labels	10,000	5.0 KBytes	Download

Or you can clone this repository, the dataset is under data/fashion. This repo contains some scripts for benchmark and visualization.

git clone git@github.com:zalandoresearch/fashion-mnist.git

Labels

Each training and test example is assigned to one of the following labels:

Label	Description
0	T-shirt/top
1	Trouser
2	Pullover
3	Dress
4	Coat
5	Sandal
6	Shirt
7	Sneaker
8	Bag
9	Ankle boot

Usage

Loading data with Python (`numpy` is required)

use utils/mnist_reader in this repo:

import mnist_reader
X_train, y_train = mnist_reader.load_mnist('data/fashion', kind='train')
X_test, y_test = mnist_reader.load_mnist('data/fashion', kind='t10k')

Loading data with Tensorflow

from tensorflow.examples.tutorials.mnist import input_data
data = input_data.read_data_sets('data/fashion')

data.train.next_batch(100)

Loading data with other languages

As one of the most popular dataset in the Machine Learning community, people have implemented MNIST loader in many languages. They can be used to load Fashion-MNIST dataset as well (may require decompressing first). Note that they are not tested by us.

Benchmark

We build an automatic benchmarking system based on scikit-learn, covering 125 classifiers with different parameters. Results can be found here.

You can reproduce the results by running benchmark/runner.py. A recommend way is to build and deploy this docker container.

You are welcome to submit your benchmark. Please create a new issue, your results will be listed here. Check out the Contributing section for details. Before submitting a benchmark, please make sure it is not listed in this list.

Classifier	Preprocessing	Test accuracy	Submitter	Code
2 Conv Layers with max pooling (Keras)	None	0.876	Kashif Rasul	zalando_mnist_cnn
2 Conv Layers with max pooling (Tensorflow)	None	0.916	Tensorflow's doc	convnet
Simple 2 layer convnet <100K parameter	None	0.925	@hardmaru	pytorch_tiny_custom_mnist_adam
GRU+SVM	None	0.888	@AFAgarap	gru_svm_zalando_dropout
GRU+SVM with dropout	None	0.855	@AFAgarap	gru_svm_zalando_dropout
WRN40-4 8.9M params	standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips)	0.967	@ajbrock	😢 NA
DenseNet-BC 768K	standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips)	0.954	@ajbrock	😢 NA

Other Explorations

Generative adversarial networks (GANs)

Visualization

t-SNE on Fashion-MNIST (left) and original MNIST (right)

PCA on Fashion-MNIST (left) and original MNIST (right)

Contributing

Thanks for your interest in contributing! There are many ways to contribute to this project. Get started here! And please check these open issues for specific tasks.

Contact

For discussion on the dataset, please use

Citing Fashion-MNIST

If you use Fashion-MNIST in a scientific publication, we would appreciate references to the following paper:

Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. Han Xiao, Kashif Rasul, Roland Vollgraf. arXiv: TBA

Bibtex entry:

TBA

The article is scheduled to be announced on arXiv at Mon, 28 Aug 2017 00:00:00 GMT.

License

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

sarthusarth/fashion-mnist