This repository holds the official code for the paper Deep Conditional Gaussian Mixture Model for Constrained Clustering (link to paper), accepted at NeurIPS 2021.
Clustering with constraints has gained significant attention in the field of constrained machine learning as it can leverage partial prior information on a growing amount of unlabelled data. Following recent advances in deep generative models, we derive a novel probabilistic approach to constrained clustering that can be trained efficiently in the framework of stochastic gradient variational Bayes. In contrast to existing approaches, our model (DC-GMM) uncovers the underlying distribution of the data conditioned on prior clustering preferences, expressed as \textit{pairwise constraints}. The inclusion of such constraints allows the user to drive the clustering process towards a desirable configuration by indicating which samples should or should not belong to the same class.
To download Reuters data, run the following:
cd dataset/reuters
sh download_data.sh
Download STL data (Matlab files) from https://cs.stanford.edu/~acoates/stl10/. Save them in dataset/stl10/stl10_matlab. Then run the following:
cd dataset/stl10
python compute_stl_features.py
To download and configure the UTKFace datset:
- Download the cropped and aligned dataset archive from https://susanqq.github.io/UTKFace/
- Extract the images from this archive to
<code root>/dataset/utkface
To run DC-GMM using the default setting on MNIST data set:
python main.py
To run DC-GMM without pairwise constraints using the default setting:
python main.py --num_constrains 0
To choose different configurations of the hyper-parameters:
python main.py --data ... num_constrains ... --alpha ... --lr ...
Important hyper-parameters:
- data: choose from MNIST, fMNIST, Reuters, har, utkface
- num_constrains: by default it should be set to 6000 (note that the total number of pairwise constraints in a dataset is O(N*N))
- alpha: measure the confidence in your labels (default is 10000)
- pretrain: False if you want to use your own pretrain weights
In the current implementation, the pairwise constraints are obtained from labels by randomly sampled two data points and assigning a must-link constraint (+1) if the two samples have the same label and a cannot-link constraint (-1) otherwise. The pairwise constraints are stored in a matrix W.
See the file:
source/data.py