Tensorflow implementation of Learning semantic similarity in a continuous space , Deudon M.
To train/test a variational siamese network, you need some data.
The w2v folder holds pretrained models such as GloVe or Word2Vec. You can drag here any pretrained or custom model (We saved our weights and vocabulary in two files: embeddings_300d.p and vocab_300d.p).
In the data folder, you should put your data. We used quora_duplicate_questions.tsv. You should further put your train/dev/test files in the data/split folder (we saved those as .txt files). Your preprocessed data will be stored as .npy files in the folder named corpus.
- To train a variational siamese network with generative pretraining (repeat), consider running the following in the folder _deepNLU:
> python train.py VAE
- For the repeat, reformulate framework, consider running:
> python train.py VAD
The model will be saved in _deepNLU/save/.
Our neural architecture is specified in nnet.py and utility functions in utils.py.
Our data loader and preprocessing is specified in data.py.
- To visualize generative and discriminative training on tensorboard, run:
> tensorboard --logdir=summary
- To test a trained model, you can run eval.ipynb with the Jupyter Notebook.
- Ecole Polytechnique for financial support
- Télécom Paris-Tech for computational resources
Special thanks:
- Magdalena Fuentes for helping with Télécom's GPU and setting up the required environment.
- Pr. Chloé Clavel, Pr. Gabriel Peyré, Constance Nozière and Paul Bertin for their critical reading of the paper.
- Pr. Francis Bach and Pr. Guillaume Obozinski, for their insightful course on probabilistic graphical models at ENS Cachan
- Pr. Michalis Vazirgiannis for his course on Text Mining and NLP at Ecole Polytechnique.
- We also thank the reviewers for their valuable comments and feedback.
Michel Deudon / @mdeudon