COLLAPSED VARIATIONAL BAYESIAN INFERENCE FOR LATENT DIRICHLET ALLOCATION (CVB-LDA) Version 0.1 David Andrzejewski (andrzeje@cs.wisc.edu) Department of Computer Sciences University of Wisconsin-Madison, USA This software implements Collapsed Variational Bayesian (CVB) inference [1] for the LDA model [2] of discrete count data. The code is based on the Teh et al paper [1], but also uses some practical implementation details kindly provided by the authors on the extremely helpful topic-models mailing list: https://lists.cs.princeton.edu/mailman/listinfo/topic-models The code implements the collapsed variational inference for LDA as a Python C extension module. DISCLAIMER Importantly, I wrote this code as an educational exercise for *my own benefit only*. This software has no connection with [1] or its authors whatsoever. Any errors contained herein are my own. BUILD/INSTALL Building this module requires installation of Python and NumPy, as well as a C compiler. From the command-line, do: % python setup.py install (Note that if NumPy or Python are installed to non-standard locations, you may need to make the appropriate changes in setup.py) You can then test the installation with: % python test/testCvbLDA.py -v There is also a simple example scipt showing how to use the software: % python example.py LOCAL INSTALL If you do not have write access to your Python installation directories, you will need to tell setup.py to install this module somewhere else. For example: % python setup.py install --prefix=~/local will install the module under a subdirectory of your home directory called "local". It may then be necessary to let Python know where that is by setting the PYTHONPATH environment variable (e.g., in .bashrc or .cshrc). For our example this might involve adding something like the line: setenv PYTHONPATH ~/local/lib/python2.5/site-packages HOW TO USE The commenting in the example.py script explains the meanings and types of all input and return arguments. For more detailed examples of good/bad inputs, you can examine the tests in test/testCvbLDA.py. LICENSE This software is open-source, released under the terms of the GNU General Public License version 3, or any later version of the GPL (see LICENSE.txt). REFERENCES [1] A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation Teh Y.W., Newman D., and Welling, M. Advances in Neural Information Processing Systems (NIPS) 19, 2007. [2] Latent Dirichlet Allocation Blei, D. M., Ng, A. Y., and Jordan, M. I. Journal of Machine Learning Research (JMLR), 3, Mar. 2003, 993-1022. VERSION HISTORY 0.1 Initial release 0.1.1 Return (phi,theta,gamma) instead of just (phi,theta) gamma is a length D List of T x Md NumPy arrays