An implementation of Variational LDA. See the related blogger blogpost for the details of how the formulas were derived. But in a nutshell.
Sorry for the LaTeX jargon! I wish github could render the formulas. To produce a PDF version of this file, try this command:
pandoc -t latex -o README.pdf README.md
- For k = 1 .. K:
$\varphi_k \sim \hbox{Dirichlet}_V(\beta)$
- For m = 1..M:
$\theta_m \sim \hbox{Dirichlet}_K(\alpha)$ - For n = 1 .. N_m:
$z_{mn} \sim \hbox{Multinomial}_K(\theta_m)$ - $w_{mn} \sim \hbox{Multinomial}V(\sum{i=1}^KZ_{mni}\varphi_i)$
- For m=1..M, n=1..N, k=1..K
$p_{z_{mnk}}=1/k$
- Repeat
- For k=1..K, v=1..V
$\beta_{\varphi_{kv}}=\beta+\sum_{m=1}^M\sum_{n=1}^Nw_{mnv}p_{z_{mnk}}$
- For m=1..M, k=1..K
$\alpha_{\theta_{mk}}=\alpha+\sum_{n=1}^Np_{z_{mnk}}$
- For m=1..M, n=1..N, k=1..K
$p_{z_{mnk}}=\exp\left(\psi(\alpha_{\theta_{mk}}) - \psi\left(\sum_{k'=1}^K\alpha_{\theta_{mk'}}\right) + \sum_{v=1}^Vw_{mnv}\left(\psi(\beta_{\varphi_{kv}})-\psi\left(\sum_{k'=1}^K\beta_{\varphi_{k'v}}\right)\right)\right)$
- For k=1..K, v=1..V