Jupyter Notebooks to demonstrate Factor Analysis (FA), t-distributed stochastic neighbor embedding (t-SNE) and Auto Encoders (AE)
The factor Analysis examples are a mix of the following three different blog posts:
- How to do factor analysis by Nick Elprin: this uses the standard R 'Psych' package.
- Fitting a Bayesian Factor Analysis Model in Stan by Rick Farouni: replicates the model in Stan.
- Probabilistic Factor Analysis Methods by Suleiman A. Khan & Joseph H. Sakaya: uses the ADVI (Automatic Variational Inference in Stan) inference method rather than MCMC sampling and combines the model with ARD (Automatic Relevance Determination, see chapter 13.7 "Automatic relevance determination (ARD)/sparse Bayesian learning (SBL)" of Machine Learning: A Probabilistic Perspective) to automatically detect the number of latent dimensions.
The below notebooks are translations of the above examples to python plus some additional features that I thought were missing from the above, e.g. an evaluation of the quality of the fit.
- Factor Analysis via R 'Psych': Using the standard R 'Psych' package.
- Factor Analysis via MCMC sampling: Reimplementing the model in Stan.
- Factor Analysis with Automatic Relevance Determination and ADVI: Adding Automatic Relevance Determination (ARD) to the model.
- Some geometric aspects of the multi-variate normal distribution as they relate to factor analysis: