---------------- | Introduction | ---------------- This package reproduces the experiments in the paper ------------------------------------------------------------------------------ Robert Peharz, Robert Gens, Franz Pernkopf and Pedro Domingos, "On the Latent Variable Interpretation in Sum-Product Networks", IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), vol. 39(10), pp. 2030-2044, 2017. ------------------------------------------------------------------------------ In particular, it provides an implementation of the EM algorithm for sum-product networks with Gaussians as input distributions, and an implementation for MPE inference in (augmented) SPNs over discrete variables. PLEASE NOTE THE ACCOMPANYING LICENSE FILE (modified BSD, 3-Clause). IF YOU USE THIS CODE FOR RESEARCH, PLEASE CITE THE PAPER ABOVE. ------------------- | Getting Started | ------------------- * Copy the content of this package to some folder, say foo. Within foo, you should then have the following folder structure. /CPP/src/ /Data/ /Matlab/ /Models/ /PoonDomingos/ /Results/EM/ /Results/MPE/ * There are C++ source files in /CPP/src/ and Matlab script files in /Matlab/ -- the other folders are empty so far. The experiments are actually performed in C++, and Matlab is used only as script language and for plotting and displaying results. So Matlab is not really necessary and can be replaced with something else, e.g. Octave. In this case, probably some minor changes to the scripts need to be done. We ran our experiments using Matlab2015. * To compile the C++ binaries, go to the folder /CPP/ and type 'make'. This should produce /CPP/bin/ containing the binaries 'MPEsynth' and 'trainEM', and /CPP/obj/ containing the object files. ------------------------------ | Running the EM experiments | ------------------------------ * Download the SPN package by Hoifung Poon and Pedro Domingos http://alchemy.cs.washington.edu/spn/ and unzip it into folder /PoonDomingos/. You should then have the paths /PoonDomingos/code/ (not required here, but can be left there) /PoonDomingos/data/ /PoonDomingos/results/ You can also unzip it to some other place; in this case adapt 'PoonDomingosRelPath' in /Matlab/setPaths.m accordingly. * Run 'convertData.m'. This converts the datasets by Poon & Domingos into a format used here and stores them in folder /Data/. * Run 'runExperimentsEM.m'. This will run EM for 7*4*3*103 = 8652 configurations (combinations of parameters weights/means/sigmas (7), original parameters and 3 random initializations (4), no missing data/33% missing data/66% missing data (3), and 103 datasets; see paper), and in total 8652 * 30 = 259560 EM iterations. If using only a single worker, this will take several weeks. However, we use a simple pseudo-locking mechanism to enable several workers sharing the same file system: The result for every training configuration is stored in a separate file in the folder /Results/EM/. When a result file is already there, a worker simply skips training for the corresponding configuration. When a certain result file is not yet there, the worker immediately creates a dummy place holder, signaling to other workers to skip this configuration. If something happens and some of your workers crash, they will leave these dummy file in the results folder. Call /Matlab/deleteDummyFiles.m to delete these. Alternatively, to reduce running time, you might want to reduce the number of training configurations. E.g. when setting MAKEMISSING_range = [0.0]; UPDATE_range = [0,0,1;... 0,1,1;... 1,0,1;... 1,1,1]; EM will only run for complete data and parameter configurations involving the sum-weights. You could also reduce the number of iterations, e.g. set numIter = 10, as EM converges pretty nicely within 10 iterations. * Run 'plotLLs.m' after EM training has been completed. This reproduces Figure 9 in the paper (EM training curves). Play around with 'updateFlags', 'includeNumRandomInits', 'includeOriginalInits', 'includeMAKEMISSING' in order to plot training curves for various other settings. * Run 'checkMonotonicityEM.m' to check monotonicity of EM on the training set (up to numerical errors after EM has converged). ------------------------------- | Running the MPE experiments | ------------------------------- * Run 'runExperimentsMPE.m'. This script just calls the C++ binary 'MPEsynth' which runs the actual experiments and writes results to /Results/MPE/. After that, it loads the results and displays them in tables.