/awesome-spn

A structured list of resources about Sum-Product Networks (SPNs)

Awesome Sum-Product Networks

awesome-spn is a curated and structured list of resources about Sum-Product Networks (SPNs), tractable deep density estimators.

This includes (even not formally published) research papers sorted by year and topics as well as links to tutorials and code and other related Tractable Probabilistic Models (TPMs). It is inspired by the SPN page at the Washington University.

Licence and Contributing

CC0

awesome-spn is released under Public Domain. Feel free to complete and/or correct any of these lists. Pull requests are very welcome!

Table of Contents

Papers

Sorted by year or topics

Year

2016

2015

2014

2013

2012

2011

Topics

#### Weight Learning - [[Jaini2016](#jaini2016)] [**Online Algorithms for Sum-Product Networks with Continuous Variables**](http://jmlr.org/proceedings/papers/v52/jaini16.pdf) `OBMM` - [[Desana2016](#desana2016)] [**Expectation Maximization for Sum-Product Networks as Exponential Family Mixture Models**](http://arxiv.org/abs/1604.07243) `EM` - [[Zhao2016b](#zhao2016b)] [**A unified approach for learning the parameters of sum-product networks**](http://arxiv.org/abs/1601.00318) - [[Zhao2016a](#zhao2016a)] [**Collapsed Variational Inference for Sum-Product Networks**](http://jmlr.org/proceedings/papers/v48/zhaoa16.pdf) `non-parametrics` - [[Rashwan2016](#rashwan2016)] [**Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks**](http://www.jmlr.org/proceedings/papers/v51/rashwan16.pdf) `OBMM` `EGD` - [[Peharz2014b](#peharz2014b)] [**Learning Selective Sum-Product Networks**](http://spn.cs.washington.edu/papers/ltpm2014_paper_9.pdf) `ML` `SSPN` - [[Poon2011](#poon2011)] [**Sum-Product Networks: A New Deep Architecture**](http://spn.cs.washington.edu/papers/spn.pdf) `EM` `Hard EM` `SGD` - [[Gens2012](#gens2012)] [**Discriminative Learning of Sum-Product Networks**](http://spn.cs.washington.edu/papers/dspn.pdf) `disc Hard EM` `disc Hard SGD`
#### Structure Learning - [[Melibari2016c](#melibari2016c)][**Dynamic Sum-Product Networks for Tractable Inference on Sequence Data**](http://arxiv.org/abs/1511.04412) `hill-climbing` - [[Rahman2016](#rahman2016)] [**Merging Strategies for Sum-Product Networks: From Trees to Graphs**](http://www.hlt.utdallas.edu/~vgogate/papers/uai16.pdf) `pruning` `dagSPN` - [[Vergari2015](#vergari2015)] [**Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning**](http://www.di.uniba.it/~vergari/papers/Simplifying,%20Regularizing%20and%20Strengthening%20Sum-Product%20Network%20Structure%20Learning.pdf) `LearnSPN-b` `LearnSPN-bt` `LearnSPN-btb` - [[Dennis2015](#dennis2015)] [**Greedy Structure Search for Sum-Product Networks**](http://www.ijcai.org/Proceedings/15/Papers/136.pdf) `dagSPN` - [[Adel2015](#adel2015)] [**Learning the Structure of Sum-Product Networks via an SVD-based Algorithm**](http://auai.org/uai2015/proceedings/papers/83.pdf) `SPN-SVD` `DSPN-SVD` - [[Nath2015](#nath2015)] [**Learning Relational Sum-Product Networks**](http://homes.cs.washington.edu/~pedrod/papers/aaai15.pdf) `relational` - [[Lee2014](#lee2014)] [Non-Parametric Bayesian Sum-Product Networks](http://spn.cs.washington.edu/papers/ltpm2014_paper_6.pdf) `non-parametrics` - [[Peharz2014b](#peharz2014b)] [**Learning Selective Sum-Product Networks**](http://spn.cs.washington.edu/papers/ltpm2014_paper_9.pdf) `SSPN` - [[Rooshenas2014](#rooshenas2014)] [**Learning Sum-Product Networks with Direct and Indirect Interactions**](http://ix.cs.uoregon.edu/~lowd/icml14rooshenas.pdf) `ID-SPN` - [[Lee2013](#lee2013)] [**Online Incremental Structure Learning of Sum-Product Networks**](https://bi.snu.ac.kr/Publications/Conferences/International/ICONIP2013_SWLee.pdf) - [[Peharz2013](#peharz2013)] [**Greedy Part-Wise Learning of Sum-Product Networks**](https://www.spsc.tugraz.at/sites/default/files/MergeSPN.pdf) `bottom-up` - [[Gens2013](#gens2013)] [**Learning the Structure of Sum-Product Networks**](http://jmlr.org/proceedings/papers/v28/gens13.pdf) `top-down` `LearnSPN` - [[Dennis2012](#dennis2012)] [**Learning the Architecture of Sum-Product Networks Using Clustering on Variables**](http://papers.nips.cc/paper/4544-learning-the-architecture-of-sum-product-networks-using-clustering-on-variables.pdf) `top-down``k-means`
#### Modeling - [[Melibari2016c](#melibari2016c)][**Dynamic Sum-Product Networks for Tractable Inference on Sequence Data**](http://arxiv.org/abs/1511.04412) `dynamic-SPN` - [[Melibari2016b](#melibari2016b)] [**Sum-Product-Max Networks for Tractable Decision Making**](http://trust.sce.ntu.edu.sg/aamas16/pdfs/p1419.pdf) `decision-diagram` - [[Melibari2016a](#melibari2016a)] [**Decision Sum-Product-Max Networks**](https://cs.uwaterloo.ca/~mmelibar/publications/melibari-aaai2016.pdf) `decision-diagram` - [[Friesen2015](#friesen2015)] [**Recursive Decomposition for Nonconvex Optimization**](https://www.cs.washington.edu/node/11282) `opt` - [[Niepert2015](#niepert2015)] [**Learning and Inference in Tractable Probabilistic Knowledge Bases**](http://homes.cs.washington.edu/~pedrod/papers/uai15.pdf) `relational` - [[Nath2015](#nath2015)] [**Learning Relational Sum-Product Networks**](http://homes.cs.washington.edu/~pedrod/papers/aaai15.pdf) `relational` - [[Nath2014](#nath2014)] [**Learning Tractable Statistical Relational Models**](http://spn.cs.washington.edu/papers/ltpm2014_paper_4.pdf) `relational` - [[Peharz2014b](#peharz2014b)] [**Learning Selective Sum-Product Networks**](http://spn.cs.washington.edu/papers/ltpm2014_paper_9.pdf) `SSPN` - [[Stuhlmueller2012](#stuhlmueller2012)] [**Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs**](http://arxiv.org/abs/1206.3555) `FSPN` - [[Poon2011](#poon2011)] [**Sum-Product Networks: A New Deep Architecture**](http://spn.cs.washington.edu/papers/spn.pdf) `SPN`
#### Applications
#### Theory - [[Peharz2016](#peharz2016)] [**On the Latent Variable Interpretation in Sum-Product Networks**](http://arxiv.org/abs/1601.06180) `EM` - [[Friesen2016](#friesen2016)] [**The Sum-Product Theorem: A Foundation for Learning Tractable Models**](http://homes.cs.washington.edu/~pedrod/papers/mlc16.pdf) `opt` `sum-prod-theorem` - [[Peharz2015b](#peharz2015b)] [**Foundations of Sum-Product Networks for Probabilistic Modeling**](https://www.researchgate.net/profile/Robert_Peharz/publication/273000973_Foundations_of_Sum-Product_Networks_for_Probabilistic_Modeling/links/54f49ff00cf2f28c1362088b.pdf) - [[Friesen2015](#friesen2015)] [**Recursive Decomposition for Nonconvex Optimization**](https://www.cs.washington.edu/node/11282) `opt` `sum-prod-theorem` - [[Zhao2015](#zhao2015)] [**On the Relationship between Sum-Product Networks and Bayesian Networks**](http://jmlr.org/proceedings/papers/v37/zhaoc15.pdf) - [[Peharz2015a](#peharz2015a)] [**On Theoretical Properties of Sum-Product Networks**](http://www.jmlr.org/proceedings/papers/v38/peharz15.pdf) - [[Martens2014](#martens2014)] [**On the Expressive Efficiency of Sum Product Networks**](http://arxiv.org/abs/1411.7717) `depth` - [[Delalleau2011](#dellaleau2011)] [**Shallow vs. Deep Sum-Product Networks**](http://papers.nips.cc/paper/4350-shallow-vs-deep-sum-product-networks.pdf) `depth`

Related Works

Arithmetic Circuits

Other TPMs

Resources

Dataset

Code

Talks and Tutorials

References

[Adel2015]
Adel, Tameem and Balduzzi, David and Ghodsi, Ali
Learning the Structure of Sum-Product Networks via an SVD-based Algorithm
Uncertainty in Artificial Intelligence 2015

*

[Amer2012]
_Amer, Mohamed and Todorovic, Sinisa_
**Sum-Product Networks for Modeling Activities with Stochastic Structure**
2012 IEEE Conference on CVPR

*

[Amer2015]
_Amer, Mohamed and Todorovic, Sinisa_
**Sum Product Networks for Activity Recognition**
IEEE Transactions on Pattern Analysis and Machine Intelligence

*

[Cheng2014]
_Cheng, Wei-Chen and Kok, Stanley and Pham, Hoai Vu and Chieu, Hai Leong and Chai, Kian Ming Adam_
**Language modeling with Sum-Product Networks**
INTERSPEECH 2014

*

[Darwiche2003]
_Darwiche, Adnan_
**A Differential Approach to Inference in Bayesian Networks**
Journal of the ACM 2003.

*

[Dellaleau2011]
_Delalleau, Olivier and Bengio, Yoshua_
**Shallow vs. Deep Sum-Product Networks**
Advances in Neural Information Processing Systems 2011.

*

[Dennis2012]
_Dennis, Aaron and Ventura, Dan_
**Learning the Architecture of Sum-Product Networks Using Clustering on Varibles**
Advances in Neural Information Processing Systems 25

*

[Dennis2015]
_Dennis, Aaron and Ventura, Dan_
**Greedy Structure Search for Sum-product Networks**
International Joint Conference on Artificial Intelligence 2015

*

[Desana2016]
_Desana, Mattia and Schn{\"{o}}rr Christoph_
**Expectation Maximization for Sum-Product Networks as Exponential Family**
arxiv.org/abs/1604.07243

*

[Friesen2015]
_Friesen, Abram L and Domingos, Pedro_
**Recursive Decomposition for Nonconvex Optimization**
Proceedings of the 24th International Joint Conference on Artificial Intelligence

*

[Friesen2016]
_Friesen, Abram L and Domingos, Pedro_
**The Sum-Product Theorem: A Foundation for Learning Tractable Models**
ICML 2016

*

[Gens2012]
_Gens, Robert and Domingos, Pedro_
**Discriminative Learning of Sum-Product Networks**
NIPS 2012

*

[Gens2013]
_Gens, Robert and Domingos, Pedro_
**Learning the Structure of Sum-Product Networks**
ICML 2013

*

[Jaini2016]
_Jaini, Priyank and Rashwan, Abdullah and Zhao, Han and Liu, Yue and Banijamali, Ershad and Chen, Zhitang and Poupart, Pascal_
**Online Algorithms for Sum-Product Networks with Continuous Variables**
International Conference on Probabilistic Graphical Models 2016

*

[Krakovna2016]
_Krakovna, Viktoriya and Looks, Moshe_
**A Minimalistic Approach to Sum-Product Network Learning for Real Applications**
ICLR 2016

*

[Lee2013]
_Lee, Sang-Woo and Heo, Min-Oh and Zhang, Byoung-Tak_
**Online Incremental Structure Learning of Sum-Product Networks**
ICONIP 2013

*

[Lee2014]
_Lee, Sang-Woo and Watkins, Christopher and Zhang, Byoung-Tak_
**Non-Parametric Bayesian Sum-Product Networks**
Workshop on Learning Tractable Probabilistic Models 2014

*

[Li2015]
_Weizhuo Li_
**Combining sum-product network and noisy-or model for ontology matching**
Proceedings of the 10th International Workshop on Ontology Matching

*

[Livni2013]
_Livni, Roi and Shalev-Shwartz, Shai and Shamir, Ohad_
**A Provably Efficient Algorithm for Training Deep Networks**
arXiv 2013

*

[Lowd2013]
_Lowd, Daniel and Rooshenas, Amirmohammad_
**Learning Markov Networks With Arithmetic Circuits**
Proceedings of the 16th International Conference on Artificial Intelligence and Statistics 2013

*

[Martens2014]
_Martens, James and Medabalimi, Venkatesh_
**On the Expressive Efficiency of Sum Product Networks**
arXiv/1411.7717

*

[Melibari2016a]
_Melibari, Mazen and Poupart, Pascal and Doshi, Prashant_
**Decision Sum-Product-Max Networks**
Thirtieth AAAI Conference on Artificial Intelligence

*

[Melibari2016b]
_Melibari, Mazen and Poupart, Pascal and Doshi, Prashant_
**Sum-Product-Max Networks for Tractable Decision Making**
Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems

*

[Melibari2016c]
_Melibari, Mazen and Poupart, Pascal and Doshi, Prashant and Trimponias, George_
**Dynamic Sum-Product Networks for Tractable Inference on Sequence Data**
International Conference on Probabilistic Graphical Models 2016

*

[Nath2014]
_Nath, Aniruddh and Domingos, Pedro_
**Learning Tractable Statistical Relational Models**
Workshop on Learning Tractable Probabilistic Models

*

[Nath2015]
_Nath, Aniruddh and Domingos, Pedro_
**Learning Relational Sum-Product Networks**
AAAI 2015

*

[Nath2016]
_Nath, Aniruddh and Domingos, Pedro_
**Learning Tractable Probabilistic Models for Fault Localization**
AAAI 2016

*

[Niepert2015]
_Niepert, Mathias and Domingos, Pedro_
**Learning and Inference in Tractable Probabilistic Knowledge Bases**
UAI 2015

*

[Peharz2013]
_Peharz, Robert and Geiger, Bernhard and Pernkopf, Franz_
**Greedy Part-Wise Learning of Sum-Product Networks**
ECML-PKDD 2013

*

[Peharz2014a]
_Peharz, Robert and Kapeller, Georg and Mowlaee, Pejman and Pernkopf, Franz_
**Modeling Speech with Sum-Product Networks: Application to Bandwidth Extension**
ICASSP2014

*

[Peharz2014b]
_Robert Peharz and Gens, Robert and Domingos, Pedro_
**Learning Selective Sum-Product Networks**
Workshop on Learning Tractable Probabilistic Models 2014

*

[Peharz2015a]
_Robert Peharz and Tschiatschek, Sebastian and Pernkopf, Franz and Domingos, Pedro_
**On Theoretical Properties of Sum-Product Networks**
Proceedings of the 18th International Conference on Artificial Intelligence and Statistics

*

[Peharz2015b]
_Peharz, Robert_
**Foundations of Sum-Product Networks for Probabilistic Modeling**
PhD Thesis

*

[Peharz2016]
_Robert Peharz and Robert Gens and Franz Pernkopf and Pedro Domingos_
**On the Latent Variable Interpretation in Sum-Product Networks**
arxiv.org/abs/1601.06180

*

[Poon2011]
_Poon, Hoifung and Domingos, Pedro_
**Sum-Product Network: a New Deep Architecture**
UAI 2011

*

[Rahman2016]
_Tahrima Rahman and Vibhav Gogate_
**Merging Strategies for Sum-Product Networks: From Trees to Graphs**
UAI 2016

*

[Rashwan2016]
_Rashwan, Abdullah and Zhao, Han and Poupart, Pascal_
**Online and Distributed Bayesian Moment Matching for Parameter Learning in Sum-Product Networks**
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics

*

[Ratajczak2014]
_Ratajczak, Martin and Tschiatschek, S and Pernkopf, F_
**Sum-Product Networks for Structured Prediction: Context-Specific Deep Conditional Random Fields**
Workshop on Learning Tractable Probabilistic Models 2014

*

[Rooshenas2014]
_Rooshenas, Amirmohammad and Lowd, Daniel_
**Learning Sum-Product Networks with Direct and Indirect Variable Interactions**
ICML 2014

*

[Rooshenas2016]
_Rooshenas, Amirmohammad and Lowd, Daniel_
**Discriminative Structure Learning of Arithmetic Circuits**
Proceedings of the 19th International Conference on Artificial Intelligence and Statistics

*

[Stuhlmueller2012]
_Stuhlmuller, Andreas and Goodman, Noah D._
**A Dynamic Programming Algorithm for Inference in Recursive Probabilistic Programs**
StaRAI 2012

*

[Vergari2015]
_Vergari, Antonio and Di Mauro, Nicola and Esposito, Floriana_
**Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning**
ECML-PKDD 2015

*

[Vergari2016]
_Vergari, Antonio and Di Mauro, Nicola and Esposito, Floriana_
**Visualizing and Understanding Sum-Product Networks**
arXiv:1608.08266

*

[Wang2015]
_Wang, Jinghua and Wang, Gang_
**Hierarchical Spatial Sum-Product Networks for action recognition in Still Images**
arXiv:1511.05292

*

[Yuan2016]
_Zehuan Yuan and Hao Wang and Limin Wang and Tong Lu and Shivakumara Palaiahnakote and Chew Lim Tan_
**Modeling Spatial Layout for Scene Image Understanding Via a Novel Multiscale Sum-Product Network**
Expert Systems with Applications

*

[Zhao2015]
_Zhao, Han and Melibari, Mazen and Poupart, Pascal_
**On the Relationship between Sum-Product Networks and Bayesian Networks**
ICML 2015

*

[Zhao2016a]
_Zhao, Han and Adel, Tameem and Gordon, Geoff and Amos, Brandon_
**Collapsed Variational Inference for Sum-Product Networks**
ICML 2016

*

[Zhao2016b]
_Zhao, Han and Poupart, Pascal_
**A Unified Approach for Learning the Parameters of Sum-Product Networks**
NIPS 2016