Matlab code and functions for the testing scenarios analysed in "A tutorial on uncertainty modeling for machine reasoning".
Increasingly we rely on machine intelligence for reasoning and decision making under uncertainty. The tutorial reviews the prevalent methods for model-based autonomous decision making based on observations and prior knowledge, primarily in the context of classification. Both observations and the knowledge-base available for reasoning are treated as being uncertain. Accordingly, the central themes of this tutorial are quantitative modeling of uncertainty, the rules required to combine such uncertain information, and the task of decision making under uncertainty. The paper covers the main approaches to uncertain knowledge representation and reasoning, in particular:
- Bayesian probability theory
- Possibility theory
- Dempster-Shafer belief functions theory (using transferrable belief model)
- Imprecise probability theory
These approaches are illustrated on several testing scenarios as outlined below:
Testing Scenario 1 - Simple Classification
- Script_1 implements a Bayesian classifier
- Script_5 implements a possibilistic classifier
- Script_9 implements a transferrable belief model classifier
Testing Scenario 2 - Classification under Randomness via Monte-Carlo Simulations
- Script_2 implements a Bayesian classifier
- Script_6 implements a possibilistic classifier
- Script_10 implements a transferrable belief model classifier
Testing Scenario 3 - Imprecise Likelihood Specification
- Script_3 implements a standard Bayesian approach
- Script_4 implements Mahler's approach using random sets
- Script_7 implements a possibilistic solution
- Script_11 implements a transferrable belief model solution
- Script_14 implements a imprecise probabilistic solution
Testing Scenario 4 - Model Mismatch
- Script_8 compares the Bayesian and possibilistic classifiers when using an incorrect confusion matrix (Monte-Carlo runs)
- Script_12 compares the Bayesian, possibilistic and transferrable belief model classifiers when using an incorrect confusion matrix (Monte-Carlo runs)
- Script_15 solves a modified version of the scenerio using Imprecise probability theory - rather than an incorrect, the confusion matrix is imprecise.
Testing Scenario 5 - Dealing with different subspaces
- Script_13 implements a transferrable belief model solution
Scripts 1 to 13 and 15 were written by B. Ristic.
Script 14 was written by A. Benavoli
- B. Ristic, C. Gilliam, M. Byrne, and A. Benavoli, "A tutorial on uncertainty modeling for machine reasoning", Information Fusion, Vol 55, pp. 30-44, 2020.