Simple app to see how posterior probability changes with variations in prior probability, specificity and sensitivity, designed for VoI (Value of Information) analysis.
To launch the app click the button below:
The idea is to estimate the posterior probability of an event H (e.g., a subsurface geological scenario being true) if we have some new data or information (A) with a certain accuracy. The conditional probability of H given A is expressed with the Bayes theorem:
P(H|A) = ( P(A|H) * P(H) )/ P(A).
The marginal probability P(A) is also equal to: P(A) = P(A|H) * P(H) + P(A|~H) * P(~H)
We can express P(A) in terms of sensitivity, i.e. the ability of the test or data to detect true positives or P(A|H), and specificity i.e. the ability of the test to detect true negatives or P(~A|~H). Sensitivity and specificity are two aspects of accuracy; please see wikipedia for further info! This is just a quick readme after all!
By replacing P(H) (prior probability) with POS (probability of success), we can write Bayes theorem in terms of POS, sensitivity and specificity that are key aspects taken into account during a VoI analysis:
P(H|A) = (sens * POS) / ( sens * POS + (1-spec) * (1-POS) )
The prior P(H) is entered as a range, and internally converted to a random gaussian distribution of n samples with mean equal to the mid value of the specified range.
The number of samples n can be selected with the three buttons Low, Medium or High that corresponds respectively to 20, 200 and 2000 samples.
By default a certain amount of uniform noise is added to both specificity and sensitivity to simulate subjectivity in the experts' assessment (in geosciences there are no repeatable experiments to be made so that we can estimate the reliability of a certain type of data or information). This noise can be removed entirely by decreasing the slider Max noise to 0.
The custom button above is made with mybinder's custom badge generator.
If you have a full conda installation with the required libraries installed (Numpy, Matplotlib and Panel and you know what you're doing, you can run the app locally by cloning this repo then run the following line from a shell:
$ panel serve --show voi_panel_app.ipynb