Calculation of Posterior Probability

First of all,

thank you very much for translating this package into Python, based on Tensorflow Probabilities! Working with it and receiving results is super smooth and simple!

However, I have some problems understanding how the posterior probabilities are calculated in order to interpret them correctly. Therefore, I would like to ask for an explanation of how the "Posterior tail-area probability p" and the "Posterior probability of a causal effect" are calculated/derived. Not in a mathematical sense, but more in an explainable way like it is done in this thread about "Posterior tail-area probability p": https://stats.stackexchange.com/questions/263763/what-does-posterior-tail-area-probability-mean-in-causal-impact

Thanks, Elias

Hi Elias,

Not sure if I can add much more to the referred answer you mentioned. The "posterior tail-area p" is computed directly by observing the likelihood of obtaining the observed response given simulated counterfactual.

In other words, here's the part in the code that computes the value 'p'. Notice that it receives two inputs, simulated_ys and post_data_sum.

The first refers to thousands of simulations of what could have been observed had no intervention taken place. Those simulations are built using the posterior samples obtained after fitting the model to training data.

The second element is just the summation of all points from the response that was observed.

The computation then is to simply see from the simulated_ys how many of those the summation surpasses the second input post_data_sum. If not that many, then we have an indication that the likelihood of observing a summation as it was observed is rare, which is equivalent of saying that p-value is small. In those cases we assert that there was a statistical valid impact.

But if from the simulated_ys there are various simulations whose summations surpasses the observed response then chances of observing it at random are higher, which increases the 'p-value' and in those scenarios we simply conclude there's no statistical valid impact.

So in a nutshell it's a straightforward hypothesis testing, only this time instead of sampling a variable from the population and obtaining its 95% confidence interval and computing the z-score, we simply simulate lots of time series and see how many of those simulations can surpass what was observed and take conclusions from that.

Not sure if that was helpful, let me know if you have any questions.

Best,

Will

Hey Will,

thanks for your detailed reply! That was really helpful to me and answered my question about the "Posterior tail-area probability p"!

Just for reference for future readers - Calculation of "Posterior probability of a causal effect" is based on the p-value whose calculation was explained by Will before and can be found here:

tfcausalimpact/causalimpact/summary/templates/summary

Line 17 in d605737

Posterior prob. of a causal effect: {{((1 - p_value) * 100) | round(digits)}}%

Thanks again for your great work Will!

Best,
Elias