Issue with maximum likelihood function in Fitting a model to data tutorial

[noahfranz13]

General information:

• emcee version: 3.1.2
• platform: Ubuntu 20.04
• installation method (pip/conda/source/other?): pip

Problem description:

In tutorial "Fitting a model to data" under the section "Maximum likelihood estimation" the maximum likelihood for that linear model is defined as

$$ \ln p(y | x,\sigma,m,b,f) = -\frac{1}{2} \sum_n \left[ \frac{(y_n-m,x_n-b)^2}{s_n^2} + \ln \left ( 2\pi s_n^2 \right ) \right] $$

But then in the code below, the log_likelihood function returns

-0.5 * np.sum((y - model) ** 2 / sigma2 + np.log(sigma2))

where it seems like the $2\pi$ from the original equation is missing in the added log term. Without the $2\pi$, while running the given tutorial it gives the following values for m, b, and log(f)

$$m=−1.007^{0.077}_{−0.081}$$

$$b=4.550^{0.366}_{−0.358}$$

$$log(f)=−0.772^{0.161}_{−0.148}$$

If I add in the factor of $2\pi$ and rerun I instead get

$$m=−1.007^{0.077}_{−0.079}$$

$$b=4.549^{0.360}_{−0.359}$$

$$log(f)=−0.770^{0.162}_{−0.150}$$

Where the values for b and log(f) are different at the last decimal place and the errors on all three are different. I have ensured that these small differences are not MCMC randomness or a seeding issue by exactly recreating the results in the tutorial consistently. I am also consistently recreating the results I get after introducing the factor of $2\pi$. For reference, I attached the jupyter notebook where I fixed this issue as a txt file (sorry about this but it wouldn't let me upload a notebook directly, if you download it and change the extension it should work).
model.txt

[dfm]

I'm not sure what you mean by "I have ensured that these small differences are not MCMC randomness", but this seems totally consistent to me! ArviZ estimates the MC error on the means as 0.001, 0.006, and 0.003, respectively so these differences are well within what you would expect for Monte Carlo sampling error.

More formally: the factor of -0.5*n*log(2*pi) is an additive constant to the likelihood and this cannot have an effect on the results since MCMC algorithms only ever evaluate differences of log probabilities. There can, however, be numerical differences introduced by floating point precision that can affect the specific traces when adding constants to the log probability, and I expect that that's what you're seeing here.

[noahfranz13]

Sorry for the confusion, I just meant that the numpy random seed were all set correctly and that this wasn't the issue.

This explanation for these small differences make a lot of sense. Thank you for your quick response!

[dfm]

Great! Yeah - I think the situation should be ok here. Feel free to re-open if related issues show up.