Histogram average gives wrong error

Question

Histogram average gives wrong error

NGoetz opened this issue 8 months ago · 9 comments

The average function of Histogram is "grob falsch". The standard error is not calculated correctly.
I would replace this function by calling average_weighted with unit weights (DRY!). We should investigate why this was not caught by tests.

Answer 1 · 2024-07-10T15:26:04.000Z

The reason why the tests let that through is because we only check for the value, but not the error in the test for average. This is sad. Additionally, average_weighted is never tested as far as I can see.

Answer 2 · 2024-07-10T15:45:39.000Z

You mean this: self.error_ = np.sqrt(np.sum(self.histograms_, axis=0) / self.number_of_histograms_) is wrong? This is just the $\sqrt{N}$ of the averaged histograms assuming Poissonian statistics if I'm right.

Answer 3 · 2024-07-10T15:51:52.000Z

I would have assumed to get the standard error just like for the average_weighted. The documentation claims that too. If we want a Poisson distributed error, that should be an optional argument, for both functions, and independently tested.

Answer 4 · 2024-07-10T15:57:16.000Z

The standard deviation for Poissonian statistics is $\sigma= \sqrt{N}$, so this statement is correct, at least for a single histogram.

Answer 5 · 2024-07-10T16:01:41.000Z

It's correct for Poissonian, but the documentation says standard error. Isn't standard error also a more reasonable default?

Answer 6 · 2024-07-10T16:04:46.000Z

We should discuss with the others on Monday, what we want to put as a default.

Answer 7 · 2024-07-11T07:01:19.000Z

I agree.
An additional idea would be to add the option of error weighted average for weighted average right out of the box.

Answer 8 · 2024-07-15T09:48:15.000Z

I think the standard error in the average_weighted is not even the standard error tbh. there is a factor missing.

Answer 9 · 2024-07-15T11:43:16.000Z

raw_count is also not updated...? Bug or feature?