Clarify simulate_log_probability_of_individuals_observed_choice().

[tobiasraabe]

• respy version used, if any: 2.0.0-dev
• Python version, if any: any
• Operating System: any

Description

Writing multiple times to the matrix value_functions does hurt the clarity of the function. The first values are indeed alternative-specific value functions, but the second time its the exponentiated difference between the alternative-specific value function and the value function.

This is a good first issue to get used to Numba and guvectorize. See the following links to understand how the functions work.

• https://numba.pydata.org/numba-doc/dev/user/vectorize.html
• or here

Todo

• Remove multiple writing to the same matrix.
• Find a better name for the second values also called value_functions.
• Make sure that the function does not become slower.

[tobiasraabe]

While looking at the function today, I thought we need to do more than some simple changes. I chose to incorporate multiple steps into the function because it made the likelihood faster. I am sure a more readable and at least as fast implementation is possible.

The function basically does this:

1. Simulate alternative-specific value functions.
2. Take the maximum over alternative-specific value functions.
3. Calculate the differences of alternative-specific value function minus the maximum.
4. Divide all differences by smoothing parameter tau to prevent probabilities of zero in the next step.
5. Pass values to a softmax function to get probabilities
6. Take the log of probabilities.

• The first three steps are similar to what happens in the interpolation routine. Maybe there are synergies.
• The softmax function is discussed in #243, but here we can summarize 5 and 6 by using a more efficient log_softmax function discussed here. It is also numerically more stable as the old one because of something similar to @janosg's fix in the likelihood 😄 .

[janosg]

Interesting how famous the trick is! Maybe I should have googled longer before wasting an afternoon on re-inventing the wheel.

It looks like the version you linked to guards only against overflow while my scaling factors avoid over and underflow at the same time.

[tobiasraabe]

I think this post might have a better explanation, but I am not sure I understand it completely.

We use a little algebraic trick to prevent overflow and underflow while preserving as many accurate leading digits in the result as possible:

log_sum_exp(u, v) = max(u, v) + log(exp(u - max(u, v)) + exp(v - max(u, v)))

The leading digits are preserved by pulling the maximum outside. The arithmetic is robust because subtracting the maximum on the inside makes sure that only negative numbers or zero are ever exponentiated, so there can be no overflow on those calculations. If there is underflow, we know the leading digits have already been returned as part of the max term on the outside.

Does it mean that they take the max because they only care about the larger values. Smaller values are irrelevant to the sum anyway and become zero in exp. In that sense your algorithm is too soft on larger values.

[janosg]

Ok, now I understand. I think my method prevents underflow in more cases, but probably those cases are not relevant and their algorithm has a higher precision in the end.

[tobiasraabe]

Closed in #278.