Replace predict_multinomial_logit with scipy.special.softmax.

[tobiasraabe]

• respy version used, if any: 2.0.0-dev
• Python version, if any: any
• Operating System: any

Description

In the estimation, the individual likelihood contributions are calculated for each type and then weighted by the type-specific probability of the individual. To calculate the type-specific probability, respy relies on predict_multinomial_logit() which is just guvectorized variant of a softmax function.

• https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.softmax.html

While coding the function, I was not aware that a multinomial logit regression is more commonly known as a softmax regression and thus I did not look for it. scipy has its own implementation of a softmax function as a special function. First, we can use this function and safe some code.

Second, the function is called every estimation and simulation step. Thus, it has to be sufficiently fast. For performance, register the special function with Numba.

• numba/numba#3086 (comment)

Todo

1. Prepare a notebook with toy data. Keane and Wolpin (1997) use two covariates/coefficients for 1,000 individuals. Maybe extend the problem size up to 10k individuals and 10 covariates/coefficients.
2. Time the existing predict_multinomial_logit() and scipy.special.softmax with and without Numba.

[tobiasraabe]

I just recognized that Numba launched a new package numba-scipy which will support scipy's special functions and other stuff in the future. It is community-driven so that the core devs are mostly working on Numba itself. If the special function is registered with Numba, we can contribute to numba-scipy and give a little bit back to a tremendous helpful project.

[tobiasraabe]

Maybe the function will also be included in https://github.com/numba/numba-scipy/pull/12.

[janosg]

I just recognized that Numba launched a new package numba-scipy which will support scipy's special functions and other stuff in the future. It is community-driven so that the core devs are mostly working on Numba itself. If the special function is registered with Numba, we can contribute to numba-scipy and give a little bit back to a tremendous helpful project.

I have already registered some special functions with numba. If you decide to make a PR we can include that code.

[tobiasraabe]

Actually, before we implement scipy.special.softmax we need to make sure that it is the numerically more stable version of the softmax discussed here. It is a similar fix like the one @janosg has implemented in the likelihood.

[tobiasraabe]

Closed in #279.