vasishth/bayescogsci

Ex 2.4: Rephrasing proposal

Closed this issue · 1 comments

As stated, Exercise 2.4 needs some repairs (just read through it once).

To mirror the sequencing of other examples and exercises, I suggest naming the parameter of interest first rather than second. Something along these lines:

Suppose that we have $n$ data points, $x_1,\ldots,x_n$, drawn independently from an exponential distribution with parameter $\lambda$. We would like to know $\lambda$. The likelihood function for our observations is
$p(x_1,\ldots,x_n|\lambda) = \lambda^n \exp\left(−\lambda\sum_{k=1}^n x_k\right).$
Starting with a Gamma prior distribution for $\lambda$ (see below), show that the posterior distribution for $\lambda$ is also a Gamma distribution. Provide formulas giving the posterior parameters $a^{*},b^{*}$ in terms of the prior parameters $a,b$ and the data. Use the following facts about Gamma distributions: $\ldots$

Yes, this is much better than what I wrote. I have rewritten this as:

Suppose that we perform $n$ independent trials until we get a success (e.g., a heads in a coin toss). For coin tosses, the possible outcomes could be, H, TH, \dots. The probability of success in each trial is $\theta$. Then, the Geometric random variable, call it $X$, gives us the probability of getting a success in $n$ trials as follows:

\begin{equation}
Prob(X=n)=\theta(1-\theta)^{ n-1}
\end{equation}

where $n=1,2,\dots$.

Let the prior on $\theta$ be $\mathit{Beta}(a,b)$, a beta distribution with parameters $a$,$b$.
The posterior distribution is a beta distribution with parameters $a*$ and $b*$. Determine these parameters in terms of $a$, $b$, and $n$.

Conjugate forms 3

Suppose that we have $n$ data points, $x_1,\dots,x_n$, drawn independently from an exponential distribution with parameter $\lambda$. The parameter of interest here (what we want to learn about from the data) is $\lambda$.

The \index{Exponential likelihood} exponential likelihood function is:

\begin{equation}
p(x_1,\dots,x_n | \lambda)=\lambda^n \exp (-\lambda \sum_{i=1}^n x_i )
\end{equation}

Starting with a Gamma prior distribution for $\lambda$ (see below), show that the posterior distribution for $\lambda$ is also a Gamma distribution. Provide formulas giving the posterior parameters $a^, b^$ in terms of the prior parameters $a, b$ and the data. Use the following facts about Gamma distributions.

The \index{Gamma distribution} Gamma distribution is defined in terms of the parameters $a$, $b$: $\mathit{Gamma}(a,b)$. In general, if there is a random variable $Y$ (where $y\geq 0$) that has a Gamma distribution as a PDF ($Y\sim \mathit{Gamma}(a,b)$), then:

\begin{equation}
\mathit{Gamma}(y | a,b)=\frac{b^a y^{a-1} \exp(-by)}{\Gamma(a)}
\end{equation}

The $\mathit{Gamma}(a,b)$ prior on the $\lambda$ parameter in the exponential distribution will be written:

\begin{equation}
\mathit{Gamma}(\lambda | a,b)=\frac{b^a \lambda^{a-1} \exp(-b\lambda)}{\Gamma(a)}
\end{equation}