# BDA3 Chapter 5 Exercise 7

Here’s my solution to exercise 7, chapter 5, of Gelman’s Bayesian Data Analysis (BDA), 3rd edition. There are solutions to some of the exercises on the book’s webpage.

## Part a

Suppose $$y \mid \theta \sim \dpois(\theta)$$ with prior $$\theta \sim \dgamma(\alpha, \beta)$$. Let’s derive the expectation and variance of $$y$$. Using equation 1.8 (page 21), the expectation is

\begin{align} \mathbb E (y) &= \mathbb E \left( \mathbb E(y \mid \theta) \right) \\ &= \mathbb E \left( \theta \right) \\ &= \frac{\alpha}{\beta} . \end{align}

Using equation 1.9 (page 21), the variance is

\begin{align} \mathbb V (y) &= \mathbb E \left( \mathbb V (y \mid \theta) \right) + \mathbb V \left( \mathbb E (y \mid \theta) \right) \\ &= \mathbb E \left( \theta \right) + \mathbb V (\theta) \\ &= \frac{\alpha}{\beta} + \frac{\alpha}{\beta^2} \\ &= \alpha \frac{1 + \beta}{\beta^2} . \end{align}

## Part b

Suppose $$y \mid \mu, \sigma \sim \dnorm(\mu, \sigma)$$ with prior $$p(\mu, \sigma^2) \propto \sigma^{-2}$$. Then the expectation of $$\mu \mid y$$ is

\begin{align} \mathbb E \left( \mu \mid y \right) &= \mathbb E \left( \mathbb E (\mu \mid \sigma^2, y) \mid y \right) \\ &= \mathbb E \left( \bar y \mid y \right) \\ &= \bar y . \end{align}

For posterior expectations, we condition on the data, which allows us to treat $$y$$, $$n$$, and $$s$$ as constants. Since $$\theta := \sqrt{n} (\mu - \bar y) / s$$ is a linear function of $$\mu$$, its posterior expectation is zero. For this to hold, it is necessary that $$n \ge 2$$ for $$s$$ to be well-defined (to avoid division by zero). Moreover, the first identity implicitly assumes that the expectation $$\mathbb E (u \mid y)$$ is well-defined. Combining the calculation of $$p(\mu \mid y)$$ with this proof shows that it is well-defined only when $$n \ge 3$$.

The posterior variance is

\begin{align} \mathbb V \left( \frac{\sqrt{n}}{s} (\mu - \bar y) \mid y \right) &= \mathbb E \left( \mathbb V \left(\frac{\sqrt{n}}{s} (\mu - \bar y) \mid \sigma^2, y \right) \mid y \right) + \mathbb V \left( \mathbb E \left(\frac{\sqrt{n}}{s} (\mu - \bar y) \mid \sigma^2, y \right) \mid y \right) \\ &= \mathbb E \left( \frac{n}{s^2} \mathbb V \left( \mu \mid \sigma^2, y \right) \mid y \right) + \mathbb V \left( 0 \mid y \right) \\ &= \mathbb E \left(\frac{n}{s^2} \frac{\sigma^2}{n} \mid y \right) + 0 \\ &= \frac{n - 1}{n - 3} . \end{align}

Since $$n - 3$$ appears in the denominator, it is necessary that $$n \ge 4$$. Again, for the first identity to hold, it is implicitly assumed that the variance on the left hand side is finite. It can be verified that the variance is finite for $$n \ge 4$$.