# BDA3 Chapter 2 Exercise 7

Here’s my solution to exercise 7, chapter 2, of Gelman’s Bayesian Data Analysis (BDA), 3rd edition. There are solutions to some of the exercises on the book’s webpage.

We show that a uniform prior on the natural parameter of a binomial model implies an improper prior under a different parameterisation.

The binomial likelihood can be written as a member of the exponential family as

\begin{align} \dbinomial(y \mid \theta) &= \binom{n}{y} \theta^y (1 - \theta)^{n - y} \\ &= \binom{n}{y} \cdot (1 - \theta)^n \cdot \exp \left(y \log \left(\frac{\theta}{1 - \theta}\right)\right) \\ &= f(y) \cdot g(\theta) \cdot \exp (\phi(\theta) \cdot u(y)) , \end{align}

where $$\phi(\theta) := \log \frac{\theta}{1 - \theta}$$, $$u(y) := y$$, $$g(\theta) := (1 - \theta)^n$$. Suppose the natural parameter $$\phi \sim \dbeta(1, 1)$$ is uniformly distributed. Then the distribution of $$\theta$$ is

\begin{align} p(\theta) &\propto p(\phi) \cdot \vert \invlogit^\prime (\phi) \vert^{-1} \\ &= \left \vert \frac{1}{1 + \exp(-\phi)}^\prime \right\vert^{-1} \\ &= \left \vert \frac{1}{\left(1 + \exp(-\phi)\right)^2} \cdot \exp(-\phi) \right\vert^{-1} \\ &= \frac{1 + 2 \exp(-\phi) + \exp(-2\phi)}{\exp(-\phi)} \\ &= \exp(\phi) + 2 + \exp(-\phi) \\ &= \frac{\theta}{1 - \theta} + 2 + \frac{1 - \theta}{\theta} \\ &= \frac{\theta^2 + 2\theta(1 - \theta) + (1 - \theta)^2}{\theta(1 - \theta)} \\ &= \frac{1}{\theta(1 - \theta)} \\ &= \theta^{-1}(1 - \theta)^{-1} \qquad \square \end{align}

This is an improper distribution on $$\theta$$ because

\begin{align} \int_0^1 \frac{1}{\theta(1 - \theta)} &\ge \int_0^1 \frac{1}{\theta} \\ &= \log\theta \vert_0^1 \\ &= \infty. \end{align}

When $$y = 0$$, then the posterior distribution is $$p(\theta \mid y = 0) \propto (1 - \theta)^{n - 1}\theta^{-1}$$. When $$y = n$$, then the posterior distribution is $$p(\theta \mid y = n) \propto \theta^{n-1}(1 - \theta)^{-1}$$. These two cases are equivalent by the change of variable $$\theta \mapsto 1 - \theta$$.

We show that the distribution is improper for $$y = 0$$ by induction. The case $$n = 0$$ is shown above (for the prior). Assume the distribution is improper for any integer $$k < n$$. Then using integration by parts yields

\begin{align} \int_0^1 \theta^{- 1}(1 - \theta)^{n - 1} d\theta &= \int_0^1 \theta^{- 1}(1 - \theta)^{n - 2} \cdot (1 - \theta)d\theta \\ &= \left[(1 - \theta)^{n-2}(1 - \frac{\theta}{2}) \right]_0^1 + \int_0^1 \frac{(1 - \theta)^{n - 2}}{\theta} + (n-2)(1 - \theta)^{n-3} - \frac{(1 - \theta)^{n-2}}{2} - (n - 2)\theta\frac{(1 - \theta)^{n-3}}{2} d\theta \\ &= c + \int_0^1 \frac{(1 - \theta)^{n - 2}}{\theta} d\theta, \end{align}

where $$c < \infty$$. By the induction hypothesis, the integral on the last line is $$\infty$$. Therefore, the distribution is also improper for $$n$$.