# Wasserman's AoS, Chapter 8, Question 5

We solve question 5 from chapter 8 os Wasserman’s “All of Statistics” making the implicit assumption that $$X_1, \dotsc, X_n$$ are iid. Other computation solutions can be found in the corresponding GitHub reepo.

Given $$X_1, \dotsc, X_n$$, the random variable $$X_i^*$$ can take any of those $$\le n$$ values. With the assumption that there are no ties (i.e. there are n distinct values), $$X_i^*$$ has a discrete uniform distribution over $$X_1, \dotsc, X_n$$. (If there are ties, then the distribution is not uniform). We use this fact multiple times to obtain the result.

\begin{align} \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) &= \frac{1}{n} \sum_1^n \mathbb E (X_i^* \vert \dotsc) \\ &= \frac{1}{n} \sum_1^n \sum_1^n X_j \frac{1}{n} \\ &= \frac{1}{n} \sum_1^n \bar X_n \\ &= \bar X_n , \end{align}

where we used The Fact for the 2nd equality.

Moreover,

\begin{align} \mathbb E (\bar X_n^*) &= \mathbb E \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) \\ &= \mathbb E \bar X_n \\ &= \mu , \end{align}

where $$\mu := \mathbb E X_1$$, assuming it exists.

For the conditional variance,

\begin{align} \mathbb V (\bar X_n^* \vert X_1, \dotsc, X_n) &= \frac{1}{n^2} \sum_1^n \mathbb V (X_i^* \vert \dotsc) \\ &= \frac{1}{n^2} \sum_1^n \sum_1^n \frac{(X_j - \bar X_n)^2}{n} \\ &= \frac{1}{n} \sum_1^n \frac{(X_i - \bar X_n)^2}{n} \\ &= \frac{S_n}{n} . \end{align}

This has expectation $$\frac{\sigma^2}{n}$$, where $$\sigma^2 := \mathbb V X_1$$, assuming it exists.

Before calculating the variance, we require one more identity - the variance of the conditional expectation.

\begin{align} \mathbb V \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) &= \mathbb V \bar X_n \\ &= \frac{\sigma^2}{n} . \end{align}

The expression for the variance now follows from Theorem 3.27.

\begin{align} \mathbb V (\bar X_n^*) &= \mathbb V \mathbb E (\bar X_n^* \vert X_1, \dotsc, X_n) + \mathbb E \mathbb V (\bar X_n^* \vert X_1, \dotsc, X_n) \\ &= \frac{\sigma^2}{n} + \frac{\sigma^2}{n} \\ &= 2 \frac{\sigma^2}{n} . \end{align}