5

I am trying to show that if $Z_1, Z_2, \ldots, Z_n\sim\mathcal{N}(0, \sigma^2)$ are i.i.d, then $\mathbb{E}[\max(|Z_1|, |Z_2|, \ldots, |Z_n|)]\leq \sigma\sqrt{2\log (2n)}$. I tried to use the tail bound:

$$\mathbb{E}[\max(|Z_1|, |Z_2|, \ldots, |Z_n|)]=\int_0^{\infty}\mathbb{P}(\max(|Z_1|, |Z_2|, \ldots, |Z_n|)>t)dt$$ However, we see that $$\mathbb{P}(\max(|Z_1|, |Z_2|, \ldots, |Z_n|)>t) = \mathbb{P}\left(\bigcup_{i=1}^n\{|Z_i|>t \}\right)\leq \sum_{i=1}^n\mathbb{P}(|Z_i|>t)\leq 2ne^{-t^2/2\sigma^2}$$ so $$\mathbb{E}[\max(|Z_1|, |Z_2|, \ldots, |Z_n|)]\leq 2n\int_0^{\infty} e^{-t^2/2\sigma^2}dt = 2n\sigma\sqrt{2/\pi}$$ which is of course, not the bound I want. Is there any way I can change some of these steps differently to get the bound I want?

(Edit: I already looked at this question Expectation of the maximum of gaussian random variables, but the top answer suggested you could also approach this by a union bound and tail bound approach rather than the very nonintuitive exponential approach, so that was what I was trying to emulate.)

Amir
  • 11,124
Aadi Rane
  • 347

1 Answers1

2

Given that $Z_1,\ldots, Z_n\sim\mathcal{N}(0, \sigma^2)$ are i.i.d,

$$\color{blue}{\mathbb{E}[\max(|Z_1|, \ldots, |Z_n|)]\leq \sigma\sqrt{2\log (2n)}.} \tag{1} $$

Note that

$$\max(|Z_1|, \ldots, |Z_n|) = \max(Z_1, \ldots, Z_{n},-Z_1, \ldots, -Z_{n}). \tag{2}$$

We can show that for $Y_1, \ldots, Y_m\sim\mathcal{N}(0, \sigma^2)$, which can be dependent, the following holds:

$$\color{blue}{\mathbb{E}[\max(Y_1, \ldots, Y_m)]\leq \sigma\sqrt{2\log (m)}} \tag{3}.$$

Then, (1) follows from (3) using (2).

To prove (3), by Jensen inequality, for any non-zero $t \in \mathbb R$, we have:

$$\mathbb{E}[\max(Y_1, \ldots, Y_m)]=\\ \frac{1}{t} \mathbb{E}\big[\log \exp(t\max(Y_1, \ldots, Y_m)\big] \\ \le \frac{1}{t} \log \mathbb{E}\big[ \exp(t\max(Y_1, \ldots, Y_m)\big] \tag{4}.$$

Moreover,

$$\mathbb{E}\big[ \exp(t\max(Y_1, \ldots, Y_m)\big]=\\ \mathbb{E}\big[ \max(\exp(tY_1), \ldots, \exp(tY_m))\big] \\ \le \sum_{i=1}^m \mathbb{E}[ \exp(tY_i)]=m e^{\frac{1}{2}t^2\sigma^2}. $$

From (4), this gives the following bound:

$$\mathbb{E}[\max(Y_1, \ldots, Y_m)] \le \frac{1}{t} \log \left ( m e^{\frac{1}{2}t^2\sigma^2} \right ),$$

which is minimized for $$t=\sqrt{\frac{2\log m}{\sigma^2}},$$

establishing the bound in (3).

Amir
  • 11,124