Bounding with exponential Markov inequality

Question

I have a question applying Markov inequality for the following question:

Let $(X_{n})$, $n∈N$ be a sequence of i.i.d. real-valued random variables with $μ := E[X_{1}]$, and $φ(λ) := \text{log}E[e^{λX_{1}}]$ well-defined for all $λ ∈ R$. Let Sn be the sum of $X_{n}$ up to $n$th term. Use exponential Markov inequality to show that for any $a > μ$, there exists $J(a) > 0$ such that $P(S_{n}/n ≥ a) ≤ e^{−nJ(a)}$ for every natural number n.

I attempted to do the following: $P(S_{n}/n \geq a) = P(tS_{n} \geq ta)$ for any real t, and $P(tS_{n} \geq ta) \leq e^{-nat}e^{tS_{n}} = e^{-n(at-\phi(t))}$, and tried to show that $(at-\phi(t))$ is always positive, but I am stuck at showing that for any choice of a, $(at - \phi(t))$ should be positive. I tried to use Taylor expansion but still cannot show that strict ineqality. Could anyone help with this problem? Thanks!

Answer 1

Minor typo: I think you mean $P(tS_n \ge nta)$ instead of $P(tS_n \ge ta)$.

It is not always true that $at - \phi(t)$ is positive. Take for example $X_i$ being standard Gaussian; then $\phi(t)=t^2/2$, so $at-t^2/2$ is not always positive.

However, by choosing $t$ to maximize $at-\phi(t)$, you have actually shown a bound of $$P(S_n/n \ge a) \le e^{-n \sup_{t \in \mathbb{R}}(at - \phi(t))},$$ which is Cramér's Theorem.

The function $\phi^*(a) = \sup_{t \in \mathbb{R}}(at - \phi(t))$ is the convex conjugate of $\phi$, and in the context of large deviations theory is called the "rate function."

• Taking $t = 0$ shows that $\phi^*(a) \ge 0$ for any $a$.
• For each $t$, the mapping $a \mapsto at-\phi(t)$ is linear. Thus $\phi^*$ is convex since it is the maximum of convex functions
• We know $\phi(t) = \log E[e^{t X}] \ge t \mu$ by Jensen's inequality, so $t \mu - \phi(t) \le 0$ for all $t$. Taking the supremum over $t$ yields $\phi^*(\mu) \le 0$, but since $\phi^*$ is nonnegative, we therefore have $\phi^*(\mu) = 0$.

The above three facts show that $\phi^*$ is convex with minimum value $\phi^*(\mu)=0$ at $\mu$. Since $\phi^*(a)$ is nondecreasing to the right of $\mu$, we must have $\phi^*(a) \ge 0$ for $a > \mu$.

I think one can also claim that $\phi^*$ is strictly convex in "most" situations, which would give you the strict inequality $\phi^*(a) > 0$ for $a > \mu$. Theorem 11.13 in "Variational Analysis" by Rockafellar et al. suggests that differentiability of $\phi$ is sufficient for this, but this is beyond my expertise.