1

Usually when I am trying to control some quantity, I try to apply the inequalities that I know in hopes of obtaining the bound I am looking for. I know to not always apply inequalities directly, but instead to try to apply them near where they are tightest, to get the best bound.

For example, in a proof of the cauchy-schwarz inequality, starting from the bound:

$$\langle a- b, a- b \rangle \geq 0\quad (1)$$

naively expanding gives us:

$$\frac{1}{2} \left (\|a\|^2+\|b\|^2\right ) \geq \langle a, b \rangle$$

which is not as tight as the cauchy-schwarz inequality (by AM-GM). However, note that $(1)$ is tight when $a = b$, so instead applying it with normalized $a/\|a\|$ and $b/\|b\|$ gives us a tighter bound, which turns out to be the inequality we are looking for.

my question

Often, I struggle with a proof of an inequality because I did not think to apply a bound which I expected to be loose (as in $(1)$ above). What are some tips/tricks for knowing when an inequality is good enough, before I even try it?

A specific example is in the application of union bounds. I usually hesitate to try a union bound since I expect it to be coarse.

Another example is the bound $1+x \leq e^x$ which is very coarse for large $x$.

EDIT:

To give a concrete example using a union bound:

Given a collection of zero mean subgaussian random variables, we have the following tail bound:

$$\mathbb P(X \geq t) \leq e^{-t^2/2\sigma^2}$$

Using this and a union bound we can show the bound on the deviation of the maximum:

$$\mathbb P(\max_i^n |X_i| \geq t) \leq 2ne^{-t^2/2\sigma^2}$$

At first glance it's not clear to me if this is a good bound or not. It seems to be quite coarse due to the leading $n$ factor. But it does allow us to show the following useful bound on the expectation:

Which implies the bound on the expectation: $$\mathbb E |X| \leq \sqrt{2\sigma^2 \log 2n}+ 1/\left(2\sqrt{2\sigma^2 \log 2n}\right)$$

While this isn't a tight bound (it is possible to show that the second term is not needed, it still gives the correct asymptotic rate.

dmh
  • 3,082
  • 1
    Sometimes introducing a parameter is profitable. In your example, let $t > 0$ and replace $a$ with $ta$ and replace $b$ with $t^{-1}b$. Now minimize the lhs over $t > 0$, and you will get the desired inequality. – Mason Oct 12 '22 at 15:18
  • A followup question would be: often times I cannot find an optimal value for $t$ easily. How do I get an idea if a given value is "good enough" before simplifying everything? See the proof here which introduces a constant $c$ and chooses a seemingly arbitrary value for it. – dmh Oct 12 '22 at 15:29
  • Use calculus to minimize. Set $d/dc$ of the bound equal to $0$. In that example you need to assume $c \geq 1$ first and replace $e^{-x^2/2}$ by $xe^{-x^2/2}$ so that you can evaluate the integral. Only then did he optimize over $c$. – Mason Oct 12 '22 at 15:49
  • I actually did not try to find an optimum using lagrange multipliers, but I expect a closed form to not be easy to find. – dmh Oct 12 '22 at 15:55
  • He proved that for $c \geq 1$, $$E(Z_n) \leq c + \frac{n}{c}\sqrt{\frac{2}{\pi}}e^{-c^2/2}.$$ Thus $$E(Z_n) \leq \inf_{c \geq 1}(c + \frac{n}{c}\sqrt{\frac{2}{\pi}}e^{-c^2/2}).$$ Closed form of the inf might involve the Lambert $W$ function. In that problem though, since the answer is already known, the choice of $c$ is obvious. – Mason Oct 12 '22 at 20:04

1 Answers1

2

Similar to the idea you mentioned, knowing where an inequality is tight can help. For example, when showing that the euclidean norm satisfies the triangle inequality one might arrive at:

$$\|a+b\| = \sqrt{\|a\|\cdot\|b\|\left ( \frac{\|a\|}{\|b\|} + \frac{\|b\|}{\|a\|} + 2\right )}$$

It might be tempting to apply AM-GM here to the two quantities under the square root. However, AM-GM is tight when the two quantities are equal, but the triangle inequality is tight simply when $a,b$ are in the same direction. This is a hint that applying AM-GM directly here is not good enough.

Instead recall that cauchy-schwarz is tight when $a,b$ are in the same direction. Then write the ratio:

$$\frac{\|a+b\|}{\|a\|+\|b\|} = \sqrt{\frac{\|a\|^2+\|b\|^2+2\langle a, b\rangle} {\|a\|^2+\|b\|^2 + 2\|a\|\cdot\|b\|}}$$

where now an application of cauchy schwartz to give an upper bound is clear.

dmh
  • 3,082