24

let $P,Q$ be two probability measures on $(\Omega, \mathscr {F})$, and let $\nu$ be a $\sigma$-finite measure on the same event space such that $P \ll v, Q \ll v$. Define $\frac{dP}{dv}=p$, $\frac{dQ}{dv}=q$.

The total variation distance between P and Q is then: $$ V(P,Q) = \sup_{A \in \mathscr{F}}|P(A) - Q(A)|= \sup_A \bigg| \int_A(p-q )d\nu \bigg| $$ I'm confused about the following, we may write: $$ V(P,Q) = \frac{1}{2} \int |p-q|d\nu $$ First, how can we bring the absolute value inside the integral and get rid of the supremum, and second, what region are we integrating over now? This statement is made frequently in the book: Introduction to nonparametric Estimation - Tsybakov

D.R.
  • 10,556
WeakLearner
  • 6,350
  • somewhat related: https://math.stackexchange.com/questions/3287889/show-that-the-total-variation-distance-of-probability-measures-mu-nu-is-equa?rq=1 – D.R. Nov 13 '19 at 04:57

1 Answers1

29

Let $B = \{p \ge q\}$. Note that \begin{align*} \int_\Omega \def\abs#1{\left|#1\right|}\abs{p-q}\, d\nu &= \int_B (p - q) \, d\nu + \int_{\Omega \setminus B} (q- p)\, d\nu\\ &\le 2 \sup_A \abs{\int_A (p-q) \, d\nu} \end{align*} On the other side, note first that $$ \int_\Omega (p-q) \,d\nu = P(\Omega) - Q(\Omega) = 0 $$ and hence $$ \int_B (p-q) \, d\nu = \int_{\Omega \setminus B} (q-p) \, d\nu $$ Now for any $A \in \mathscr F$, we have \begin{align*} \abs{\int_A (p-q)\, d\nu} &= \max\left\{\int_A (p-q)\, d\nu, \int_A (q-p)\, d\nu\right\}\\ &\le\max\left\{ \int_{A\cap B} (p-q)\, d\nu, \int_{A \cap (\Omega \setminus B)} (q-p)\, d\nu\right\}\\ &\le \max\left\{ \int_{B} (p-q)\, d\nu, \int_{\Omega \setminus B} (q-p)\, d\nu\right\}\\ &= \int_B (p-q)\, d\nu\\ &= \frac 12 \int_\Omega \abs{p-q}\,d\nu \end{align*} Taking the supremum over $A \in \mathscr F$, gives $$ \sup_A \abs{\int_A (p-q)\, d\nu} \le \frac 12 \int_\Omega \abs{p-q}\, d\nu $$ which is the other needed inequality.

martini
  • 86,011
  • 1
    why does $ \int_B (p-q), d\nu = \frac 12 \int_\Omega \abs{p-q},d\nu$ ? – Monolite Jul 10 '18 at 01:23
  • 2
    This is because $$\int_B (p-q) d\nu + \int_{\Omega\backslash B } (p-q) d\nu = \int_\Omega(p-q) d\nu= 0$$, thus $\int_B (p-q) d\nu = - \int_{\Omega\backslash B } (p-q) d\nu$. However $p\geq q$ on $B$ and $p<q$ on the complement of $B$, meaning that $$\int_B |p-q| d\nu=-\int_{\Omega\backslash B } (p-q) d\nu = \int_{\Omega\backslash B } |p-q| d\nu$$ The leftmost and rightmost integrals are the same and sum to the integral $I=\int_\Omega |p-q|d\nu$, so they must be half of $I$. – wilsonw Aug 05 '20 at 10:52