Convergence in proba and weakly

Here is a proof that convergence in proba implies convergence weakly, without having to deal with «$ε$-distance sets» manually. I don’t claim novelty; this is about writing stuff down as a way to understand it, and figure out the relation between the pieces.

Let’s first factor out a few too many simple lemmas:

Lemma 1

If $D_n$ is a sequence of random variables (almost surely) in $[0,K]$, then the following are equivalent:

1. $𝐄 D_n → 0$;
2. For all $ε>0$, $𝐏(D_n > ε) → 0$.

The proof uses the Markov inequality for one direction, and splitting the expectation for the other (OK, that’s where the «$ε$-distance sets» are hiding):

Proof of lemma 1

Assume first that $𝐄 D_n → 0$ and fix $ε>0$. We have $$𝐏(D_n > ε) ≤ 𝐄 D_n / ε → 0,$$ by assumption, hence $𝐏(D_n > ε) → 0$.

Conversely, assume that for all $ε>0$, $𝐏(D_n > ε) → 0$. Then, for any $ε>0$ $$\begin{aligned} & 𝐄 D_n \\ = \quad & 𝐄(D_n·𝟙_{D_n>ε}) + 𝐄(D_n·1_{D_n≤ε}) \\ ≤ \quad & K𝐏(D_n>ε) + ε \\ → \quad & ε \end{aligned}$$ by assumption that $𝐏(D_n>ε) → 0$. Since this holds for any $ε>0$, we conclude $$𝐄 D_n → 0.$$

Lemma 2

If $D_n$ is a sequence of random variables (almost surely) $≥0$, then, for any $K>0$ the following are equivalent:

1. For all $ε>0$, $𝐏(D_n > ε) → 0$;
2. For all $ε>0$, $𝐏(D_n∧K > ε) → 0$.¹

Proof of lemma 2

Clearly (1)$⇒$(2). Conversely, assuming (2) holds, $𝐏(D_n > ε) = 𝐏(D_n∧K > ε) → 0$ for $ε < K$, while for $ε'≥K$, $𝐏(D_n > ε') ≤ 𝐏(D_n > ε) → 0$ with $ε<K$ chosen arbitrarily.

In other words, the convergence $𝐏(D_n > ε) → 0$ is monotonous in $ε$: if it holds for $ε$, it holds for any $ε'>ε$.
This implies we only need to check it for $ε<K$, for which it doesn’t matter that we truncate with $K$.

Lemma 3

If $X_n, X$ are random variables (in some metric space), then the following are equivalent (for any $K$):

1. $X_n →_p X$;
2. $𝐄(d(X_n,X)∧K) → 0$.

This is part of (Kallenberg 2021, lemma 5.2 p102).

Proof of lemma 3

We have the following sequence of equivalent statements: $$\begin{aligned} & X_n →_p X \\ ⇔ \quad & ∀ ε>0,\ 𝐏(d(X_n,X) > ε) → 0 \quad & \text{(def)}\\ ⇔ \quad & ∀ ε>0,\ 𝐏(d(X_n,X)∧K > ε) → 0 \quad & \text{(lemma 2)}\\ ⇔ \quad & 𝐄(d(X_n,X)∧K) → 0 \quad & \text{(lemma 1)} \end{aligned}$$

A simple consequence is

Lemma 4

If $d(X_n, X) ≤ K$ for all $n$, then $X_n →_p X$ iff $𝐄 d(X_n,X) → 0$.

Proof of lemma 4

(omitted)

We can now prove that convergence in proba implies convergence in distribution.

Proof that $→_p ⇒ →_w$

By the Portmanteau theorem, recall that $X_n →_w X$ iff for any countinuous bounded function $f$, we have $𝐄 f(X_n) → 𝐄 f(X)$. So, fix any such $f$. By the continuous mapping theorem, since $X_n →_p X$, we also have $f(X_n) →_p f(X)$; furthermore $f(X_n), f(X)$ are bounded, say by $K/2$. Then $$\begin{aligned} & |𝐄f(X_n) - 𝐄f(X)| \\ ≤ \quad & 𝐄|f(X_n)-f(X)| \quad & \text{(triangle ineq)}\\ → \quad & 0 \quad & \text{(lemma 4)}, \end{aligned}$$ as needed.

Here is the morale I get out of this story: For bounded random variables (in $ℝ$), convergence in probability is convergence to zero of their expected distance (lemma 4). Convergence weakly is convergence of the expected values, under any bounded continuous function. Since those preserve convergence in probability, and force boundedness, we can use the aforementioned characterization of convergence in proba (and the triangle inequality).

Admittedly, with all the extracted lemmas above taken together, this makes for a long proof of $→_p⇒→_w$, but lemma 3 can be considered known, and that’s really all that’s needed for the proof.

Addendum

Actually, after reading this math.se question, I think lemma 1 can be adapted to cover $D_n$s such that $0 ≤ D_n ≤ Y$ with $𝐄 Y < ∞$ (the proof goes through by simply replacing $K$ by $𝐄 Y$). This means that for non-negative dominated sequences $D_n$, $L¹$ convergence is equivalent to convergence in proba, in both cases to $0$. Then, given $X_n,X$, with $|X_n - X|$ dominated by $Y≥0$ with $𝐄 Y^p < ∞$, we can apply the adapted lemma 1 to $D_n = |X_n - X|^p ≤ Y^p$, so that $$\begin{aligned} & X_n →_{L^p} X \\ ⇔ \quad & 𝐄 D_n → 0 \quad &\text{(def)} \\ ⇔ \quad & D_n →_p 0 \quad &\text{(lemma 1, adapted)} \\ ⇔ \quad & (D_n)^{p⁻¹} →_p 0 \quad &\text{(cont' map' thm)} \\ ⇔ \quad & |X_n - X| →_p 0 \quad &\text{(def)} \\ ⇔ \quad & X_n →_p X \quad &\text{(def)}. \end{aligned}$$ This is pretty neat.

References