Intuition for sampling random variables?

Question

I have recently come to understand random variables from the perspective as deterministic measurable functions $X: \Omega \to \mathbb{R}$. I've been rereading some old statistics text books and realized that in this framework I no longer understand what it means to sample something.

For example, in a recent text, I read something along the lines of "you can sample a geometric random variable with parameter $p$ by flipping a $p$-weighted coin and counting the number of flips until tails is turned". I am not sure how to intuitively interpret this as a random variable in this measure theory framework.

Somewhat similarly, when texts say "sample i.i.d $X_1,\ldots, X_n \sim \mathcal{N}(\mu, \sigma^2)$" what exactly does this mean? By what process do we actually accomplish this? Does this just mean that we explicitly choose a bunch of functions $X_k: \Omega\to \mathbb{R}$ satisfying equality of distribution functions $F_{X_k}(\alpha) = \Phi(\alpha)$ and independence of laws: $\mathcal{P}_{(X_i, X_j)} = \mathcal{P}_{X_i}\times \mathcal{P}_{X_j}$?

Any intuitive clarifications would be really helpful! Huge plus if there's a nice way to formalize these methods for sampling and sampling i.i.d etc.

Answer 1

I'm going to assume two things:

(i) On the interval $(0,1)$, we can define a probability measure that gives each inverval of the form $(a,b)$ with $0<a<b<1$ the measure $b-a$.

(ii) On the product set $(0,1)^n = (0,1) \times \cdots \times (0,1)$ ($n$-dimensional unit box), we can extend the measure from (i) to a new probability measure that gives each rectangle of the form $(a_1,b_1) \times \cdots \times (a_n,b_n)$ with $0<a_i<b_i<1$ measure $(b_1-a_1) \cdots (b_n-a_n)$.

In words, we are assuming that we have a way of measuring subsets of $(0,1)$ and $(0,1)^n$, respectively, that will return the length of an interval and the volume of a box, respectively, when applied to these simple sets.

To generate a single random variable $X$ with the $N(\mu,\sigma^2)$ distribution, you proceed as follows: Define the measurable function $X: (0,1) \to \mathbb{R}$ (on the probability space defined by the measure in (i)) so that $X(u) = F^{-1}(u)$ for any $u \in (0,1)$, where $F$ is the cdf of a $N(\mu,\sigma^2)$ random variable. ($F$ is continuous and strictly increasing, so it is invertible.) To see that $X$ so defined has the correct distribution, note that for any $x \in \mathbb{R}$, we have $$ P(\{X \leq x\}) = P(\{u : F^{-1}(u) \leq x\}) = P(\{u: u \leq F(x)\}) = P([0,F(x)]) = F(x). $$ This is a way to define $X \sim N(\mu,\sigma^2)$ as a measurable function from $(0,1)$ to $\mathbb{R}$.

You can then define $n$ independent copies $X_1,\ldots,X_n$ as follows: First create a function $$ X(u_1,\ldots,u_n) = (F^{-1}(u_1),\ldots,F^{-1}(u_n)) $$ from $(0,1)^n \to \mathbb{R}^n$. Then define the coordinate maps $$ \pi_i(x_1,\ldots,x_n) = x_i $$ from $\mathbb{R}^n \to \mathbb{R}$ for $i=1,\ldots,n$. The composite functions $X_1 = \pi_1 \circ X,\ldots,X_n = \pi_n \circ X$ then give you $n$ random variables that are all defined on the same probability space $(0,1)^n$, with the measure given in (ii). By using the fact that $\{X_i \leq x_i\} = [0,F(x_i)]$ (by the same argument as above), and the fact that the measure of a box is its volume by (ii), you can easily show that $X_1,\ldots,X_n$ are i.i.d. with distribution $N(\mu,\sigma^2)$.