1

I previously treated the sum of r.v. as the trivial sum as functions: $$ (X+Y)(\omega):=X(\omega)+Y(\omega).$$ But a confusion arises when I encountered the Strong Large Number thm.:

Let $(X_n)_(n\geq 1)$ be independent and identically distributed (i.i.d.) and defined on the same space. Let $\mu=E{X_i}$ and $\sigma^2=\sigma_{X_j}^2 \lt \infty$. Let $S_n=\sum^n_{j=1} X_j$. Then $\lim_{n \to \infty}\frac{S_n}{n} = \lim_{n \to \infty} \frac{1}{n}\sum_{j=1}^{n}X_j=\mu$ a.s and in $L^2$.

Since it says $X_i$ are i.i.d, they must be defined on the same state sapce. But I think the sum $S_n$ here cannot be viewed as the ordinary sum of functions. For example, let the state space $\Omega$ be $\{H, L\}$ where H means head and L means tail in experiments of tossing coin and $X_i(H) := 1, X_i(T) :=0$ for $i\geq 1$. If we treat the sum as ordinary sum of functions, there will be only two values in the range of $S_n$: $S_n(H) = \sum_{i=1}^{n}X_i(H)=n$ and $S_n(T)=\sum_{i=1}^{n}X_i=0$. Then, there is nothing interesting to study.

So, the proper way to define the sum may be extending the domain of $S_n$ from $\Omega$ to $\prod_{i=1}^{\infty}\Omega$, the sigma algebra from $\mathcal{A}$ to $\otimes_{i=1}^{\infty}\mathcal{A}$ and the measure from $P$ to $\otimes_{i=1}^{\infty}P$. In this case, $S_n(\omega)=S_n((\omega_1,\dots,\omega_n,\omega_{n+1},\dots)):=\sum_{i=1}^{n}X_i(\omega_i)$.

I've seen that the well known properties concerning expectation (i.e. integration) such as $E\{X+Y\}=E\{X\}+E\{Y\}$ also hold by the Fubini thm. from this perspective.

But I failed to find similar contents in all resources accessible to me. So I'm wondering if I'm correct?

WJH20
  • 11
  • 2
    When you want to model outcomes from repeated trials, your sample space must be ample enough to host all the possible sequences of outcomes from the repeated trials. One such choice of sample space is $\Omega={H,L}^{\mathbb{N}}$, and then the $i$th observation is modeled by the $i$th projection: $$X_i(\omega_1,\omega_2,\ldots)=\omega_i$$ – Sangchul Lee Oct 13 '22 at 04:58
  • 1
    See this answer and in more generality the Ionescu-Tulcea theorem. You're right, and check out Klenke's book for the proof of the I-T theorem. – Sarvesh Ravichandran Iyer Oct 13 '22 at 05:04
  • Thanks. I think i've fixed it essencially. But a new tiny question occurs, in the above definiton of $S_n$, i skipped define the corresponding r.v. defined on $\prod_{i=1}^\infty \Omega$ derived from $X$. But I think another way is to extend $X_i$ first:$$\tilde{X}i(\omega):=X_i(\omega_i)$$and$$S_n:=\sum{i=1}^\infty \tilde{X}_i.$$ Are there any difference between them? – WJH20 Oct 14 '22 at 00:36

1 Answers1

2

You are absolutely right about this. At the beginning of probability theory, we have the universe $\Omega$ which encodes all possible information about the universe. Each random variable (even if known) can be considered as partial information on the space. But in practice, the universe and/or the $\sigma$-algebra may potentially change, as in the situation of studying martingales.

I was confused about this too, in particular when it comes to the strong law of large numbers. What's the definition of probability for event like $\frac{1}{n}\sum_{i=1}^n X_i\rightarrow 0$?

In the case of i.i.d. random variables $\{X_i\}_{i=1}^\infty$ on the same space $\Omega$, a point in the universe should tell us everything we need to know about the variables. This universe should be exactly $\prod_{i=1}^\infty \Omega$. And there is a standard procedure to introduce a $\sigma$-algebra and probability measure on this space such that it's compatible with the measurability and probabilities of $X_i$'s.

Now $\sum_{i=1}^n X_i$ should be understood as a function on the universe $\prod_{i=1}^\infty\Omega$. And the event $\frac{1}{n}\sum_{i=1}^n X_i\rightarrow 0$ is a measurable subset of $\prod_{i=1}^\infty\Omega$ which has a well-defined probability.

More details and references can be found in answers for this question. The key to all this is Kolmogorov's extension theorem.

Just a user
  • 22,048
  • Thanks. I think i've fixed it essencially. But a new tiny question occurs, in the above definiton of $S_n$, i skipped define the corresponding r.v. defined on $\prod_{i=1}^\infty \Omega$ derived from $X$. But I think another way is to extend $X_i$ first:$$\tilde{X}i(\omega):=X_i(\omega_i)$$and$$S_n:=\sum{i=1}^\infty \tilde{X}_i.$$ Are there any difference between them? – WJH20 Oct 14 '22 at 00:40
  • I don't think so. But if $\prod_{i=1}^\infty$ is not formed yet, where does $\omega$ live? – Just a user Oct 15 '22 at 01:42