2

I'm studying probability theory with the concepts of measure theory. Now I'm learning about discrete distribution, however I'm having a hard time trying to find examples of random variable (measurable function) which has discrete distribution.

So my (wiki like) question is: what are random variables with discrete distributions that often appear when solving common probability problems (those that appear in exams, for example)?


I was only able to find the following:

Let $(\Omega,\Sigma,\mathbb{P} )$ be a probability space.

Given any $m,n\in\mathbb{N}$ with $m\leq n$, denote $\color{red}{[m:n]}:=\{k\in\mathbb{N}:m\leq k\leq n\}$.

  1. If $\mathbf{1}_B:\Omega\to \mathbb{R}$ is the indicator function of $B\in \Sigma$, then $\mathbf{1}_B$ has binomial distribution $\mathcal{B}(1,\mathbb{P}(B))$
  2. Suppose that $\Omega :=[1:6],\Sigma :=2^\Omega$, and $\mathbb{P}(A):=|A|/|\Omega |$. Define $X:\Omega \to \mathbb{R}$ by $X(\omega ):=\omega$. Then $X$ has uniform distribution
  3. Let $M,N,n\in\mathbb{N}^\times$ with $M\leq N$. Suppose that $\Omega$ is the set of all maps $f:[1:n]\to [1:N]$, $\Sigma :=2^\Omega $ and $\mathbb{P}(A):=|A|/|\Omega |$. Define $X:\Omega \to \mathbb{R}$ by $X(f):=|f^{-1}(\{M\})|$. Then $X$ has binomial distribution $\mathcal{B}(n,M/N)$. This is a model of the following problem: an urn contains $N$ balls among which $M$ are black. One draws $n$ balls with replacement. What is the probability that $k$ among these $n$ balls are black (find the probability $\mathbb{P}(X=k)$)?
  4. Let $M,N,n\in\mathbb{N}^\times$ with $M\leq N$. Suppose that $\Omega =\big\{A\subseteq [1:N]:|A|= n\big\}$, $\Sigma :=2^\Omega $ and $\mathbb{P}(A):=|A|/|\Omega |$. Define $X:\Omega \to \mathbb{R}$ by $X(\omega ):=|\{n\in \omega :n\leq M\}|$. Then $X$ has hypergeometric distribution $\mathcal{H}(N,M,n)$. This is a model of the following problem: an urn contains $N$ balls among which $M$ are black. One draws $n$ balls without replacement. What is the probability that $k$ among these $n$ balls are black (find the probability $\mathbb{P}(X=k)$)?

EDIT:

Please, give me examples of a probability space $(\Omega,\Sigma,\mathbb{P} )$ together with a random variable $X:\Omega\to\mathbb{R}$ such that $X$ has a discrete distribution. I'm only interested in the following discrete distributions:

  • Uniform distribution
  • Binomial distribution
  • Poisson distribution
  • Hypergeometric distribution
  • Geometric and Negative Binomial Distribution

Thank for your attention!


EDIT 2:

Please, avoid the notion of independent r.v. in the examples!

rfloc
  • 1,501
  • Any “categorical” variable in ML will have discrete distribution. States of discrete Markov chains (both discrete and continuous time) support discrete distributions. Special cases are random walks and queues: like what’s the probability that you have $n$ customers waiting for your reply while you’re busy answering the current one? – SBF Feb 04 '23 at 16:13

1 Answers1

3

While one may come up with ad-hoc probability spaces on which then discretely-valued random variables are defined, in modelling practice one does not concern themselves with $(\Omega,\mathscr{F},P)$ (the abstract measure space) but rather than with the law of the random variable $X:\Omega\to \mathbb{R}$ (or $\mathbb{R}^n$), that is the pushforward probability measure $P_X:\mathscr{B}(\mathbb{R})\to [0,1]$ defined with $P_X(B)=P(X^{-1}(B))=P(\{X\in B\})$. Therefore, we focus on the measure space $(\mathbb{R},\mathscr{B}(\mathbb{R}), P_X)$ and model $P_X$ directly, and this is the standard practice when probability is applied to common probability problems. This is because, given a law $P_X$, we know we can associate it to a random variable on some probability space (see also here). Note further that we can then also ensure the existence of IID sequences of random variables.

There is a good list on Wikipedia of discrete probability distributions, and some of these arise naturally in probability problems and other discrete (and continuous) distributions are closely related to them through the construction of random variables. Two examples are:

Bernoulli distribution: a Bernoulli IID sequence $(X_n)_{n \in \mathbb{N}}$ with $P_{X_1}(\{1\})=P(X_1=1)=p\in (0,1)$ models classic problems such as coin tosses. Another two notable discrete distributions arise naturally from this one. If you introduce the stopping time $\tau(\omega)=\inf\{n:X_n(\omega)=1\}$ (the time of first success) we get $$P(\tau>n)=P(\cap_{k\leq n}\{X_k=0\})=(1-p)^n\implies P(\tau=n)=(1-p)^{n-1}p$$ The distribution of $\tau$ is called the geometric distribution. If instead you consider only a subset of the IID sequence $(X_\ell)_{\ell\leq n}$ for a known $n$ and want to consider the probability observing some $0\leq \ell \leq n$ events of $1$ in the sequence (which we may call 'success'), you get $$P(\ell \textrm{ successes in }n\textrm{ trials})=P\bigg(\sum_{k\leq n}\mathbf{1}_{\{1\}}(X_k)=\ell\bigg)=\binom{n}{\ell}p^\ell (1-p)^{n-\ell}$$ where $\binom{n}{\ell}$ is the number of ways you can arrange $\ell$ ones and $n-\ell$ zeroes in a finite sequence of length $n$. This is the binomial distribution.

Poisson distribution: this distribution arises naturally in the classic framework of events happening randomly over a continuous time index, but also in more advanced settings such as the Lévy processes, under the form of the Poisson process. Indeed, it can be proved that if a Lévy process moves only with unit jumps then it is a Poisson process, so the Poisson distribution, while artificial looking at first sight, is actually very special. Consider the exponential distribution and an IID sequence $(Y_n)_{n \in \mathbb{N}}$ of exponentially distributed waiting times between events with rate $\lambda$. The sum $T_n(\omega)=Y_1(\omega)+Y_2(\omega)+...+Y_n(\omega)$ indicates the smallest time at which we observed $n$ events, and it is Gamma distributed with shape $n$ and rate $\lambda$. Now if $(N_t(\omega))_{t \geq 0}$ indicates the collection of the random variables indicating the number of the events at the fixed times $t\in [0,\infty)$, we get (recall $\Gamma(n)=(n-1)!$) $$P(N_1< n)=P(T_n> 1)=\frac{\lambda^n}{\Gamma (n)}\int_1^{\infty}t^{n-1}e^{-\lambda t}dt=\frac{\Gamma (n,\lambda)}{(n-1)!}$$ where $\Gamma(x,s)$ is the upper incomplete Gamma function. This then implies $$P(N_1=n)=\frac{e^{-\lambda}\lambda^n}{n!}$$

Snoop
  • 18,347
  • Snoop, thank you for your answer! I already knew about the things you said about the measure $P!_X$ and I haven't studied about independent r.v. yet. Right now I'm only interested in examples like the ones I mentioned. I didn't find, for instance, a good example of random variable with geometric distribution. Thank you for your attention! – rfloc Feb 05 '23 at 01:35
  • @rfloc as a first impression, you may try using a variation of the infinite coin toss model and a function like the stopping time I mentioned – Snoop Feb 05 '23 at 02:26