Singular memoryless distribution

Question

Context: A probability distribution is said to be memoryless if its range $R$ is stable under addition, and for all $a,b\in\mathbb R$, $$\mathbb P(X>a+b|X>b)=\mathbb P(X>a)$$ It is well known geometric and exponential distributions are memoryless. Conversely, if $X$ satisfies the memoryless property on $\mathbb R$, then $X$ is exponential. In the discrete case, Roger B. Nelsen proved in his 1987 article Consequences of the Memoryless Property for Random Variables that a discrete memoryless distribution has range $t\mathbb N$ for some $t>0$ and $X/t$ is geometric.

Question: on the Wikipedia entry for "Memorylessness", it reads "Only two kinds of distributions are memoryless: geometric and exponential probability distributions.". Apart from the fact that I would add "dilated" before "geometric" to match with Nelsen's result, is there any reason we don't search for singular memoryless distribution, i.e. atomless and concentrated on a Lesbegue set of measure $0$?

Some ideas: let us denote $R$ the range of such a random variable. $R$ must be of Lebesgue measure $0$, stable under addition and uncountable. Is there any reason such set couln't exist? Moreover, if $0\in R$, then $$\mathbb P(X>0)=\mathbb P(X>0+0|X>0)=1$$ so $R\subset \mathbb R_+$.

Answer 1

Characterization of Memoryless Distributions

Let $X$ be a random variable on $\mathbb{R}$. Suppose there exists a Borel-measurable set $S \subseteq \mathbb{R}$ satisfying the following conditions:

1. Support: $\mathbf{P}(X \in S) = 1$
2. Semigroup Property: For any $a, b \in S$, $a + b \in S$
3. Memorylessness: For any $a, b \in S$ with $\mathbf{P}(X > b) > 0$, $$ \mathbf{P}(X > a) = \mathbf{P}(X > a + b \mid X > b) $$

Define the set of "right-limit" points of $S$ as:

$$ S^* \triangleq \{ \inf(S \cap [a, \infty)) : a \in \mathbb{R} \} $$

Equivalently, $S^*$ is the set of all points $a$ in $\mathbb{R}$ such that there exists a sequence $(a_n) \to a$ in $S \cap [a, \infty)$.

Then, exactly one of the following statements holds:

1. $S^* \subseteq [0, \infty)$, and there exists $\lambda > 0$ such that $$ \mathbf{P}(X > x) = e^{-\lambda x} \quad \forall x \in S^* $$

2. $X$ is a degenerate random variable with a non-negative value, i.e., there exists $c \geq 0$ such that $$ \mathbf{P}(X = c) = 1 $$

Remarks

1. This characterization encompasses not only the exponential and geometric distributions but also other distributions satisfying the memorylessness property. The key constraint on $S$ is that it must be a semigroup of $\mathbb{R}$. While the memorylessness property further restricts the possible choices of $S$, any "right-closed" semigroup $S \subseteq [0, \infty)$ (i.e., $S = S^*$) is admissible. For instance, $S = \{1,\frac{4}{3},2,\frac{8}{3}\}\cup[3,\infty)$ is a valid choice. This result does not contradict the findings in Nelsen (1987), where memorylessness is examined under less stringent conditions.

2. The only continuous (atomless) distributions satisfying the memorylessness property are the exponential distributions. This follows from the fact that any right-closed semigroup $S \subseteq [0, \infty)$ for which $0$ is an accumulation point must be equal to $[0, \infty)$. However, it is possible to construct mixtures of singular and atomic distributions that exhibit the memorylessness property.

---

Proof. For simplicity, we introduce the survival function $R(\cdot)$ defined by

$$ R(x) \triangleq \mathbf{P}(X > x). $$

Note that $R(\cdot)$ is right-continuous. Moreover, the memorylessness property reads in terms of $R(\cdot)$ as:

$$ R(a) = \frac{R(\max\{0, a\} + b)}{R(b)} \qquad \forall a, b \in S \text{ with } R(b) > 0. \tag{1}\label{e:mem0} $$

Now we make a series of observations on $S$ and $R(\cdot)$ that leads to the proof of the main claim:

1. $S \subseteq [0, \infty)$.

   Suppose there exists $a \in S \cap (-\infty, 0)$. Then $R(na) > 0$ for some $n \in \mathbb{Z}_{>0}$, hence $$ R(a) = \frac{R(\max\{0,a\}+na)}{R(na)} = \frac{R(na)}{R(na)} = 1. $$

   Now let $b \in S$ be arbitrary. Then we can choose $m \in \mathbb{Z}_{>0}$ such that $b+ma < a$. Since $R(ma) = R(b+ma) = 1$, it follows that

   $$ R(b) = \frac{R(\max\{0,b\}+ma)}{R(ma)} = 1. $$

   Let $\beta = \sup S$. Then for any $b < \beta$, there exists $b' \in (b, \beta] \cap S$, hence $$ \mathbf{P}(X \leq b) \leq \mathbf{P}(X \leq b') = 1 - R(b') = 0. $$ Letting $b \nearrow \beta$, we get $\mathbf{P}(X < \beta) = 0$. Together with $\mathbf{P}(X \in S) = 1$, this forces that $\mathbf{P}(X = \beta) = 1$, and in particular, $\beta \in S$. However, this contradicts $R(\beta) = 1$. Therefore $S$ cannot contain negative numbers.

The above observation resolves the degenerate case. From this point on, we assume $X$ is nondegenerate.

2. $R(0) = 1$.

   Recall the assumption that $\mathbf{P}(X \neq 0) > 0$. Since $\mathbf{P}(X \geq 0) = 1$, this implies that $R(0) = \mathbf{P}(X > 0) > 0$. Now assume $R(0) < 1$ for the sake of contradiction. Then $\mathbf{P}(X = 0) = 1 - R(0) > 0$, hence $0 \in S$. However, this then implies that $R(0) = \frac{R(0)}{R(0)} = 1$, a contradiction.

Now, we define $h$ by

$$ h \triangleq \inf\{a \in S : R(a) < 1\}. $$

We will see that the value of $h$ plays an important role throughout the proof, and in particular, it determines the distribution of $X$.

3. $R(a) > 0$ for all $a \in \mathbb{R}$, and the following identity holds:

   $$ R(a + b) = R(a)R(b) \qquad \forall a, b \in S^*. \tag{2}\label{e:mem} $$

   Since $R$ is right-continuous, the memorylessness property $\eqref{e:mem0}$ holds for any $a, b \in S^*$ such that $R(b) > 0$, and so, it suffices to prove that $R(a) > 0$ for all $a \in \mathbb{R}$. If $R(h) = 0$, then $h > 0$ and $R(a) = 1$ for all $a \in [0, h)$, meaning $$ \mathbf{P}(X = h) = \lim_{a \nearrow h} (R(a) - R(h)) = 1 $$ This contradicts the nondegeneracy assumption. Hence, $R(h) > 0$ and there exists $a > 0$ such that $R(a) > 0$. (If $h > 0$, simply set $a = h$. Otherwise, $h = 0$ is the right-limit point of $S$ and we can invoke the right-continuity of $R(\cdot)$ to choose such $a$.) Then $R(na) = R(a)^n > 0$ for all $n \in \mathbb{Z}_{>0}$ and hence the desired claim follows.

4. $R(a) < 1$ for all $a \in S^* \cap (0, \infty)$.

   If $a \in S^* \cap (0, \infty)$ and $R(a) = 1$, then by $\eqref{e:mem}$, $R(na) = 1$ for all positive integer $n$, leading to a contradiction.

5. $R(x) = e^{-\lambda x}$ for all $x \in S^*$ for some $\lambda > 0$.

   First consider the case $h = 0$, then for any $\varepsilon > 0$, there exists $a \in S \cap (0, \infty)$ such that $a\mathbb{Z}_{>0} \subseteq S$, and it is easy to check that this implies $S^* = [0, \infty)$. Then this and $\eqref{e:mem}$ together imply that $X$ has an exponential distribution.

   Next, assume $h > 0$. Then $R(h) \in (0, 1)$. Set $$ \lambda = -\frac{1}{h}\log R(h) \in (0, \infty), $$ so that $R(h) = e^{-\lambda h}$. We claim that $R(a) = e^{-\lambda a}$ for any $a \in S^*$. Since this is trivial when $a = 0$, we may assume $a > 0$. Choose a sequence $p_n/q_n$ of rational numbers with $p_n, q_n \in \mathbb{Z}_{>0}$ that approximates $\frac{a}{h}$ from below, i.e., $\frac{p_n}{q_n} \nearrow \frac{a}{h}$. Then $ p_n h \leq q_n a$, and $$ R(a)^{q_n} = R(q_n a) \leq R(p_n h) = R(h)^{p_n} = e^{-\lambda p_n h} $$ Hence $$ R(a) \leq e^{-\lambda p_n h / q_n} \to e^{-\lambda a} $$ Likewise, approximating $\frac{a}{h}$ from above, we get $R(a) \geq e^{-\lambda a}$ and hence the desired claim follows.