Proving the Memoryless Property ONLY Applies to Exponential Distributions

Question

\begin{equation} \quad P(X > s + t | X > s) = P(X > t) \quad \text{for all $s, t \geq 0$} \quad \end{equation}

The Memoryless Property (above) of the Exponential Distribution means that if you have already waited $s$ minutes for the bus - the probability that the bus comes in $s+t$ minutes .... is the same as if you only waited $t$ minutes. Unfortunately, the amount of time $s$ you have already waited does not count.

Here is my attempt to show the Memoryless Property applies to the Exponential Distribution using Cumulative Distributions and Conditional Probability Law:

\begin{equation} \text{Exponential PDF:} \quad f(x; \lambda) = \lambda e^{-\lambda x} \end{equation}

\begin{equation} \text{Exponential CDF:} \quad P(X \leq x) = F(x; \lambda) = 1 - e^{-\lambda x} \end{equation}

\begin{equation} \text{1 - CDF:} \quad 1 - P(X \leq x) = P(X > x) = e^{-\lambda x} \end{equation}

Using Law of Conditional Probability:

\begin{equation} P(X > s + t | X > s) = \frac{P(X > s + t \text{ and } X > s)}{P(X > s)} = \frac{P(X > s + t)}{P(X > s)} \end{equation}

\begin{equation} P(X > s + t | X > s) = \frac{e^{-\lambda (s + t)}}{e^{-\lambda s}} = e^{-\lambda t} = P(X > t) \end{equation}

My Question: In my classes, we were told again that the Memoryless Property ONLY applies to the Exponential Distribution - I am curious as to why it ONLY applies to the Exponential Distribution. I am able to show that this applies to the Exponential Distribution (i.e. above) - but how do I show that the Memoryless Property ONLY applies to the Exponential Distribution? (e.g. not other distributions)?

As an isolated example, I tried to verify that the Memoryless Property DOES NOT apply to the Gamma Distribution, but this was quite complicated:

$$ \text{Gamma PDF:} \quad f(x; k, \theta) = \frac{x^{k-1}e^{-x/\theta}}{\theta^k\Gamma(k)}$$ $$ \text{Gamma CDF:} \quad F(x; k, \theta) = \frac{1}{\Gamma(k)}\gamma(k, x/\theta)$$

If I try to use these definitions within the Memoryless Property:

$$\frac{\int_{s+t}^{\infty} f(x; k, \theta) dx}{\int_{s}^{\infty} f(x; k, \theta) dx} = \int_{t}^{\infty} f(x; k, \theta) dx$$

$$\frac{\int_{s+t}^{\infty} \frac{x^{k-1}e^{-x/\theta}}{\theta^k\Gamma(k)} dx}{\int_{s}^{\infty} \frac{x^{k-1}e^{-x/\theta}}{\theta^k\Gamma(k)} dx} = \int_{t}^{\infty} \frac{x^{k-1}e^{-x/\theta}}{\theta^k\Gamma(k)} dx$$

I am not sure how to prove that the above relationship is not true in general. I know that for a Gamma Distribution, when $k=1$, the Gamma Distribution becomes an Exponential Distribution - therefore, the above relationship (memoryless property) will be true for when $k=1$. However, I do not know how to prove that that when $k \neq 1$, the above relationship will not be true. And even if I did understand this, I would have to show that the Memoryless Property does not apply to all other distribution (e.g. Weibull, etc.) - and this would involve proving it for the generic distribution.

• Can someone please help me understand why the Memoryless Property does not hold in the case of the Gamma Distribution?
• And more generally, can someone please help me understand that why the Memoryless Property ONLY holds for the Exponential Distribution?

Thanks!

Update: I tried to present a proof of this question over here https://math.stackexchange.com/a/4807952/791334

Answer 1

OP here, this my own attempt to prove this (I am citing a previous answer I wrote https://math.stackexchange.com/a/4807952/791334):

I tried to approach this proof using concepts from Survival Analysis (https://en.wikipedia.org/wiki/Survival_analysis)

Part 1: Typically, we say that the Survival Function is given by $S(t) = P(x>t) = 1 - F(t)$.

In the case of the Exponential Distribution, the Survival Function is given by : $S(t) = P(x>t) = 1 - F(t) = e^{-\lambda t}$

The law of conditional probability tells us that $P(A | B) = \frac{P(A \cap B)}{P(B)}$ . If $A$ and $B$ are independent, then $P(A | B) = \frac{P(A \cap B)}{P(B)} = \frac{P(A) *P(B)}{P(B)}$ . We can apply this law for $P(x>s+t | x>s)$ :

$$ P(X>s+t | x>s) = \frac{P(x>s+t \cap x>s)}{P(x>s)} = \frac{P(x>s+t) }{P(x>s)} = \frac{S(s+t) }{S(s)} $$

Note that $P( x>s)$ is implicitly contained within $P(x>s+t)$. This is because the probability of surviving time $s+t$ implies that a minimum of time $s$ has already been survived.

Part 2:

(A) If the Memoryless Property were to hold, then:

$$P(X>s+t | X>s) = P(X>t)$$

and $$S(s+t | s) = S(t)$$

(B) But in general (whether the Memoryless Property is true or not true, does not matter):

$$P(X>s+t | X>s) = \frac{P(x>s+t \cap x>s)}{P(x>s)} = \frac{P(x>s+t )}{P(x>s)} $$

and

$$S(s+t | s) = \frac{S(s+t \cap s)}{S(s)} = \frac{S(s+t )}{S(s)} $$

Now, equating (A) and (B) :

$$ S(t) = \frac{S(s+t )}{S(s)} $$ $$S(s+t) = S(s)*S(t)$$

Part 3: Using the relationship $S(s+t) = S(s)*S(t)$, we can take the natural logarithm on both sides:

$$ln[S(s+t)] = ln[S(s)*S(t)] = ln[S(s)] + ln[S(t)] $$

If we now take the derivative of both sides with respect to $s$:

$$ \text{LHS:} \quad \frac{d}{ds} \ln[S(s+t)] = \frac{S'(s+t)}{S(s+t)} $$ $$ \text{RHS:} \quad \frac{d}{ds} \ln[S(s)] + \frac{d}{ds} \ln[S(t)] = \frac{S'(s)}{S(s)} $$

Since $ \text{LHS = RHS}$, then:

$$\frac{S'(s+t)}{S(s+t)} = \frac{S'(s)}{S(s)} = c \quad \text{where c is some constant, e.g.,} \quad c = - \lambda$$

Now, we integrate the above equation with respect to $s$:

$$\int \frac{S'(s)}{S(s)} \, ds = \int -\lambda \, ds$$

$$ ln[S(s)] = - \lambda * S$$

$$ S(s) = e^{ -\lambda * s} $$

And $S(s) = e^{ -\lambda * s}$ is the survival function of the Exponential Distribution. This means that only a distribution with a Survival Function of $S(s) = e^{ -\lambda * s}$ would satisfy the Memoryless Property. And I think that only the Exponential Distribution will have a Survival Function of $S(s) = e^{ -\lambda * s}$.

Ideally, this would have concluded the proof.

However, am not sure if this is a valid proof - for example, perhaps some other probability distribution $g(x)$ might somehow share the same survival function ... but I am not sure how to proceed.