1

Suppose that there have been $n$ days and that the sun has risen on all of them. What’s the chance that the sun will rise tomorrow? Assuming that we start with a uniform prior on the probability that the sun will rise, then the chance [edit: this should say 'expected value of the probability'] is $$ \frac{n+1}{n + 2}$$ (See here for the details and relevant background assumptions.)

So if there have been $10$ days, the chance that the sun will rise tomorrow is $\frac{11}{12}$. Moreover, the chance that the sun will rise both tomorrow and the day after is $\frac{11}{12} \times \frac{12}{13} = \frac{11}{13}$.

Now let’s re-describe events: there have been $5$ double-days (a double-day is two normal days) and the sun has double-risen 5 times (the sun double-rises if it rises on two consecutive days). What’s the chance that the sun will double-rise again? Using the formula from before, it is $\frac67$. But $\frac67 \neq \frac{11}{13}$! What's going on here?

Itzhak Rasooly
  • 37
  • I think you need a fixed pairing of days. If you just say "the sun double-rises if it rises on two consecutive days", then the sun has double-risen $9$ times. If you group, say, each odd-numbered day of the month with the following even-numbered day, it has only double-risen $5$ times. – joriki Apr 29 '20 at 18:38
  • Nice point! Yes the example could have been set up better – Itzhak Rasooly Apr 30 '20 at 10:52

2 Answers2

2

A model of sunrises in which the sun rises each day with probability $p$ gives us a model of double-sunrises in which the sun double-rises on each double-day with probability $p^2$. So a uniform prior over $p$ in the first model corresponds to a non-uniform prior in the second model, but you've used a uniform prior in your calculation.

Karl
  • 12,752
2

This is not really all that suprising, as a flat prior for the probability of the sun rising on a certain day is inconsistent with a flat prior for the probability of the sun double-rising on a certain double-day, given that the latter is given in terms of the former by $p_2=p_1^2$, so $f_{p_1}(p_1)=1$ implies $f_{p_2}({p_2})=\frac1{2\sqrt{p_2}}$.

What this shows is not so much a contradiction but the arbitrariness in choosing a flat prior for a variable in some representation when we could just as well choose a flat prior in a transformed representation. The general idea is that the differences are typically small (as in this case) for reasonable choices of variables and are quickly drowned out by the data, so they don’t matter that much in practical applications.

In the present case, the entire premise of a flat prior for something that seems to be governed by laws of nature is in any case questionable. Based on our overall experience of the law-like nature of much of reality, a realistic prior should emphasize probabilities near $0$ and $1$.

joriki
  • 242,601
  • Thanks for the illuminating answer! In defence of the flat prior assumption, I guess it could be said that the situation is meant to start before we have acquired any experience of reality, which makes it seem a bit more reasonable. Also, I am not sure that "law like" implies probabilities near 0 or 1: see, e.g., quantum mechanics – Itzhak Rasooly Apr 30 '20 at 10:56
  • @ItzhakRasooly: You're welcome. Next time I'll try to write a double-illuminating answer ;-) Good point about quantum mechanics – but there, too, there are lots of laws that are essentially $0/1$. E.g., if I observe an atom of a radioactive isotope (a paradigmatically unpredictable quantum system) for $1$ second and I observe whether it decays, and then I want to estimate the probability that another atom of the same isotope will decay in the next second, I should put a lot of weight in the prior near $0$ and $1$, since the chance that the half-life is of the order of $1$ second is quite low. – joriki Apr 30 '20 at 11:08
  • One more question if I may (and a chance to double illuminate!) In the question, I wrote our estimate of the chance that the sun rises twice is $11/12 \times 12/13$. This makes sense to me: when considering whether the sun will rise again, we should condition on the fact that the sun has already risen. However, my friend says that it should be $(11/12)^2$ since events are conditionally independent in the Bernouilli model (and I take your suggestion that $p_2 = p_1^2$ to amount to the same point). Perhaps you can adjudicate amongst us? N.B. See later edit in the Q which may clarify this – Itzhak Rasooly Apr 30 '20 at 19:17
  • @ItzhakRasooly: Since my first answer turned out to be illuminating, I now have a $\frac{1+1}{1+2}=\frac23$ chance of double-illuminating (assuming a flat prior for my illumination probability). My response will be a bit long for a comment; I'll edit it into the answer. – joriki Apr 30 '20 at 19:32
  • I think $\frac{11}{12}\frac{12}{13}$ is right. The Bernoulli model says that the events are conditionally independent given $p$, not that they're marginally conditionally independent. If they were, Bayesian inference would be useless. – Karl Apr 30 '20 at 21:48
  • (oops, "marginally conditionally independent" should be "marginally independent") – Karl May 01 '20 at 00:07
  • @Karl thanks for the clarification! – Itzhak Rasooly May 02 '20 at 10:18
  • 1
    @ItzhakRasooly: Actually, I just realized that I answered exactly that question just recently: https://math.stackexchange.com/questions/3640533. So you're right and your friend is wrong (and Karl is right). This has nothing to do with $p_2=p_1^2$, which is a statement about the (unknown) actual probabilities $p_1$ and $p_2$, not about our prior or posterior distributions for them. – joriki May 02 '20 at 10:42