0

"Uncertainty", "unpredictability", "randomness", "information content" have all been used to describe what differential entropy measures. The first two definitions rein in unrequited comparisons with variance and volatility. To me, "randomness" resonates the best, having a stats background, but I struggle to convey the meaning of the last one especially.

If entropy measures information content of a random variable's statistical distribution, how can "information content" be put in layman's terms for a different scientific discipline that doesn't understand bits or any computer science jargon, nor deals with messengers and receivers, given that a low entropy variable is said to be "more informative" than a high entropy variable? maybe what I'm really looking for is a synonym for usage in place of "information"

develarist
  • 1,584
  • I am increasingly of the opinion that it's actually quite misleading to think of entropy in terms of "information content." There's an important sense in which random noise, which has the highest possible entropy, contains no information whatsoever! – Qiaochu Yuan Sep 27 '20 at 07:45
  • yes, "content" suggests the polar towards the maximum rather than the minimum, counterintuitive to the negative relationship between entropy's magnitude and "informativeness". how can the term be inverted? "information [antonym of content here]". but that would only correct the polar direction, and we're still stuck with explaining what information even means – develarist Sep 27 '20 at 07:49
  • How about "allows the greatest variety of expression"? – kimchi lover Sep 27 '20 at 12:53
  • Probably dup: https://math.stackexchange.com/questions/2599817/entropy-average-information-vs-uncertainty – leonbloy Sep 27 '20 at 13:20
  • @QiaochuYuan A highly compressed file (say, a .mp4 video) "is" white noise (in the statistical sense of "is"). Same for the digits of $\pi$. – leonbloy Sep 27 '20 at 13:23
  • Looking into this more i felt that information content actually means "unexpected events" which makes more sense. Could someone use this to come full circle with a good laymans description – develarist Sep 29 '20 at 21:15

0 Answers0