Shannon's entropy for an image

Question

Shannon's entropy [plog(1/p)] for an image is a probabilistic method for comparing two pixels or a group of pixels.Suppose an image with a matrix of 3x3 has pixel intensity values

1 1 2
2 3 3
4 4 5

and another image with 3x3 matrix has group of pixels having intensity values

5 5 6
6 7 7
8 8 9

Then shannon's entropy for the images would be the same.So in this case the entropy values would point out that the images are same though in actual they are different.So image matching using this technique doesn't help.On basis of supervised classification where I classify an image based on trained databases of shannon's entropy ,we use the concept of entropy to find similarity between two images.Is there any method or research paper where this entropy can be used or modified for image matching for the above case..?

score 5 · Answer 1 · answered Oct 07 '12 at 23:05

Shannon's entropy works as a hashing or fingerprinting function here; they are not injective which means we have to be careful not to interpret too much into relations of values.

If $H(I_1) \neq H(I_2)$, then images $I_1$ and $I_2$ are certainly not the same.

If $H(I_1) = H(I_2)$ (or even $H(I_1) \approx H(I_2)$), however, we know nothing. The images might be similar, but the values can also be close because of non-injectivity.

We would like to have some smoothness, that is that two images are more different if $|H(I_1) - H(I_2)|$ is larger. Whether this is the case is certainly a quality criterion of such fingerprinting functions. Given your example, it does not seem as if entropy was a good fingerprint in this sense.

It would be confusing if any kind of entropy would be. Any rectangle of one color holds no information (entropy zero), yet two rectangles of different colors are as different as two images get.

score 2 · Answer 2 · edited May 23 '17 at 12:37

One related approach would be to compute a histogram of the pixel intensities for each of the two images, then compare those two histograms. More typically, we'd compute a histogram of some measure of color (e.g., the hue), instead of pixel intensities. This is a crude measure that can sometimes nonetheless be helpful: e.g., for distinguishing a picture of one room from a picture of another (e.g., if their walls are painted different colors, or they have a different distribution of colors).

There are many ways to compare two histograms. You can think of each histogram as a distribution for some random variable, and then compare the two distributions using of a number of methods. For instance, you could compare them using KL-divergence. Another approach is to use any standard distance measure on vector spaces, e.g., the ${\cal L}_2$ measure (sum of squared differences). You'll often want to bin the values first, before computing the histogram.

Shannon's entropy for an image

3 Answers3