Entropy (ENT)
Since ln(0) is undefined, assume that 0 * ln(0) = 0:
For a brief refresher on logarithms, click here.
Entropy equation
Entropy is usually classified as a first degree measure, but should properly be a "zeroth" degree!
The maximum value of ENT is .5 (more information below). Thanks to Lokesh Setia of the Institute for Computer Science , Albert-Ludwigs-University, Freiburg, Germany, for pointing this out to me. A previous version of this tutorial misstated this.
The term "entropy"
Entropy is a notoriously difficult term to understand; the concept comes from thermodynamics. It refers to the quantity of energy that is permanently lost to heat ("chaos") every time a reaction or a physical transformation occurs. Entropy cannot be recovered to do useful work. Because of this, the term is used in non technical speech to mean irremediable chaos or disorder. Also, as with ASM, the equation used to calculate physical entropy is very similar to the one used for the texture measure.
Energy is, in this context, the opposite of entropy. Energy can be used to do useful work. In that sense it represents orderliness. This is why "Energy" is used for the texture that measures order in the image.
For the mathematically inclined: More details about the maximum value of entropy from Lokesh Setia:
The term P * ln(P) is maximized where its
derivative with respect to. P is 0.
By the product rule, this derivative is P * d(ln(P))/d(P) + d(P)/d(P) * ln(P)
which simplifies to 1 + ln(P) = 0, yielding P = 1/e. This means that the
maximum of the term to be summed occurs when P is 1/e, which is about 0.378.
However by definition the sum of Pij = 1. With this constraint, the overall
maximum of the sum (i.e. of ENT) is 0.5. This maximum is reached when all
probabilities are equal.
Conceptually this makes sense because when all probabilities of DN pairs are equal, we have a random distribution of DN values, which would yield maximum "chaos" or entropy.
Exercise in calculating Entropy