The differential entropy of a continuous random variable with probability density function is

When we have a continuous random vector that consists of random variables , , ..., , the differential entropy of is defined as the -fold integral

where is the joint probability density function of .

Thus, for example, the differential entropy of a multivariate Gaussian random variate with positive definite covariance matrix is

Additional properties of differential entropy include

where is a constant and

where is a scaling factor and is a scalar random variable. The above property can be generalized to the case of a random vector premultiplied by a matrix ,

where is a square invertible matrix and is its determinant.

For , Pang (2026) proved that among projections of the uniform distribution on the hypercube onto the line in a unit vector direction, differential entropy is maximized exactly when all coordinate magnitudes of that direction are equal. Equivalently, if , ..., are independent uniform random variables on and , then

with equality iff for every .

See also

Entropy, Hirschman Uncertainty Principle, Isotropic Constant, Logarithmically Concave Distribution, Shannon Entropy, Strong Isotropic Constant Conjecture

Portions of this entry contributed by Marwan A. Mattar

Portions of this entry contributed by Matthew R. Rudary

Explore with Wolfram|Alpha

References

Cover, T. M. and Thomas, J. A. Elements of Information Theory. New York: Wiley, 1991.Pang, Z. "Entropy Maximization for Cube Projections." 10 Nov 2026. https://arxiv.org/abs/2611.13533v1.

Referenced on Wolfram|Alpha

Differential Entropy

Cite this as:

Weisstein, Eric W., with contributions by Marwan A. Mattar and Matthew R. Rudary. "Differential Entropy." From MathWorld--A Wolfram Resource. https://mathworld.wolfram.com/DifferentialEntropy.html

Subject classifications