Question

Show that any covariance matrix is positive semidefinite.

Answer 1

Let C be a covariance matrix, and X be a random vector such that C is it's covariance matrix.

C is positive semi definite if and only if it's quadratic form is non negative.

Quadratic form of a matrix C is a^tCa.

So, quadratic form of the matrix C is non negative means a^tCa 0 for every vector a in Rⁿ, where n is the dimension of the matrix C.

We will show that a^tCa is the variance of the random variable a^tX. As variance of a random variable is always non negative. We have that a^tCa is greater than or equal to zero.

Without loss of generality assume that mean of X is 0 vector.

Then C = E(XX^t) and mean of the random variable a^tX is zero.

Variance of the random variable a^tX is E{(a^tX)²}

a^tX being a 1×1 matrix, it is a symmetric matrix. So a^tX =X^ta.

So Var(a^tX) = E(a^tX.X^ta)=a^tE(XX^t)a = a^tCa.