### Quadratic Forms and Transformation

Let \(A = \{a_{ij}\}\) be an \(n\times n\) matrix. A quadratic function of \(n\) variables \(x = (x_1,\ldots, x_n)’\) is defined as

\[

f(x) = x’ A x = \sum_{i,j} a_{ij} x_i x_j.

\]

Without loss of generality, assume \(A\) is symmetric; otherwise replace \(A\) by \((A+A’)/2\).

Since \(A\) is symmetric, it has spectral decomposition

\[

A = Q’ \Lambda Q.

\]

\(\Lambda\) is diagonal and the diagonal elements \(\lambda_1, \ldots, \lambda_n\) are eigenvalues of \(A\). \(Q = (q_1, \ldots, q_n)\) is a orthogonal matrix with the eigenvectors \(q_i\) as columns.

Let \(y = Q’x = Q^{-1} x\). Then we have

\[

f (x) = x’A x = x’ Q \Lambda Q’ x = y’ \Lambda y = \sum_{i} \lambda_i y_i^2 =\sum_{i} ||q_i’ x||^2 .

\]

### Random Variables

Let \(X= (X_1,\ldots, X_n)’\) be a random vector, with expectation \(\mu\) and covariance matrix \(\Sigma\):

\[

\mu = E[X] = (E[X_1], \ldots, E[X_n])

\]\[

\Sigma = E[(X-\mu) (X-\mu)’]

\]

The covariance matrix \(\Sigma\) is symmetric and positive semi-definite. This is because, for any vector \(b\) and \(Y= b’X\),

\[

0 \leq Var[Y] = Var[b’X] = b’ \Sigma b.

\]

Let \(A\) be a symmetric matrix, and define random variable \(Y = X’AX\). Then,

\[

E[Y] = E[X’A X] = tr(E[X’A X]) = E[ tr(X’A X) ] = E[ tr(A X X’) ]

\]\[

= tr(A E[ X X’ ]) = tr(A (\Sigma + \mu \mu’)) = tr(A\Sigma) + \mu’A\mu

\]