# Math: Vectors and matrices

You are currently browsing the archive for the Math: Vectors and matrices category.

## Useful properties of idempotent matrices

Idempotent matrices appear in regression analysis, so can’t be ignored. Here are commonly used properties:

Properties of idempotent matrices

• The eigenvalues of an idempotent matrix are 0 and 1.
• The rank of an idempotent matrix is equal to its trace (sum of its diagonal elements).

## Independence of quadratic forms

The following result is useful for deriving F-tests.

Independence of quadratic forms for orthogonal projection matrices

Let $\boldsymbol{ \epsilon} \sim N(0, \sigma \mathbf{I_n})$, and let $n \times n$ matrices $\mathbf{A}$ and $\mathbf{B}$ be orthogonal projections – and as such they must be symmetric and idempotent.
Then, the quadratic forms $\mathbf{\epsilon'A\epsilon}$ and $\mathbf{\epsilon'B\epsilon}$ are independently distributed if and only if

$\mathbf{AB} = \mathbf{BA} =0$

An example of an orthogonal projection matrix is regression is the Hat matrix, typically denoted in econometrics by $\mathbf{P}$, or $\mathbf{H}$, and $\mathbf{H = X(X'X)^{-1}X'}$. It’s called a Hat matrix cuz it puts a hat on the dependent variable, $\mathbf{HY}$ like this:

$\mathbf{HY} = \mathbf{\hat{Y}}$

## Matrix differentiation for econometrics: function of a vector and quadratic form

Let f be a scalar function of a $n \times 1$ vector $\mathbf{x}=(x_1,x_2,\cdots,x_n)'$. Then partially differentiating $f(\mathbf{x})$ with respect to each $x_i$ and arranging the derivatives into a column vector gives us

$\frac{\partial f}{\partial \mathbf{x}} = \begin{pmatrix} \frac{\partial f}{\partial x_1} \\ \frac{\partial f}{\partial x_2} \\ \vdots \\ \frac{\partial f}{\partial x_n} \end{pmatrix}$

Vector differentiation
Suppose $f(\mathbf{x})$ is a linear function of the column vector $\mathbf{x}$

$f(\mathbf{x}) = \mathbf{a'x}$

where $\mathbf{a}$ is a $n \times 1$ vector of constants. Then

$\frac{\partial f}{\partial \mathbf{x}} = \frac{\partial (\mathbf{a'x})}{\partial \mathbf{x}}=\frac{\partial \mathbf{x'a}}{\partial \mathbf{x}} = \mathbf{a}$

Derivative of a quadratic form
Denote by $\mathbf{A}$ a $n \times n$ matrix of constants. Let $f(\mathbf{x})$ be the quadratic form

$f(\mathbf{x}) = \mathbf{x'Ax}$

Then

$\frac{\partial (\mathbf{x'Ax})}{\partial \mathbf{x}} = (\mathbf{A} + \mathbf{A'})\mathbf{x}$

If $\mathbf{A}$ is symmetric (as is commonly the case in econometrics and statistics) we have

$\frac{\partial (\mathbf{x'Ax})}{\partial \mathbf{x}} = 2\mathbf{Ax}$