Next: Catch-up and review Up: Describing Data Previous: Regression diagnostics and cautions:   Contents

## The matrix approach

• The matrix approach to multiple regression
• Some matrix algebra basics
• a matrix is a set of numbers arranged into rows and columns.
• Matrices are usually represented with bold-face capital type.
• bold-faced lower case letters represent vectors, which are single columns of numbers.
• In some matrix X, we refer to the value in the th row and th column as .
• Matrix Addition: matrices of the same dimension can be added together by simply adding the elements: X+Y=
• To multiply two matrices, the columns of one must equal the rows of the other.
• To do the multiplication, we multiple all the elements on the row of the first matrix by the elements on the column of the second and add up these products. Then we do this for the second column of the second variable and so forth. (Show graphically - use Ken's thumping technique)

• AB is not necessarily the same as BA. Multiplying an NxK matrix by a KxJ matrix will result in an NxJ matrix. We are usually interested in the special case where B is a Kx1 vector. Then our result is an Nx1 vector.

• The transpose (symbolized with a ') of a matrix is the matrix with the rows and columns reversed. (show with A above)

• The inverse of a matrix is the matrix which when multiplied by the original matrix results in the identity matrix, I.

• A diagonal matrix is any matrix in which the only non-zero elements are along the diagonal.
• Matrix notation for regression
• treat the matrix X as a matrix of data from observations (rows) and variables (columns), with an extra column of 1's at the front.

• treat the vector y as a vector of the dependent variables for observations.

• treat the vector as a vector of error terms.

• treat the vector as a vector of regression coefficients.

• Then the regression equation can be restated in matrix form as:

• Estimation using matrix format
• Our goal is to minimize the sum of squares. In matrix notation the sum of squares is:

(this is simply the FOIL in matrix form)

• We want to minimize this value. It turns out that this value is minimized when its derivative with respect to is zero:

• Through some matrix algebra, we can show that our estimates should be:

• Let's do an example

Inverting is more difficult, but computers will do it for you quickly

Next: Catch-up and review Up: Describing Data Previous: Regression diagnostics and cautions:   Contents
Aaron 2005-12-20