The Matrix Reference Manual

Copyright © 1998-2005 Mike Brookes, Imperial College, London, UK
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license is included in the section entitled "GNU Free Documentation License".

To cite this manual use: Brookes, M., "The Matrix Reference Manual", [online] http://www.ee.ic.ac.uk/hp/staff/dmb/matrix/intro.html, 2005

Introduction

This manual contains reference information about linear algebra and the properties of real and complex matrices. The manual is divided into the following sections:

• Main Index: Alphabetical index of all entries.
• Properties : Properties and numbers associated with a matrix such as determinant, rank, inverse, …
• Eigenvalues : Theorems and matrix properties relating to eigenvalues and eigenvectors.
• Special : Properties of matrices that have a special form or structure such as diagonal, traingular, toeplitz, …
• Relations : Relations between matrices such as equivalence, congruence, …
• Decompositions : Decomposing matrices as sums or products of simpler forms.
• Identities : Useful equations relating matrices.
• Equations : Solutions of matrix equations
• Calculus : Differentiating expressions involving matrices whose elements are functions of an independent variable.
• Stochastic : Statistical properties of vectors and matrices whose elements are random numbers.
• Signals : Properties of observation vectors and covariance matrices from stochastic and deterministic signals.
• Examples: 2#2 : Examples of 2#2 matrixes with graphical illustration of their properties.
• Formal Algebra : Formal definitions of algebraic constructs such as groups, fields, vector spaces, …
• GNU Free Documentation License

Format of Manual Entries

The general format of each entry is as follows:

1. Definition of the term
2. Outline of why it is important
3. Geometric Interpretation
   The geometric interpretation of a matrix property or theorem is generally described for 2 or 3 dimensions. Words prefixed by + should be altered appropriately for other dimensions. Thus the word +area should be replaced by volume for 3-D spaces and by hyper-volume for larger spaces.
4. List of properties and theorems:
   • Theorems apply to real matrices or complex matrices of arbitrary shape unless explicitly stated.
     • Most but not all theorems are also true for matrices whose elements are from other fields; in particular fields with characteristic 2 cause many properties to fail because x = -x does not imply x = 0.
   • Matrix dimensions are assumed to allow the sums and products in a theorem.
   • If a theorem explicitly involves inv(A) or division by A, then A is assumed to be non-singular.
   • If all matrices, vectors and scalars in a theorem are of a particular form, the theorem is prefaced thus: [Real] or [Complex n#n].
   • If some matrices are of a particular form, the theorem is prefaced thus: [A,B:n#n, A:symmetric].
   • denotes a hypertext link to a proof.
   • denotes a hypertext link to an example.
5. Links to related topics

Notation

The notation is based on the MATLAB software package; differences are notes below. All vectors are column vectors unless explicitly written as transposed.

• Matrices are represented as bold upper case (A), column vectors as bold lower case (a) and real or complex scalars as italic lower case (a).
• The matrix A_[2#3] has 2 rows and 3 columns while the column vector a_[4] has 4 elements.
• A matrix can be specified explicitly by listing its elements and using a semicolon to separate each row. Thus [1 2 3; 4 5 6] is a matrix with 2 rows and 3 columns. This notation can be used to compose large matrices from smaller ones: [A B;C D]. Each row must have the same total number of columns and each matrix within a row must have the same number of rows.
• Operators
  • Operator Precedence:
    • (1)  Superscripts, powers and : suffix
    • (2)  scalar and matrix multiplication/division
    • (3)  ¤ (Kroneker product)
    • (4)  •  ÷  (elementwise multiplication/division)
    • (5)  Addition/Subtraction
  • A • B and A ÷ B denote element-by-element multiplication and division
  • A ¤ B = KRON(A,B) is the Kronecker product of A and B. If A is m#n and B is p#q then A ¤ B is mp#nq and equals the block matrix [a(1,1)B ... a(1,n)B ; ... ; a(m,1)B ... a(m,n)B].
  • A: denotes the large column vector formed by concatenating all the columns of A. If A is m#n, then A: = [a_1,1 a_2,1 … a_m,1 a_1,2 a_2,2 … a_m,n]^T.
  • The following superscripts are used:
    • A^C denotes the complex conjugate of A.
    • A^H denotes the conjugate transpose of A. If A is real then A^H = A^T.
    • A^R and A^I are the real and imaginary parts of  A = A^R + j A^I .
    • A^T denotes the transpose of A.
    • A^-1, A^# and A⁺ denote respectively the inverse, generalized inverse and pseudoinverse of A.
• For real matrices only, A>B means that each element of A is greater than the corresponding element of B. Similar definitions apply to <, >= and <=.
• |A| and ||A||_F denote the determinant and Frobenius norm of A.
• ||a|| denotes the euclidean norm of a
• |a| denotes the absolute value of a

Subscripts

• a_ij or a_i,j denotes the element of matrix A in row i of column j. Row and column indices begin at 1.
• A_2:5,6:7 denotes the 4#2 submatrix of A consisting of row 2,3,4,5 and columns 6 and 7.
• a_j denotes the j'th column of matrix A.
• A_X,Y defines a matrix of the same size as X and Y (which must be the same size). Subscripts are taken from corresponding positions in X and Y. [Different from MATLAB]

Special Matrices

• The dimensions of the following special matrices are normally deduced from context but are occasionally specified explicitly (e.g. 0_[m#n]):
  • The matrix or vector 0 consists entirely of zeros.
  • The matrix or vector 1 consists entirely of ones.
  • The matrix I denotes the square identity matrix with 1's down the main diagonal and 0's elsewhere.
  • The matrix J denotes the square exchange matrix with 1's along the main anti-diagonal and 0's elsewhere.
• m:n denotes a column vector of length |1+n-m| whose elements go from m to n in steps of +1 or -1 according to whether m<n or m>n. [Different from MATLAB]

Functions

Several of the functions listed below have different meanings according to whether their argument is a scalar, vector or matrix. The form of the result is indicated by the function's typeface.

• ABS(A) and abs(a) involves taking the absolute value of each matrix or vector element.
• ADJ(A) is the adjoint of the square matrix A.
• CHOOSE(n,r) is a matrix with n!/(r! (n-r)!) rows, each a different choice of r numbers out of the numbers 1:n. Each row is listed in ascending order.
• CONJ(A) is the complex conjugate of A.
• conv(a_[m],b_[n])_[m+n-1] is the convolution of a and b, i.e. a vector whose i'th element is the sum of a(j)b(i-j+1) where j goes from 1 to i.
• det(A) or |A| is the determinant of the square matrix A.
• diag(A) is the vector consisting of the diagonal elements of A.
• DIAG(a) is the diagonal matrix whose diagonal elements are the elements of a.
• DIAG(A,B,C) denotes the matrix [A 0 0; 0 B 0; 0 0 C]
• eig(A) is a vector containing the eigenvalues of A. If A is Hermitian, they are sorted into descending order.
• floor(x) is the most positive integer <= x
• INV(A) or A^-1 is the inverse of A.
• KRON(A,B) = A ¤ B is the Kronecker product of A and B. If A is m#n and B is p#q then A ¤ B is mp#nq and equals the block matrix [a(1,1)B ... a(1,n)B ; ... ; a(m,1)B ... a(m,n)B].
• max(a | "condition") is the maximum of a subject to (an optional) "condition".
• min(a | "condition") is the minimum of a subject to (an optional) "condition".
• PERM(n) is a matrix with n! rows, each a different permutation of the numbers 1:n.
• pet(A) is the permanent of A.
• prod(a) is the product of the elements of a.
• prod() is the vector formed by multiplying together the elements of each row of A. [Different from MATLAB].
• rho(A) is the spectral radius of A.
• rows(A) is the number of rows in the matrix A.
• tr(A) is the trace of A.
• rank(A) is the dimension of the subspace spanned by the columns of A.
• sgn(a) equals +1 or -1 according to the sign of a or 0 if a=0.
• sgn(a) equals +1 or -1 according to the signature of the permutation needed to sort the elements of a into ascending order.
• sgn(A) is a vector giving the permutation signatures of each row of A. Each entry equals +1 or -1.
• SKEW(a) is the 3#3 skew-symmetric matrix [0 -a₃ a₂; a₃ 0 -a₁; -a₂ a₁ 0] where a is a 3-element vector. The vector cross product is given by a × b = SKEW(a) b = -SKEW(b) a. [See skew-symmetric for more properties of SKEW()].
• sum(a) is the sum of the elements of a.
• sum(A) is the vector formed by summing the rows of A. [Different from MATLAB].
• sum(A) is the scalar formed by summing all the elements of A. [Different from MATLAB].
• TOE(a_[m+n-1])_[m#n] is the m#n matrix with constant diagonals whose i,j^th element is a_i-j+n.
• TVEC(m,n) is an orthogonal mn#mn permutation matrix whose i,j^th element is 1 if j=1+m(i-1)-(mn-1)floor((i-1)/n) or 0 otherwise [see vectorized transpose].

Other Web Sites

• Linear Algebra Glossary by John Burkardt [R.4]
• Mathworld by Eric Weisstein: Linear Algebra [R.14]
• The Matrix Cookbook by Kaare Brandt Petersen [R.13]

Acknowledgements

No originality is claimed for any of the material in this reference manual. The following books have in particular been very helpful:

• A Survey of Matrix Theory and Matrix Inequalities by M Marcus & H Minc, Prindle, Weber & Schmidt, 1964 / Dover, 1992 [R.11]
• Matrix Analysis and Topics in Matrix Analysis by R A Horn & C R Johnson, CUP 1990/1994, [R.9, R.10]
• Applied Linear Algebra by B. Noble and J.W.Daniel, Prentice-Hall, 1988 [R.12]
• Finite Dimensional Vector Spaces by P.R.Halmos, D Van Nostrand, 1958 [R.7]
• Generalized Inverses by A.Ben-Israel and T.N.E.Greville, Wiley1974 [R.2]
• Matrix Computations by G.H.Golub & C.F.Van Loan, John Hopkins University Press, 1983 ISBN 0-946536-00-7/05-8 [R.6]
• Matrix Methods in Stability Theory by S.Barnett and C.Storey, Nelson, 1970 [R.1]

I would like to thank the following people who have made suggestions or corrections to this website and apologise to anyone whose name I have omitted from this list: Gerard Baron, Mike Fairbank, Carlos Fernandes, Thomas Foregger, John Halleck, Kaare Brandt Petersen, Jacopo Piazzi, Martin Zimmermann.