Matrix Relations

Go to: Introduction, Notation, Index

Congruence

Square matrices A and B are congruent if there exists a non-singular X such that B= X^TAX .

For Hermitian congruence, see Conjuctivity.

Congruence implies equivalence.

• Congruence is an equivalence relation.
• Congruence preserves symmetry, skewsymmetry and definiteness
• A is congruent to a diagonal matrix iff it is Hermitian.

Conjunctivity

Square matrices A and B are conjunctive or hermitely congruent if there exists a non-singular X such that B= X^HAX.

• If A is hermitian, it is conjunctive to a matrix of the form DIAG(I,-I,0).
• If A is skew-hermitian, it is conjunctive to a matrix of the form DIAG(jI,-jI,0).
• A is conjunctive to I iff it is positive definite hermitian in which case A=U^HU for some non-singular upper triangular U.

Equivalence

Two m*n matrices, A and B, are equivalent iff there exists a non-singular m*m matix Mand a non-singular n*n matrix N with B=MAN .

• Equivalence is an equivalence relation.
• A and B, are equivalent iff they have the same rank.

Hadamard (or Schur) Product

The Hadamard product of two m#n matrices A and B, written in this website A • B, is formed by the elementwise multiplication of their elements. The matrices must be the same size.

• A • B = B • A
• A^T • B^T = (A • B)^T
• (a • b)(c • d)^T = ac^T • bd^T = ad^T • bc^T
• If A and B are +ve definite then A • B is +ve definite.
• If A and B are +ve semi-definite then  A • B is +ve semi-definite and rank(A • B) <=rank(A)×rank(B)

Kronecker Product

The Kronecker product of A_[m#n] and B_[p#q],written in this website A ¤ B or KRON(A,B), is equal to the mp#nq matrix [a(1,1)B … a(1,n)B ; … ; a(m,1)B … a(m,n)B ]. It is also known as the direct product or tensor product of A and B. The Kronecker Product operation is often denoted by a × sign enclosed in a circle which we approximate with ¤. Note that in general A ¤ B != B ¤ A. In the expressions below a : suffix denotes vectorization.

• Associative: A ¤ B ¤ C = A ¤ (B ¤ C) = (A ¤ B) ¤ C
• Distributive: A ¤ (B+C) = A ¤ B + A ¤ C
• Not Commutative: A ¤ B = B ¤ A iff A = cB for some scalar c
• det(A_[m#m] ¤ B_[n#n]) = det(A)ⁿdet(B)^m.
• tr(A_[n#n] ¤ B_[n#n]) = tr(A) tr(B)
• A^T ¤ B^T = (A ¤ B)^T
• A^H ¤ B^H = (A ¤ B)^H
• A^-1 ¤ B^-1 = (A ¤ B)^-1
  • A ¤ B  is singular iff A or B is singular.
• A ¤ B = I iff A = cI and B = c^-1I  for some scalar c.
  • I_[m#m] ¤ I_[n#n] =  I_[mn#mn]
• A ¤ B is orthogonal iff cA and c^-1B are orthogonal for some scalar c.
• A ¤ B is diagonal iff A and B are diagonal.
• If Aa=pa and Bb=qb then (A ¤ B)(a ¤ b)=pq(a ¤ b). The algebraic multiplicity of the eigenvalue pq is the product of the corresponding multiplicities of p and q.
• If Aa=pa and Bb=qb then (A ¤ I + I ¤ B)(a ¤ b)=(p+q)(a ¤ b)
• (ba^T): = (a^T ¤ b): =a ¤ b  where the : suffix denotes vectorization.
• a ¤ (BC)= (a ¤ B)C
  • a^T ¤ (BC)= B(a^T ¤ C)
  • (AB) ¤ c= (A ¤ c)B
    • (AB) ¤ c^T= A(B ¤ c^T)
  • a ¤ b^T = ab^T
    • a^T ¤ b = ba^T
• a^T ¤ BC^T ¤ d = (B ¤ d)(a ¤ C)^T
• a ¤ BC^T ¤ d^T = (a ¤ B)(C ¤ d)^T
•  (a ¤ b)(c ¤ d)^T = (ba^T): (dc^T):^T = a ¤ bc^T ¤ d^T= c^T ¤ ad^T ¤ b =  ac^T ¤ bd^T
• AB ¤ CD = (A ¤ C)(B ¤ D)
  • A_[m#n] ¤ B_[p#q] = (A ¤ I_[p#p])(I_[n#n] ¤ B) = (I_[m#m] ¤ B)(A ¤ I_[q#q])
    • a_[m] ¤ B_[p#q] = (a ¤ I_[p#p])B
    • A_[m#n] ¤ b_[p] = (I_[m#m] ¤ b)A
    • a_[m] ¤ B_[p] = (a ¤ I_[p#p])b = (I_[m#m] ¤ b)a
  • I_[n#n] ¤ AB = (I_[n#n] ¤ A)(I_[n#n] ¤ B)
  • AB ¤ I_[n#n] = (A ¤ I_[n#n])(B ¤ I_[n#n])
  • ab^H ¤ cd^H = (a ¤ c)(b ¤ d)^H = (ca^T):(db^T):^H
  • a^Hbc^Hd = a^Hb ¤ c^Hd = (a ¤ c)^H(b ¤ d) = (ca^T):^H(db^T):
  • (A ¤ B)^H(A ¤ B) = A^HA ¤ B^HB
• (ABC): = (C^T ¤ A) B:
  • (AB): = (I ¤ A) B: = (B^T ¤ I) A:= (B^T ¤ A) I:
  • (Abc^T): = (c ¤ A) b = c ¤ Ab
  • ABc = (c^T ¤ A) B:
  • a^TBc = (c ¤ a)^T B: = (ac^T):^T B: = (c ¤ a)^T B: = (a ¤ c)^T B^T: =  B:^T (a ¤ c) = B:^T (ca^T):
• (ABC):^T =  B:^T (C ¤ A^T)
  • (AB):^T = B:^T (I ¤ A^T)  = A:^T (B ¤ I) = I:^T (B ¤ A^T)
  • (Abc^T):^T =  b^T(c^T ¤ A^T) = c^T ¤ b^TA^T
  • a^TB^TC = B:^T (a ¤ C)
• If Y=AXB+CXD+... then Y: = (B^T ¤ A + D^T ¤ C+...) X: however this is a slow and often ill-conditioned way of solving such equations for X.

In the identities below, I_n = I_[n#n] and T_m,n = TVEC(m,n) [see vectorized transpose]

• B_[p#q] ¤  A_[m#n] = T_p,m (A  ¤  B) T_n,q
  • (A_[m#n]  ¤  B_[p#q]) T_n,q  = T_m,p (B ¤ A)
  • a_[m]  ¤  B_[p#q]  = (a ¤ I_p)B = T_m,p (B ¤ a)
  • A_[m#n] ¤ b_[p] = (I_m] ¤ b)A = T_m,p (b ¤ A)
  • a_[m] ¤ b_[p] = (a  ¤ I_p)b = (I_m] ¤ b)a
• (A ¤ b): = A: ¤ b
• (a_[m] ¤ B_[p,q]): = (T_q,m  ¤ I_p)(a ¤ B:) = (I_q ¤ a ¤ I_p)B:
• (A ¤ B): = (I_n ¤ T_q,m ¤ I_p)(A: ¤ B:) = (I_n ¤ T_q,m ¤ I_p)(A ¤ B:): = (T_n,q ¤ I_mp)(A: ¤ B): = (I_nq ¤ T_m,p)(B: ¤ A):

Loewner Partial Order

We can define a partial order on the set of Hermitian matrices by writing A>=B iff A-B is positive semidefinite and A>B iff A-B is positive definite.

• The partial order is:
  • reflexive: A>=A for all A.
  • antisymmetric: A>=B and B>=A are both true iff A=B.
  • transitive: If A>=B and B>=C then A>=C.
• Any pair of hermitian matrices, A and B, satisfy precisely one of the following:
  1. None of the relations A<B, A<=,B A=B, A>=B, A>B is true.
  2. A<B and A<=B only are true.
  3. A<=B only is true.
  4. A=B, A<=B and A>=B only are true.
  5. A>=B only is true.
  6. A>B  and A>=B only are true.
• A>=B iff x^HAx >= x^HBx for all x where >= has its normal scalar meaning (likewise for >)
• A>=B iff D^HAD >= D^HBD for any, not necessarily square, D. (not true for >).
• A>B iff D^HAD > D^HBD for any non-singular D.

Orthogonal Similarity

Real square matrices A and B are orthogonally similar if there exists an orthogonal Q such that B= Q^TAQ .

Orthogonal similarity implies both similarity and congruence.

See also: Unitary similarity

Similarity

Square matrices A and B are similar if there exists a non-singular X such that B=X^-1AX .

Similar matrices represent the same linear transformation in a different basis. Similarity implies equivalence .

• Similar matrices have the same trace, determinant, eigenvalues and characteristic equation.
• Two matrices are similar iff their Jordan forms contain the same hypercompanion blocks.

Unitary Similarity

Square matrices A and B are unitarily similar if there exists a unitary Q such that B= Q^HAQ .

Unitary similarity implies both similarity and conjunctivity. If A and B are real, they are orthogonally similar .