Unitary Matrix  \(U^H=U^{-1}\).  The columns are orthonormal.

Transpose denoted by \(A’\). rows and columns are interchanged: \(a_{ij} \leftarrow a_{ji}\)

Symmetric Matrix  \(A=A’\)

Positive Definite Matrix  Symmetric matrix A such that \(x^TAx > 0 \quad \forall x \neq 0\).  All eigenvalues of A are > 0.

Frobenius Norm

\left\|A\right\|=\sqrt{\sum_{i=1}^m\sum_{j=1}^n a_{ij}^2}

Orthonormal A set of vectors \(v_i\) is orthonormal if they all have length 1 and are orthogonal to each other, that is, \(\langle v_i, v_j \rangle = v_i{\,’} v_j = 0\) if \(i\neq j\).

Singular Value Decomposition

where U has orthonormal columns and V has orthonormal rows, and S is diagonal and has non-negative entries. If \(A\) is square, then \(U,V\) are unitary.