The Frobenius norm

To measure the size of a matrix, deep learning most often uses the Frobenius norm: the square root of the sum of every squared entry, ‖A‖_F = √(Σᵢ,ⱼ Aᵢ,ⱼ²). It is the matrix analogue of the L² norm of a vector.

For the 2 × 2 matrix with rows [1, 2] and [2, 4]: ‖A‖_F = √(1 + 4 + 4 + 16) = √25 = 5.

It can also be written with the trace (see The trace): ‖A‖_F = √Tr(AAᵀ).