Project DelphiTensors Workshop

The cost table

Leading-term flop counts for a dense m × n factorization with m ≥ n, from Trefethen & Bau, Numerical Linear Algebra:

Method Flops The question it answers
Cholesky (n × n SPD) n³/3 Solve Ax = b many times, A symmetric positive definite
LU (n × n) 2n³/3 The same, A merely square
QR (m × n) 2mn² − 2n³/3 Least squares, without forming XᵀX
Eigendecomposition (n × n sym.) ≈ 9n³ What does repeated application converge to?
Thin SVD (m × n) 2mn² + 11n³ The best rank-k approximation of anything
Randomized SVD (rank k) ≈ 4mnk The same, when k ≪ n and A is dense
Lanczos (rank k, sparse) O(k · nnz(A)) The same, when A is sparse

Cholesky is the cheapest and the fussiest: it needs symmetric positive definiteness, and says so.