Choosing the rank — a scree for Tucker, pruning for CP

HOSVD gives one singular-value spectrum per mode. Unfold along mode n, take the singular values, plot the retained energy, and pick Rₙ at the elbow. The guarantee (De Lathauwer et al. 2000): truncating every mode loses at most the discarded energy, summed over modes. But on the ResNet-18 kernel the spectra have no elbow — 90% of the energy already needs more than 150 components per channel mode. A scree finds a rank when the tensor has one.

CP has no spectrum. Instead: over-parameterize, then prune. One fit with too many components and a penalty that switches whole components off. The group-lasso proximal step

a ← a · max(0, 1 − ημ / ‖a‖₂)

sets a column shorter than ημ to exactly zero. On a synthetic tensor of 6 true terms, the fit starts at rank 30: without a penalty all 30 survive and the spares fit noise; group lasso finds 6, and denoises.