18 · Feature Compression

Deep dive 18 of Tensors for Machine Learning. One practical question:

What does it mean to compress a feature map?

Unfold an (H, W, C) tensor into an (H, W · C) matrix, keep its dominant patterns with a truncated SVD, and fold it back. Then measure three things that are easy to confuse: retained energy, storage cost and visual fidelity.

Two traps run through it. Energy is squared amplitude, not a sum of singular values. And the reconstruction is as big as the input — the compression lives in the factors, which at high rank cost more than the data.

This deep dive is take-home from the start: it is not one of the live labs.

Study here, then run the real thing: notebook 18 on Colab · all notebooks

Built from the workshop’s own material by Ravi Kalia and Sebastian Laverde Chunza, CC BY 4.0.