CP is four convolutions, Tucker-2 is three
A 2-D convolution layer’s weights form a 4-way tensor: out-channels × in-channels × height × width. Substitute a rank-R CP and move each sum inward, and the layer becomes four layers (Lebedev et al. 2015):
- a 1×1 conv, 256 → R channels
- a 3×1 depthwise conv on each channel
- a 1×3 depthwise conv on each channel
- a 1×1 conv, R → 256 channels
Depthwise (groups=R) means each channel is convolved on its own: a rank-one term never mixes with another — the superdiagonal core again. The chain stores 518R weights; R = 36 stores 32× fewer numbers than the dense kernel.
Tucker-2 compresses only the two channel modes (Kim et al. 2016) and becomes three layers: a 1×1 conv, an ordinary 3×3 conv whose kernel is the core, and another 1×1 conv.