2024 FFT

Self Supervised Learning: Harmonic Analysis is Central to Today’s AI

Randall Balestriero (Citadel)

Abstract: The latest AI systems are trained on large unlabeled data collections with Self Supervised Learning (SSL). SSL enables unsupervised training of Deep Networks (DNs) by introducing auxiliary objectives such as consistency of representations under perturbations. We will prove that central to SSL lies a fundamental concept deeply rooted in signal processing: harmonic analysis on graphs.

In particular, we will demonstrate how SSL methods recover local and global spectral embedding solutions such as Laplacian Eigenmaps. Our findings make SSL analysis amenable to harmonic analysis researchers—a crucial step to move towards more principled and provable AI solutions.