The proof of Eckhart-Young Theorem

In Understanding Singular Value Decomposition (SVD), we demonstrate that SVD serves as an optimal lower-rank approximation, drawing upon the Eckhart-Young Theorem. Our focus of this article lies in proving this theorem.

To set the stage, we introduce Weyl’s inequality. While its proof is beyond the scope of this article, interested readers can find it on wikipedia. The crux of its proof hinges on leveraging the min-max theorem.

Now, going back to the Eckhart-Young Theorem.