==== Spectral norm ====

The spectral norm of a matrix $A^{m\times n}$ is equal to its largest singular value:
$$
 \|A\|_2 = \max_{\|x\|_2=1} \|Ax\|_2 = \sigma_{\max}
$$

\\

==== Alternate definition ====

We show that $ \|A\|_2 = \max_{\|x\|=\|y\|=1} x^T A y $, where $x$ and $y$ are vectors.

=== Proof: ===

Let $A=U^T\Sigma V$ be its SVD. 

\begin{eqnarray*}
\|A\|_2 &=& \max_{\|x\|_2=\|y\|_2=1} (xU)^T \Sigma (Vy) \\
  &=& \max_{\|x'\|_2=\|y'\|_2=1} x'^T \Sigma y' \\
  &=& \max_{\|x'\|_2=\|y'\|_2=1} \sum_i \sigma_i x_i y_i \\
  &\leq& \sigma_{\max} \sqrt{\|x\|_2^2 \|y\|_2^2} = \sigma_{\max}
\end{eqnarray*}

The equality holds when $x'=y'= [1,0,\cdots,0]^T$ (assuming $\sigma_1$ is the largest singular value).

\\

==== Power method ====