Steepest-Decent-Derivation

Definitions:
Assuming that a square matrix A is positive definite and symmetric and A\in\mathscr{R}^{n x n} the quadratic form of a given matrix from of the matrix equation Ax = b is given. This form recasts the linear system (Ax = b) as the condition that the vector x minimizes this quadratic form:

(1)   \begin{equation*} \phi(x) = \frac{1}{2}x^TAx - x^Tb \end{equation*}

Claim:
The gradient of the function \phi(x) is equal to Ax - b iff A is a positive definite and symmetric matrix.

(2)   \begin{equation*} \nabla\phi(x) = Ax - b \end{equation*}

Direct Proof:
In this proof, both sides of the equation are fully expanded to derive equivalent statements given the specific properties of A. For simplicity in the expansion we assume n = 2, this derivation can then be generalized to any n.

(3)   \begin{equation*} \nabla\phi(x) =  \def\arraystretch{1.4} \Large \begin{pmatrix} \frac{\partial\phi}{\partial x_1} \\ \frac{\partial\phi}{\partial x_2} \end{pmatrix} \end{equation*}