Fundamental increment lemma

In single-variable differential calculus, the fundamental increment lemma is an immediate consequence of the definition of the derivative $ f'(a)$ of a function $ f$  at a point $ a$ :
 * $$f'(a) = \lim_{h \to 0} \frac{f(a+h) - f(a)}{h}.$$

The lemma asserts that the existence of this derivative implies the existence of a function $$\varphi$$ such that
 * $$\lim_{h \to 0} \varphi(h) = 0 \qquad \text{and} \qquad f(a+h) = f(a) + f'(a)h + \varphi(h)h$$

for sufficiently small but non-zero $ h$. For a proof, it suffices to define
 * $$\varphi(h) = \frac{f(a+h) - f(a)}{h} - f'(a)$$

and verify this $$\varphi$$ meets the requirements.

The lemma says, at least when $$h$$ is sufficiently close to zero, that the difference quotient
 * $$\frac{f(a+h) - f(a)}{h}$$

can be written as the derivative f' plus an error term $$\varphi(h)$$ that vanishes at $$h=0$$.

I.e. one has,
 * $$\frac{f(a+h) - f(a)}{h} = f'(a) + \varphi(h).$$

Differentiability in higher dimensions
In that the existence of $$\varphi$$ uniquely characterises the number $$ f'(a)$$, the fundamental increment lemma can be said to characterise the differentiability of single-variable functions. For this reason, a generalisation of the lemma can be used in the definition of differentiability in multivariable calculus. In particular, suppose f maps some subset of $$\mathbb{R}^n$$ to $$\mathbb{R}$$. Then f is said to be differentiable at a if there is a linear function
 * $$M: \mathbb{R}^n \to \mathbb{R}$$

and a function
 * $$\Phi: D \to \mathbb{R}, \qquad D \subseteq \mathbb{R}^n \smallsetminus \{ \mathbf{0} \},$$

such that
 * $$\lim_{\mathbf{h} \to 0} \Phi(\mathbf{h}) = 0 \qquad \text{and} \qquad f(\mathbf{a}+\mathbf{h}) - f(\mathbf{a}) = M(\mathbf{h}) + \Phi(\mathbf{h}) \cdot \Vert\mathbf{h}\Vert$$

for non-zero h sufficiently close to 0. In this case, M is the unique derivative (or total derivative, to distinguish from the directional and partial derivatives) of f at a. Notably, M is given by the Jacobian matrix of f evaluated at a.

We can write the above equation in terms of the partial derivatives $$\frac{\partial f}{\partial x_i}$$ as
 * $$ f(\mathbf{a} + \mathbf{h}) - f(\mathbf{a}) = \displaystyle\sum_{i=1}^n \frac{\partial f(a)}{\partial x_i} + \Phi(\mathbf{h}) \cdot \Vert\mathbf{h}\Vert$$