Wikipedia:Reference desk/Archives/Mathematics/2016 February 5

= February 5 =

Hessian Matrix Meaning
Let $$f:\R^n\to\R$$ be a smooth function. Let $$x\in\R^n$$, such that the gradient of f at x is zero. Let H be the Hessian matrix of f at the point x. Let V be the vector space spanned by the eigenvectors corresponding to negative eigenvalues of H. Let $$y\in V$$. Then, f(x)>f(y)? or maybe f(x)>f(x+y)?

In other words, does negative eigenvalue imply maximum point at the direction of the corresponding eigenvector, or maybe this is a maximum in another direction, and not in the direction of the eigenvector? עברית (talk) 06:45, 5 February 2016 (UTC)


 * See Morse lemma.  S ławomir  Biały  12:23, 5 February 2016 (UTC)


 * You're kind of circling around the second derivative test for functions of several variables. The Taylor expansion for f at x is
 * $$f(\mathbf{x}+\mathbf{y}) \approx f(\mathbf{x}) + \mathbf{y}^\mathrm{T} \mathrm{D} f(\mathbf{x}) + \frac{1}{2!} \mathbf{y}^\mathrm{T} \mathrm{D}^2 f(\mathbf{x}) \mathbf{y} + \cdots$$
 * where Df is the gradient and D2f is the Hessian. In this case the gradient is 0 at x so this reduces to
 * $$f(\mathbf{x}+\mathbf{y}) \approx f(\mathbf{x}) + \frac{1}{2!} \mathbf{y}^\mathrm{T} \mathrm{D}^2 f(\mathbf{x}) \mathbf{y} + \cdots .$$
 * Let e be an eigenvector with eigenvalue λ, and wlog take e to be length 1. If y = te, then
 * $$f(\mathbf{x}+\mathbf{y}) \approx f(\mathbf{x}) + \frac{1}{2!} \lambda t^2 + \cdots $$
 * so f has a local minimum or maximum along the line parallel to e though x, depending on whether λ is positive or negative. If e, f ... are several linearly independent eigenvectors, with eigenvalues λ, μ, ..., and y = te + uf + ... , then
 * $$f(\mathbf{x}+\mathbf{y}) \approx f(\mathbf{x}) + \frac{1}{2!} (\lambda t^2 + \mu u^2 + \cdots) + \cdots $$
 * so f has a local minimum or maximum in the relevant space though x provided λ, μ, ... have the same sign. (The eigenvectors may be taken to be orthogonal since D2f is symmetric.) Note, this is only valid for y sufficiently small, otherwise the higher order terms in the Taylor series become significant and the approximation is no longer valid. --RDBury (talk) 12:46, 5 February 2016 (UTC)


 * Oh, great! Thank you! :) עברית (talk) 08:38, 6 February 2016 (UTC)