Calculate Hessian of a "weird" function

let be $L $ an invertible matrix, $b in mathbb{R}^{n}$ and $mu in mathbb{R}$

and $J: mathbb{R}^{n} rightarrow mathbb{R}$ defined as $J(x)=|L x|_{2}^{2}+mu x^{t} x-b^{t} x$

How do I calculate its hessian matrix?
I cant operate as a usual function and calculate its gradient and also i dont know what to do with that norm


$|L x|_{2}^{2} =sum_{i=1}^{n}sum_{j=1}^{n} (l_{i,j}cdot x_{j})^2$

$mu x^{t}cdot x =sum_{i=1}^{n} mu cdot x_{i}^2$

$-b^{t} x = – sum_{i=1}^{n} b_{i} cdot x_{i}$

so $J(x)=\sum_{i=1}^{n}sum_{j=1}^{n} (l_{i,j}cdot x_{j})^2+sum_{i=1}^{n} mu cdot x_{i}^2 – sum_{i=1}^{n} b_{i} cdot x_{i} $

so the gradient is

$nabla J(x)=(sum_{i=1}^{n} l_{i,1} + 2 cdot mu cdot x_{1} – b_{1}, …, sum_{i=1}^{n} l_{i,n} + 2 cdot mu cdot x_{n} – b_{n} ) $

and the Hessian is the diagonal matrix of $2 cdot mu$

Dunno if it is right and I also think there has to be an easier way of doing it, like operating with the vectors and matrix instead of deriving each variable on their own.


As some comments sugested(thank you all) i can write the function as :
$J(x)=x^tL^tLx + mu x^tx -b^tx$


$ nabla J(x)= 2(L^tL+Imu)x-b$


$HJ(x)=2LL^t+2 I mu$

Mathematics Asked by Ben Schneider on November 12, 2021

2 Answers

2 Answers

Since the Hessian is the matrix of second partial derivatives $frac{partial^2}{partial x_i partial x_j}$, the linear term will vanish. You can rearrange the second order term to:

$$(Lx)^t(Lx) + mu x^tx = x^t(L^tL + mu I)x.$$

Each second derivative will then select one of the coefficients of this second-degree homogeneous polynomial. Can you see your way to the end of the computation from here?

As for thinking of this in terms of matrices, think first of the derivative. Every linear function can be written as $p_1(x) = v^Tx$ for some vector $v$. The gradient of a smooth function $f$ at a point is defined as this vector for the best linear approximation to $f$ at that point.

Every homogeneous second-degree polynomial $p$ can be written as $$ p(x) = x^THx $$ for some symmetric matrix $H$. The Hessian of a smooth function $f$ at a point is defined as this matrix for the best second-order approximation to $f$ at that point.

So now you see the root of the strategy to isolate the second order term.

Answered by Neal on November 12, 2021

Hint When $h to 0$, we have begin{equation} J(x + h) = J(x) + text{linear terms} + |L h|_2^2 + mu h^t h end{equation}

Answered by Gribouillis on November 12, 2021

Add your own answers!

Related Questions

A question on linear maps itself

3  Asked on March 16, 2021 by popping900


How to find the generators of a principal Ideal?

1  Asked on March 10, 2021 by ayoub-rossi


Prove that $int_0^b x^3 dx = frac{b^4}{4}$

0  Asked on March 6, 2021 by dansidorkin


Ask a Question

Get help from others!

© 2021 All rights reserved.