I know this might seem as a simple question, but I just hope that I can learn how would people normally approach this question? Thanks in advance!
Compute the Fenchel conjugate of the following function:
$f(x) = e^x, x \in \Bbb R$
The Fenchel conjugate is defined as:
$f^*(x^*) = sup\{\langle x, x^*\rangle - f(x)\}$
$\endgroup$ 41 Answer
$\begingroup$This is a real function, so the inner product here is simply multiplication. You need to find$$f^*(y) = \sup_{x \in \Bbb{R}} yx - f(x) = \sup_{x \in \Bbb{R}} yx - e^x.$$This is a relatively simple optimisation problem. We have this differentiable function of $x$, with some fixed unknown $y$, and we want to find the maximum value of this function.
First, let's analyse the stationary points. We have, when differentiating (partially) with respect to $x$ and setting to $0$,$$y - e^x = 0 \implies x = \ln y,$$when $y > 0$. So, in this case, we get one stationary point at $x = \ln(y)$, producing a possible maximum of$$y\ln(y) - e^{\ln(y)} = y \ln y - y.$$We just need to compare this with the limits as $x \to \pm\infty$. As $x \to \infty$, then $yx - e^x \to -\infty$, regardless of the value of $y > 0$. As $x \to -\infty$, again assuming $y > 0$, we have $yx - e^x \to -\infty$ as well. Thus, the maximum must occur at the one and only stationary point, implying,$$f^*(y) = y\ln(y) - y$$when $y > 0$.
If $y \le 0$, then $y - e^x < 0$ for all $x$. This means that the function $yx - e^x$ is strictly decreasing in $x$, and hence the supremum is the limit as $x \to -\infty$. If $y < 0$, this limit is $\infty$. If $y = 0$, then the limit is $0$. So, our full conjugate is$$f(y) = \begin{cases} y \ln(y) - y & \text{if } y > 0 \\ 0 & \text{if } y = 0 \\ \infty & \text{if } y < 0. \end{cases}$$
$\endgroup$
