Let X be a random variable whose PDF is $f(x)$, and $g$ a function of random variable X. I want to prove that $$E[g(X)] = \int{g(x)f(x)dx} $$ I've perfectly understood it in discrete case and I managed to prove also for continous case when $g$ is an increasing function. What is a formal proof for this formula? And what is a good intuition for it?
$\endgroup$ 41 Answer
$\begingroup$At first reading, it looks like you are trying to "prove" a definition. Indeed, on the Wikipedia page, the definition is given as:
In general, if X is a random variable defined on a probability space (Ω, Σ, P), then the expected value of X, denoted by E[X], ... is defined as the Lebesgue integral $$\operatorname{E} [X] = \int_\Omega X \, \mathrm{d}P = \int_\Omega X(\omega) P(\mathrm{d}\omega) $$
Here the space and the value space are both $\Bbb{R}^1$, $P(d\omega)$ is $f(x)dx$, and the variable is $g(x)$.
However, matters are more subtle than that, so your question does have some meat to it. In fact, the formula you are trying to prove is called the "law of the unconscious statistician" and it does not always work (in the continuous case) if the values of $X$ are vectors (or even complex numbers)!
You have to start with the actual definition, $$ E[X] = \int X(\omega)dP$$
The random variable $X$ induces a probability distribution α on the Borel subsets of the line $\alpha = PX^{-1}$. The distribution function F(x) corresponding to α is $$f(x) = \alpha | \alpha \in (-\infty,x) \wedge \alpha = P[\omega: X(\omega) \leq x]$$ The measure $\alpha$ is called the distribution of $X$ and $f(x)$ is called the distribution function of $X$.
Now if you have a function of a real variable $g(x)$ then $Z(\omega) = g(X(\omega)$ is again a random variable. The distribution of $Z$ is $\alpha g^{-1} \alpha$
By the "change of variables" theorem, $$E[X] = \int X[\omega]dP = \int x \, d\alpha$$
then $$E[g(x)] = \int g(X(\omega))dP = \int g(x) d\alpha = \int g(x) f(x) dx $$
I'm not 100% comfortable with this demonstration, since at some point (I think in the very last assertion) the reasoning must break down if the the function $g(x)$ is not real-valued..
$\endgroup$ 2
