How do we go about deriving the values of mean and variance of a Gaussian Random Variable $X$ given its probability density function ?
$\endgroup$ 34 Answers
$\begingroup$UPDATE 21-03-2017
A much faster way is to differentiate both sides of
$$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx=1$$ with respect to the two parameters $\mu$ and $\sigma^2$ (RHS will then be zero).
The Gaussian pdf is defined as $$f_X(x) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}$$
where $\mu$ and $\sigma$ are two parameters, with $\sigma >0$. By definition of the mean we have $$E(X) = \int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$ which using integral properties can be written as
$$E(X) = \int_{-\infty}^{\infty}(x+\mu)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$
$$=\int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \;+\; \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \qquad [1]$$
For the first integral, call it $I_1$ we have using additivity
$$I_1 = \int_{-\infty}^0x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$ Swapping the integration limits in the first we have
$$I_1 = -\int_{0}^{-\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$
and using again integral properties we have
$$I_1 = \int_{0}^{\infty}(-x)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(-x)^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$
$$\Rightarrow I_1 = -\int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx = 0\qquad [2]$$
So we have that
$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$
Multiply by $\sigma \sqrt2$ to obtain
$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sqrt{\pi}}e^{-x^2} dx = \mu\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx$$
...the last term because the integrand is an even function.
Now $$\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx = \lim_{t\rightarrow \infty}\frac{2}{\sqrt{\pi}}\int_{0}^{t}e^{-x^2} dx = \lim_{t\rightarrow \infty} \text{erf}(t) = 1$$
where "erf" is the error function. So we end up with $$E(X) = \mu$$ i.e. that the parameter $\mu$ is the mean of the distribution.
VARIANCE
We have
$$\text {Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$
Applying the same tricks as before we have
$$\int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx = \int_{-\infty}^{\infty}x^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$
$$=\sigma \sqrt2\int_{-\infty}^{\infty}(\sigma \sqrt2x)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(\sigma \sqrt2x)^2}{2\sigma^2}\right\}dx = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}x^2e^{-x^2}dx$$
Define $t=x^2\Rightarrow x= \sqrt t$ and $dt = 2xdx = 2\sqrt tdx \Rightarrow dx = (2\sqrt t)^{-1}dt$. Substituting
$$V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}(\sqrt t)^2(2\sqrt t)^{-1}e^{-t}dt = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \int_{0}^{\infty}t^{\frac 32 -1}e^{-t}dt= \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \Gamma\left(\frac 32\right)$$
$$\Rightarrow V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \frac {\sqrt \pi}{2} = \sigma^2$$
where $\Gamma()$ is the Gamma function. So the parameter $\sigma$ is the square-root of the variance, i.e. the standard deviation.
$\endgroup$ 5 $\begingroup$If $x\mapsto f(x)$ is the density function of a random variable $X$ with expected value $0$ and variance $1$, then $x\mapsto \frac1\sigma f\left(\frac{x-\mu}{\sigma}\right)$ is the density function of $\mu+\sigma X$, and thus of a random variable with expected value $\mu$ and variance $\sigma^2$.
That can be shown by thinking about the substitution $u = \dfrac{x-\mu}{\sigma}$ and $du=\dfrac{dx}\sigma$.
Therefore the problem reduces to this: How do we show that$$ x\mapsto \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right) $$is the density of a distribution with expectation $0$ and variance $1$?
If you know that the expectation of a distribution with density $f$ is $\int_{-\infty}^\infty xf(x)\,dx$, then you know that you need to find$$ \int_{-\infty}^\infty x \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx. $$
Since this is the integral of an odd function over an interval that is symmetric about $0$, it must be equal to $0$ unless the positive and negative parts both diverge to $\infty$. The positive part is a constant times$$ \int_0^\infty x \exp\left(\frac{-x^2}{2}\right)\,dx = \int_0^\infty \exp(u)\,du, $$where $u=-x^2/2$ so $du=x\,dx$. Clearly this is finite, and the negative part can be treated the same way.
The variance is\begin{align} & \int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2 \int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \Big(x\Big)\Big(x\exp\left(\frac{-x^2}{2}\right)\,dx\Big) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int x\,dv \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left( xv-\int v\,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left(\left[-x\exp\left(\frac{-x^2}{2}\right)\right]_0^\infty -\int_0^\infty -\exp\left(\frac{-x^2}{2}\right) \,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int_0^\infty \exp\left(\frac{-x^2}{2}\right) \,dx. \end{align}
You know that this is equal to $1$ if you know how the normalizing constant in the density function was found.
$\endgroup$ 2 $\begingroup$$$E(X)=E(x - \mu) + \mu$$So we're done by showing that $E(x - \mu)=0$
$$E(x- \mu)= \int_{ -\infty }^\infty \frac{(x-\mu)}{\sqrt{2\pi}\sigma}\cdot e^{-\frac {(x-\mu)^2}{2\sigma^2}}\,dx$$
Using the subsitiution $z=\frac{x-\mu}{\sigma}$ we get
$$E(x- \mu)= \frac {\sigma}{\sqrt{2\pi}} \int_{ -\infty }^\infty z \cdot e^{z^2/2}\,dz$$The function $f(z) = z\cdot e^{z^2/2}$ is obviously odd. Which makes$$\lim_{a \to \infty }\int_{ -a }^a z \cdot e^{z^2/2}\,dz=0 $$and thus we're left with$$E(X)= \mu$$
$\endgroup$ $\begingroup$Another way that might be easier to conceptualize:
As defined earlier, 𝐸(𝑋)= $\int_{-∞}^∞ xf(x)dx$
To make this easier to type out, I will call $\mu$ 'm' and $\sigma$ 's'
f(x)= $\frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2}}$}. So, putting in the full function for f(x) will yield
E(x)= $\int_{-∞}^∞ x \frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2} }dx$. Pretty gross to look at.
Looking at the exponent that e is being raised to, it seems like an ideal candidate for a u-subsitution. I will set u= $\frac {x-m}{s\sqrt2}$. That means that du= $\frac{dx}{s\sqrt2}$, or in other words dx= du•$s\sqrt2$.
Let's simplify the integral as much as possible using what we now know. Recalling the term $\frac{1}{\sqrt{(2πs^2)}}$, we shall rewrite it as $\frac{1}{s\sqrt{(2π)}}$.
E(x)= $\int_{-∞}^∞ x \frac{1}{s\sqrt{(2π)}}$ exp {$-u^2$}du•$s\sqrt2$.
The $s\sqrt2$'s from the first and last terms will cancel, leaving a constant of $\frac{1}{\sqrtπ}$. Let's put this in front of the integral for now to make this look a little better.
E(x)= $\frac{1}{\sqrtπ}$$\int_{-∞}^∞ x $ exp {$-u^2$}du.
The derivative of $\int_{-∞}^∞ $ exp {$-u^2$}du is in the form of a Gaussian integral. I'm not going to pretend that I can easily derive it (multivariable calc???) but it is widely known that the value of this integral is $\sqrtπ$.
We can use this knowledge to perform integration by parts to determine the value of the integral. I like to use u and v for this, but this u is not the same as the u from earlier. I hope this still makes sense.
$\int{udv}$=uv-$\int{ vdu}$
Set u=x. This means u'= 1dx. In terms of dx from earlier, u'= $s\sqrt2$ du. v'= $e^{(-u^2)}$du, and so by definition v= $\sqrtπ$.
uv= x$\sqrtπ$ and vdu=$\sqrtπ s\sqrt2 du$.
The overall integral is now equal to $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\int_{-∞}^∞ \sqrtπ s\sqrt2 du$] evaluated from ∞ to -∞.
Antideriving $\int vdu$ produces $s\sqrt2π$u. Recall our earlier value of u, u= $\frac {x-m}{s\sqrt2}$.
E(x)= $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $s\sqrt2π$$\frac{x-m}{s\sqrt2}$] or $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\sqrtπ$(x-m)] from ∞ to -∞.
Within the brackets, the terms being evaluated from ∞ to -∞ are:
x$\sqrtπ$-x$\sqrtπ$ + m$\sqrtπ$= m$\sqrtπ$.
E(x)= $\frac{1}{\sqrtπ}$ • m$\sqrtπ$ = m, or the mean, just like the definition of the expected value says.
$\endgroup$
