CME 106 - Introduction to Probability and Statistics for Engineers

Some key concepts explained

By Afshine Amidi and Shervine Amidi

Use a distribution table to compute a probability

Let $X\sim\mathcal{N}(\mu,\sigma)$ with $\mu, \sigma$ known and $a,b\in\mathbb{R}$.

❐ Question: Compute $P(a\leqslant X\leqslant b)$.



❐ Step 1 ― Standardize $X$

We introduce $Z$, such that

\[Z=\frac{X-\mu}{\sigma}\sim\mathcal{N}(0,1)\]


❐ Step 2 ― Express the probability in terms of $Z$

We have:

\[P(a\leqslant X\leqslant b)=P\bigg(\frac{a-\mu}{\sigma}\leqslant\underbrace{\frac{X-\mu}{\sigma}}_{Z}\leqslant\frac{b-\mu}{\sigma}\bigg)=P\left(Z\leqslant \frac{b-\mu}{\sigma}\right)-P\left(Z\leqslant\frac{a-\mu}{\sigma}\right)\]


❐ Step 3 ― Find each term using the distribution table

Given that the values of $\frac{a-\mu}{\sigma}$ and $\frac{b-\mu}{\sigma}$ are known, we just have to look them up in a distribution table similar to this one.



Summing up We just computed the value of the probability by standardizing the normal variable to be able to look up the values in a standard normal distribution table.



Confidence intervals

Compute the confidence interval for $\mu$

Note: the example below is specific to the case where the variance is known and $n$ is large. The following reasoning can be reproduced for other cases in a similar fashion.

Let $X_1, ..., X_n$ be a random sample with mean $\mu$ and standard deviation $\sigma$ where only $\sigma$ is known, and let $\alpha\in[0,1]$.

❐ Question: Compute a confidence interval on $\mu$ with confidence level $1-\alpha$, that we note $CI_{1-\alpha}$.



❐ Step 1 ― Write in mathematical terms what we are seaching for

We want to find a confidence interval $CI_{1-\alpha}$ of confidence level $1-\alpha$ for $\mu$:

\[P(\mu\in CI_{1-\alpha})=1-\alpha\]


❐ Step 2 ― Consider the sample mean of $X$

We consider $\overline{X}$, which is such that:

\[\overline{X}=\frac{1}{n}\sum_{i=1}^nX_i\]


❐ Step 3 ― Standardize $\overline{X}$

We introduce $Z$, such that:

\[Z=\frac{\overline{X}-\mu}{\frac{\sigma}{\sqrt{n}}}\underset{n\gg1}{\sim}\mathcal{N}(0,1)\]

In general, this relationship is valid for large $n$ but it is always true in the particular case when the $X_i$ are normal.



❐ Step 4 ― Use $Z$ to find the quantiles

We can find the quantiles of $Z$ which are such that:

\[P(-z_{\frac{\alpha}{2}}\leqslant Z\leqslant z_{\frac{\alpha}{2}})=1-\alpha\]

Given that $Z$ follows a standard normal distribution, the quantity $z_{\frac{\alpha}{2}}$ can be found in the distribution table.



❐ Step 5 ― Re-write $Z$ in terms of $\overline{X}$

Knowing that $Z=\frac{\overline{X}-\mu}{\frac{\sigma}{\sqrt{n}}}$, we can re-write the previous expression:

\[P\left(\overline{X}-z_{\frac{\alpha}{2}}\frac{\sigma}{\sqrt{n}}\leqslant \mu\leqslant \overline{X}+z_{\frac{\alpha}{2}}\frac{\sigma}{\sqrt{n}}\right)=1-\alpha\]


❐ Step 6 ― Deduce the confidence interval

By taking into account steps 1 and 5, we can now deduce the confidence interval for $\mu$:

\[\boxed{CI_{1-\alpha}=\left[\overline{X}-z_{\frac{\alpha}{2}}\frac{\sigma}{\sqrt{n}},\overline{X}+z_{\frac{\alpha}{2}}\frac{\sigma}{\sqrt{n}}\right]}\]



Compute the confidence interval for $\sigma^2$

Let $X_1, ..., X_n$ be a random sample with mean $\mu$ and standard deviation $\sigma$ where $\sigma$ is unknown, and let $\alpha\in[0,1]$.

❐ Question: Compute a confidence interval on $\sigma^2$ with confidence level $1-\alpha$, that we note $CI_{1-\alpha}$.



❐ Step 1 ― Write in mathematical terms what we are seaching for

We want to find a confidence interval $CI_{1-\alpha}$ of confidence level $1-\alpha$ for $\sigma^2$:

\[P(\sigma^2\in CI_{1-\alpha})=1-\alpha\]


❐ Step 2 ― Consider the sample variance of $X$

We consider $s^2$, which is such that:

\[s^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\overline{X})^2\]


❐ Step 3 ― Standardize $s^2$

We introduce $K$, such that:

\[K=\frac{s^2(n-1)}{\sigma^2}\sim\chi_{n-1}^2\]

Here, $K$ follows a $\chi^2$ distribution with $n-1$ degrees of freedom.



❐ Step 4 ― Use $K$ to find the quantiles

We can find the quantiles $\chi_1^2, \chi_2^2$ of $K$ which are such that:

\[P(\chi_1^2\leqslant K\leqslant \chi_2^2)=1-\alpha\]

Given that $K$ follows a $\chi^2$ distribution with $n-1$ degrees of freedom, the quantiles can be found in the distribution table.



❐ Step 5 ― Re-write $K$ in terms of $s^2$

Knowing that $K=\frac{s^2(n-1)}{\sigma^2}$, we can re-write the previous expression:

\[P\left(\frac{s^2(n-1)}{\chi_2^2}\leqslant \sigma^2\leqslant\frac{s^2(n-1)}{\chi_1^2}\right)=1-\alpha\]


❐ Step 6 ― Deduce the confidence interval

By taking into account steps 1 and 5, we can now deduce the confidence interval for $\sigma^2$:

\[\boxed{CI_{1-\alpha}=\left[\frac{s^2(n-1)}{\chi_2^2},\frac{s^2(n-1)}{\chi_1^2}\right]}\]