The Empirical Rule and Other Rules in Statistics
In any Statistics class you will find very frequently that certain "rules" are commonly referred to. Those rules are usually intended to simplify your life and help you with making some calculations easier. But not all those rules are made equal. Indeed not all of those rules are actual "rules", as some are only approximations, and as such, may have some specific use only, or even limited use sometimes.
In the following paragraphs we will discuss a few of those Stats rules and approximations that are commonly used. Those are quite simple in general, but you need to know exactly how to use them in the way that is intended.
Empirical Rule For the Normal Distribution
This is by far one of the most broadly known "rule" in Statistics. I keep writing "rule" with quotes, because this is not really a rule but an approximation. The Empirical Rule states that if a variable is normally distributed, the approximately 68% of the distribution is within one standard deviation of the mean, 95% of the distribution is within two standard deviations of the mean and 99.7% of the distribution is within three standard deviations of the mean.
First of all, let us see why this makes sense. The event that corresponds to the values that are within one standard deviation of the mean is \(\left\{ \mu -\sigma \le X\le \mu +\sigma \right\}\), and if we normalize (subtract by \(\mu\) and divide by \(\sigma\)), we get the following equivalent events:
\[\left\{ \mu -\sigma \le X\le \mu +\sigma \right\}=\left\{ -\sigma \le X-\mu \le \sigma \right\}=\left\{ -1\le \frac{X-\mu }{\sigma }\le 1 \right\}\]
But, if \(X\) is normally distributed with mean \(\mu\) and standard deviation \(\sigma\), we know that the variable \(\frac{X-\mu }{\sigma}\) has a standard normal distribution (this is a normal distribution with mean 0 and standard deviation 1). Typically, the variable \(\frac{X-\mu }{\sigma}\) is written as \(Z\), so then what we have is
\[\left\{ \mu -\sigma \le X\le \mu +\sigma \right\}=\left\{ -\sigma \le X-\mu \le \sigma \right\}=\left\{ -1\le \frac{X-\mu }{\sigma }\le 1 \right\}=\left\{ -1\le Z\le 1 \right\}\]where \(Z\) has a standard normal distribution. If we use a calculator, or a spreadsheet program like Excel, we find that the probability of the event that corresponds to the values that are within one standard deviation of the mean is
\[Pr \left( \mu -\sigma \le X\le \mu +\sigma \right)=\Pr \left( -1\le \frac{X-\mu }{\sigma }\le 1 \right)=\Pr \left( -1\le Z\le 1 \right)\] \[=\Pr \left( Z\le 1 \right)-\Pr \left( Z\le -1 \right)\approx 0.\text{841345}-0.\text{158655}\approx 0.\text{682689}\]So, the true percentage of values within one standard deviation of the mean is something like 68.2689492%, which still is only an approximation, but this approximation is much better than the 68% stated by the empirical rule.
Similarly, we can compute that
\[\Pr \left( \mu -2\sigma \le X\le \mu +2\sigma \right)=\Pr \left( -2\le \frac{X-\mu }{\sigma }\le 2 \right)=\Pr \left( -2\le Z\le 2 \right)\] \[=\Pr \left( Z\le 2 \right)-\Pr \left( Z\le -2 \right)\approx 0.\text{977249868}-0.0\text{2275}0\text{132}\approx 0.\text{9544997}\]So, the true percentage of values within two standard deviations of the mean is something like 95.4499736% (approximately), but this approximation is much better than the 95% stated by the empirical rule.
Finally, we can compute that
\[\Pr \left( \mu -3\sigma \le X\le \mu +3\sigma \right)=\Pr \left( -3\le \frac{X-\mu }{\sigma }\le 3 \right)=\Pr \left( -3\le Z\le 3 \right)\] \[=\Pr \left( Z\le 3 \right)-\Pr \left( Z\le -3 \right)\approx 0.\text{99865}0\text{1}0\text{2}-0.00\text{1349898}\approx 0.\text{9973}00\text{2}\]So, the true percentage of values within two standard deviations of the mean is approximately something like 99.7300204% but this approximation is still more accurate than the 99.7% stated by the empirical rule.
Caution: Some textbooks won't even say that this is an approximation, and they may say that "68% of the distribution is within one standard deviation of the mean, 95% of the distribution is within two standard deviations of the mean and 99.7% of the distribution is within three standard deviations of the mean", as if that was an exact number. That may cause you confusion because when you make the calculation on Excel (or using normal probability tables from the back of your book), you will find that 68%, 95% and 99.7% are not actually accurate. Make sure you use it in your tests or homework exactly how your instructor told you to do so, but don't forget that it is JUST AN APPROXIMATION.
The Rule of Thumb for the Standard Deviation
This rule is another rough approximation that is used to estimate the standard deviation by using the range. The rule says that the standard deviation can be approximated with the following formula:
\[s\approx \frac{Range}{4}\]Simple. In some cases or applications you won't have access to the data themselves, but you will know the range. If that is the case, all you have to do is take a the range and divide by 4.
Chebyshev's Rule
This is a very fine rule. Well, it is actually an inequality. It is some kind of Empirical Rule, but it applies for ALL distributions (yes, you heard right), not just for the normal distribution. Chebyshev's rule provides a lower bound for the percentage of the distribution that will be within k standard deviations from the mean. Indeed, we have that
\[\Pr \left( \mu -k\sigma \le X\le \mu +k\sigma \right)\ge 1-\frac{1}{{{k}^{2}}}\]What does Chebyshev's rule say for \(k = 2\)? It says
\[\Pr \left( \mu -2\sigma \le X\le \mu +2\sigma \right)\ge 1-\frac{1}{{{2}^{2}}}=0.75\]This is: At least 75% of the distribution is within 2 standard deviations of the mean. Right you say. What is that good for? You may be thinking t hat you knew something much better from the Empirical Rule. Yes, you knew that 95% (or about 95%) of the distribution is within 2 standard deviations of the mean. What does this stinky 75% has to say here. Yeah, the 95% is right, but it works ONLY for normal distributions. The statement that at least 75% of the distribution is within 2 standard deviations of the mean obtained with Chebyshev's rule work for ALL distributions......Enough said.