Variability and Uncertainty are different concepts and should not be used interchangeably.
Variability is a phenomenon in the physical world to be measured, analyzed and where appropriate explained. By contrast uncertainty is an aspect of knowledge. - Sir David Cox
(This quote is taken from: Section 2.4, Risk Analysis: A Quantitative Guide, by David Vose).
Uncertainty and variability are statistically very different, and it is common for them to be kept separate in risk analysis modelling.
Moreover, a "Best Estimate" without associated measures of variability and uncertainty is meaningless.
Let’s look at the example of a Coin vs a Roulette Wheel.
No Uncertainty about Variability
Example 1: Fair coin
When you toss a fair coin N times, you know the all the binomial probabilities exactly. There is no uncertainty in your knowledge about the variability of the coin.
If N equals 100 then:
- The exact mean (of the number of heads) is 50.
- The standard deviation is 5.
- The 100% confidence interval of the mean is [50, 50].
In 95% of experiments of tossing the coin 100 times, the number of heads will be in the interval [40, 60]. The interval [40,60] is a prediction interval (of a random outcome).
If you make the length of the prediction interval [40, 60] shorter, the probability reduces. In other words, we cannot reduce the variability in the coin – no matter how many experiments we do.
(A confidence interval is an interval for a parameter. A prediction interval is an interval for a random outcome).
Now, take a fair roulette wheel with 101 slots numbered from 0 to 100.
- The probability of a particular outcome is 1/101.
- The mean outcome of spinning the roulette wheel is 50, but the standard deviation is 29.
The 100% confidence interval for the mean is [50,50]. However, a 95% prediction interval of a random outcome is [2, 98].
In both the examples above, our knowledge about the variability is perfect and the uncertainty about the variability is zero.
Would you charge the same premium for each risk – given that the mean is the same? Obviously not.
De Moivre’s gamblers ruin problem
If you charge the mean repeatedly (ie for every game) and do not have infinite capital, the probability of ultimate ruin is 1.
The mean alone does not reflect the true risk (just as illustrated in the examples above).
Uncertain knowledge about Variability
Uncertainty is the assessor’s lack of knowledge about the parameters that characterize the physical system that is being modelled.
As soon as the coin or roulette wheel are mutilated, or even if we simply do not know whether it is indeed fair, inferences can only be drawn on the basis of past observed data.
In this case we have uncertainty about the variability.
Suppose in the coin example, before you toss the coin 100 times, you have data based on 10 prior tosses and you happen to observe 5 heads.
Now your estimate of the mean number of heads in 100 future tosses is still 50, but is subject to uncertainty.
Given that the true mean could be larger than 50, or smaller than 50, a 95% prediction interval will necessarily be wider than [40, 60].
Parameter uncertainty increases the width of a prediction interval.
A Best Estimate is meaningless without the associated measurement of variability and uncertainty.