Ergodicity and the law of large numbers (wonkish)1 March, 2016 at 08:51 | Posted in Statistics & Econometrics | 9 Comments
If n identical trials A occurs v times, and if n is very large, then v/n should be near the probability p of A …This is one form of the law of large numbers and serves as a basis for the intuitive notion of probability as a measure of relative frequencies …
It is usual to read into the law of large numbers things which it definitely does not imply. If Peter and Paul toss a perfect coin 10 000 times, it is customary to expect that Peter will be in the lead roughly half the time. This is not true. In a large number of different coin-tossing games it is reasonable to expect that at any fixed moment heads will be in the lead in roughly half of all cases. But it is quite likely that the player who ends at the winning side has been in the lead for practically the whole duration of the game. Thus contrary to widespread belief, the time average for any individual game has nothing to do with the ensemble average at any given moment.
When giving my yearly PhD course in statistics at Malmö University, Feller’s book is as self-evident a reference as when I started my own statistics studies forty years ago.