## Ergodicity and the law of large numbers (wonkish)

1 March, 2016 at 08:51 | Posted in Statistics & Econometrics | 9 Comments

If n identical trials A occurs v times, and if n is very large, then v/n should be near the probability p of A …This is one form of the law of large numbers and serves as a basis for the intuitive notion of probability as a measure of relative frequencies …

It is usual to read into the law of large numbers things which it definitely does not imply. If Peter and Paul toss a perfect coin 10 000 times, it is customary to expect that Peter will be in the lead roughly half the time. This is not true. In a large number of different coin-tossing games it is reasonable to expect that at any fixed moment heads will be in the lead in roughly half of all cases. But it is quite likely that the player who ends at the winning side has been in the lead for practically the whole duration of the game. Thus contrary to widespread belief, the time average for any individual game has nothing to do with the ensemble average at any given moment.

When giving my yearly PhD course in statistics at Malmö University, Feller’s book is as self-evident a reference as when I started my own statistics studies forty years ago.

1. I actually find this a quite confusing example. Take the statement:
.
“If Peter and Paul toss a perfect coin 10 000 times, it is customary to expect that Peter will be in the lead roughly half the time. This is not true.”
.
and
.
“But it is quite likely that the player who ends at the winning side has been in the lead for practically the whole duration of the game.”
.
My question is: How do you know that that is Peter? Isn’t Paul as likely to be the winner? And if he is, isn’t it correct to assume the Peter will be in the lead roughly half of the time?
.
Of course, this changes if we say “Conditional on knowing that Peter wins the game, he is more likely than Paul to be in the lead at any given point in time”. But that’s an obvious statement, and can easily be calculated in a game of only two coin tosses (conditional on Peter winning, the probability he is in the lead after one coin toss is 3/5).

• Possibly Feller was trying to say was that although the ‘mathematical expectation’ is that Peter will be in the lead about half the time, he will very rarely find that he has been in the lead about half the time. He will much more often have had a long run in the lead, or a long run behind. (I’m not sure even this is as clear as it should be – I sympathize with Feller.)

• That sounds like the statement is about being in the lead/behind is bimodal.

• Yes. I think Laplace observed how easy it was to get confused in such circumstances: I know that being in the lead about half the time is unlikely, but I expect it!

• 3/5?

• Peter can win in five ways: 2-0, 2-1 (x2), and 1-0 (x2). Out of these he must be in the lead (i.e. up 1-0 after the first toss) in the 2-0 case, in one of the 2-1 cases, and in one of the 1-0 cases.

That is, conditional on Peter winning, the probability he is in the lead after one round is 3/5.

2. It is more ‘intuitive’ to think of it in terms of Peter betting on (say) Heads coming up and that he wins the two coins game if both coins show Heads. Of the four possible ‘outcomes’ (HH, HT, TH,TT) that means ‘conditional on Peter winning’ the probability he is in the lead after one toss is 1 — confirming even stronger Feller’s point that it is “quite likely that the player who ends at the winning side has been in the lead for practically the whole duration of the game.”

• So you mean in Feller’s game, Peter wins if he has had 10,000 heads in a row? Yeah, if that the game, conditional on Peter winning, he has been in the lead for the entire duration of the game with probability one.
.
I don’t find that particularly enlightening or deep.

• Hmmm …
You wrote in your earlier comment: “But that’s an obvious statement, and can easily be calculated in a game of only two coin tosses (conditional on Peter winning, the probability he is in the lead after one coin toss is 3/5).”
So, whether deep or enlightening, I think be just drop the ‘discussion’ here.