## Ergodicity and the law of large numbers (wonkish)

1 March, 2016 at 08:51 | Posted in Statistics & Econometrics | 9 CommentsIf n identical trials A occurs v times, and if n is very large, then v/n should be near the probability p of A …This is one form of the

law of large numbersand serves as a basis for the intuitive notion of probability as a measure of relative frequencies …It is usual to read into the law of large numbers things which it definitely does not imply. If Peter and Paul toss a perfect coin 10 000 times, it is customary to expect that Peter will be in the lead roughly half the time.

This is not true. In a large number ofdifferentcoin-tossing games it is reasonable to expect that at any fixed moment heads will be in the lead in roughly half of all cases. But it is quite likely that the player who ends at the winning side has been in the lead for practically the whole duration of the game. Thus contrary to widespread belief, thetime averagefor any individual game has nothing to do with theensemble averageat any given moment.

When giving my yearly PhD course in statistics at Malmö University, Feller’s book is as self-evident a reference as when I started my own statistics studies forty years ago.

## 9 Comments »

RSS feed for comments on this post. TrackBack URI

### Leave a Reply

Blog at WordPress.com.

Entries and comments feeds.

I actually find this a quite confusing example. Take the statement:

.

“If Peter and Paul toss a perfect coin 10 000 times, it is customary to expect that Peter will be in the lead roughly half the time. This is not true.”

.

and

.

“But it is quite likely that the player who ends at the winning side has been in the lead for practically the whole duration of the game.”

.

My question is: How do you know that that is Peter? Isn’t Paul as likely to be the winner? And if he is, isn’t it correct to assume the Peter will be in the lead roughly half of the time?

.

Of course, this changes if we say “Conditional on knowing that Peter wins the game, he is more likely than Paul to be in the lead at any given point in time”. But that’s an obvious statement, and can easily be calculated in a game of only two coin tosses (conditional on Peter winning, the probability he is in the lead after one coin toss is 3/5).

Comment by pontus— 1 March, 2016 #

Possibly Feller was trying to say was that although the ‘mathematical expectation’ is that Peter will be in the lead about half the time, he will very rarely find that he has been in the lead about half the time. He will much more often have had a long run in the lead, or a long run behind. (I’m not sure even this is as clear as it should be – I sympathize with Feller.)

Comment by Dave Marsay— 1 March, 2016 #

That sounds like the statement is about being in the lead/behind is bimodal.

Comment by pontus— 1 March, 2016 #

Yes. I think Laplace observed how easy it was to get confused in such circumstances: I know that being in the lead about half the time is unlikely, but I expect it!

Comment by Dave Marsay— 1 March, 2016 #

3/5?

Comment by Lars Syll— 1 March, 2016 #

Peter can win in five ways: 2-0, 2-1 (x2), and 1-0 (x2). Out of these he must be in the lead (i.e. up 1-0 after the first toss) in the 2-0 case, in one of the 2-1 cases, and in one of the 1-0 cases.

That is, conditional on Peter winning, the probability he is in the lead after one round is 3/5.

Comment by pontus— 1 March, 2016 #

It is more ‘intuitive’ to think of it in terms of Peter betting on (say) Heads coming up and that he wins the two coins game if both coins show Heads. Of the four possible ‘outcomes’ (HH, HT, TH,TT) that means ‘conditional on Peter winning’ the probability he is in the lead after one toss is 1 — confirming even stronger Feller’s point that it is “quite likely that the player who ends at the winning side has been in the lead for practically the whole duration of the game.”

Comment by Lars Syll— 1 March, 2016 #

So you mean in Feller’s game, Peter wins if he has had 10,000 heads in a row? Yeah, if that the game, conditional on Peter winning, he has been in the lead for the entire duration of the game with probability one.

.

I don’t find that particularly enlightening or deep.

Comment by pontus— 1 March, 2016 #

Hmmm …

You wrote in your earlier comment: “But that’s an obvious statement, and can easily be calculated in a game of only two coin tosses (conditional on Peter winning, the probability he is in the lead after one coin toss is 3/5).”

So, whether deep or enlightening, I think be just drop the ‘discussion’ here.

Comment by Lars Syll— 1 March, 2016 #