Sunday, January 06, 2008

Things I learnt from William Feller

1. The trial of 1 coin thrown 1000 times is different from the trial of 1000 coins in one instant. That is, given the results (statistical characteristics) of both experiments, one can deduce if its the former or the latter.

Reason:
The laws governing prolonged series of individual observation (random walk) is entirely different from the laws derived for a whole population (law of large numbers).
Time average does not have bounded expectation and hence DOES NOT obey Law of Large Number (LLN). Instead it has its own set of "arc sine" laws strictly for waiting times.Ensemble average has bounded expectation and hence obey LLN.

It sounds absolutely preposterous. I must admit the math on LLN and CLT (Central Limit Theorem) get a bit too dense for me here, but please read Feller thoroughly before dismissing the idea. Perhaps ensemble average lacks the dimension of time--that's what makes it different.

2. Given a fixed probability average (which is reasonable if we apply Laws of Large Numbers), increased uniformity in each independent Bernoulli trial increases the variances. For example, given a certain quality p of n machines, the output will be the least uniform if all machines are uniformly equal.

Given a probability distribution of Bernoulli trials with variable probability,




To summarise: Set of the wildly fluctuating {0.1, 0.4, 0.7} is good. Set of the more uniform {0.4, 0.4, 0.4} is no good.

Another shocking revelation. I hope the Six Sigma folks are aware of this fact. I would like to venture forth an analogy from thermodynamics: Entropy of a system is at its maximum when the system is isothermally uniform. Similarly, variance of a system is at its maximum when every point within the system is uniform. Makes one think about our intuitive understanding of the word "variance".

What is Entropy? by Erwin Shrödinger

3. All it takes is 23 people to make it more likely that at least 2 of them share the same birthday than otherwise.

I suspect people who share the same birthdays have an instant affinity to each other because of the perceived rarity of such events. But I for sure won't flinch again if another May girl comes along.

4. German bombs felled over London were found to be perfectly random and homogeneous, despite apparent evidences of some areas being more heavily bombed.

Reason: to the untrained eye, randomness appears as regularity or tendency to cluster.

Anyway, that's what the chi-square tests are for--to determine if the pattern we are seeing is indeed an anomalous pattern or a good fit to the Poisson or Normal distributions, which are essentially random.

5. Given a sampling of German planes and their number plates, statisticians guesstimated the total enemy plane productions in World War 2.

Assumption: the number plates were given sequentially.

6. Even if a game is fair, where the expectation of winning (per trial) = entrance fee (per trial), there is nothing in the law of large numbers to prevent you from losing money with a probability of 1. (= sure lose).
(LLN only says your loss is limited to a magnitude less than n).

Assumption: we are dealing with random variables with divergent expectations (eg random walks)

7. We live in a world of no advance knowledge. For example, even the prospect of the sun rising tomorrow is subject to the conditional probability below:

P( sun will rise tomorrow | sun has risen for past observable 1826213 days)
= n+1/n+2 (Law of Succession of Laplace)
= 1826214 / 1826215
= 0.999999

Assume that we have no prior knowledge of the motions of the celestial bodies that causes the phenomenon we call "sunrise".

Wiki on Sunrise Problem

Proof. Bear with me on this hypothetical situation. Imagine there are 20 parallel universes, each created by a supremely bored Creator who, for pure amusement, determine the lifespan of each universe he created by drawing a ball from an urn containing red balls and white balls for each universe.A red ball drawn denotes the survival of the sun for that year. A white ball drawn means the sun must be extinguished by that year. The 1st universe has an urn of 0 red and 19 white balls. The 2nd universe has an urn of 1 red and 18 white balls and so on... Hence, each universe has different likelihood of being extinguished, with the 1st universe most likely to be dead, and the 20th universe least likely (in fact, impossible, since there are 19 red balls, and 0 white balls, are contained in its urn).

Say, human beings live on one of the universes, but they have no idea which universe they belong to. They have thus far survived for 10 years (i.e. 10 red balls have been drawn).

Let total number of balls in each urn be N.
Let total number of universes be N+1.
Let current number of years the Universe has been around be 10.

The probability of 10 red balls drawn = P(A) =
P(10 red balls from Universe 1).P(Universe 1) + P(10 red balls from Universe 2).P(Universe 2) + ...+ P(10 red balls from Universe 20).P(Universe 20)

The probability of the 11th ball being drawn is red = P(B) = 1/12.
Hence, the probability of the 11th ball being drawn is red, given that the 1st 10 balls drawn are red
=P(B|A) = P(AB) / P(A)
=P(B) / P(A) (as P(AB) = P(B)...think about this, they are mathematically equivalent)
=(1/12) / (1/11)
= 11/12
= (n+1) / (n+2) where n = number of successful observations

Note the interesting and immensely useful approximation:
which works as the Riemann rectangles are being approximated (though underestimated) by a continuous curve x^n, much like how the binomial distribution is approximated by the normal curve. The error becomes smaller as n goes to infinity.

Of course Feller warned that such ideas were already discredited, and could very well be labelled pseudo-science. But the mathematical developments of hyperspace have thrown up the distinct possibility of parallel universes co-existing with our own. Maybe ours might just be the one whose sun goes out tomorrow. Anyway, Pink Martini seems to agree:

If tomorrow's sun doesn't shine,
And no creatures stir in the morning time,
If the clouds go still in the sky,
at least I'll have my Clementine.

If tomorrow's moon doesn't show,
And our dreams go lost in the winter snow,
If the flowers wither and die,
at least I'll have my Clementine.

Download clip sung by China Forbes here

No comments: