Randomness and probability — a theoretical reexamination

10 Mar, 2020 at 15:51 | Posted in Statistics & Econometrics | 14 Comments

Modern probabilistic econometrics relies on the notion of probability. To at all be amenable to econometric analysis, economic observations allegedly have to be conceived as random events. But is it really necessary to model the economic system as a system where randomness can only be analyzed and understood when based on an a priori notion of probability?

In probabilistic econometrics, events and observations are as a rule interpreted as random variables as if generated by an underlying probability density function, and a fortiori – since probability density functions are only definable in a probability context – consistent with a probability. As Haavelmo (1944:iii) has it:

For no tool developed in the theory of statistics has any meaning – except , perhaps for descriptive purposes – without being referred to some stochastic scheme.

When attempting to convince us of the necessity of founding empirical economic analysis on probability models, Haavelmo – building largely on the earlier Fisherian paradigm – actually forces econometrics to (implicitly) interpret events as random variables generated by an underlying probability density function.

This is at odds with reality. Randomness obviously is a fact of the real world. Probability, on the other hand, attaches to the world via intellectually constructed models, and a fortiori is only a fact of a probability generating machine or a well constructed experimental arrangement or “chance set-up”.

Just as there is no such thing as a “free lunch,” there is no such thing as a “free probability.” To be able at all to talk about probabilities, you have to specify a model. If there is no chance set-up or model that generates the probabilistic outcomes or events – in statistics one refers to any process where you observe or measure as an experiment (rolling a die) and the results obtained as the outcomes or events (number of points rolled with the die, being e. g. 3 or 5) of the experiment –there strictly seen is no event at all.

Probability is a relational element. It always must come with a specification of the model from which it is calculated. And then to be of any empirical scientific value it has to be shown to coincide with (or at least converge to) real data generating processes or structures – something seldom or never done!

And this is the basic problem with economic data. If you have a fair roulette-wheel, you can arguably specify probabilities and probability density distributions. But how do you conceive of the analogous nomological machines for prices, gross domestic product, income distribution etc? Only by a leap of faith. And that does not suffice. You have to come up with some really good arguments if you want to persuade people into believing in the existence of socio-economic structures that generate data with characteristics conceivable as stochastic events portrayed by probabilistic density distributions!

From a realistic point of view we really have to admit that the socio-economic states of nature that we talk of in most social sciences – and certainly in econometrics – are not amenable to analyze as probabilities, simply because in the real world open systems that social sciences – including econometrics – analyze, there are no probabilities to be had!

The processes that generate socio-economic data in the real world cannot just be assumed to always be adequately captured by a probability measure. And, so, it cannot really be maintained – as in the Haavelmo paradigm of probabilistic econometrics – that it even should be mandatory to treat observations and data – whether cross-section, time series or panel data – as events generated by some probability model. The important activities of most economic agents do not usually include throwing dice or spinning roulette-wheels. Data generating processes – at least outside of nomological machines like dice and roulette-wheels – are not self-evidently best modelled with probability measures.

If we agree on this, we also have to admit that probabilistic econometrics lacks a sound justification. I would even go further and argue that there really is no justifiable rationale at all for this belief that all economically relevant data can be adequately captured by a probability measure. In most real-world contexts one has to argue one’s case. And that is obviously something seldom or never done by practitioners of probabilistic econometrics.

Econometrics and probability are intermingled with randomness. But what is randomness?

In probabilistic econometrics it is often defined with the help of independent trials – two events are said to be independent if the occurrence or nonoccurrence of either one has no effect on the probability of the occurrence of the other – as drawing cards from a deck, picking balls from an urn, spinning a roulette wheel or tossing coins – trials which are only definable if somehow set in a probabilistic context.

But if we pick a sequence of prices – say 2, 4, 3, 8, 5, 6, 6 – that we want to use in an econometric regression analysis, how do we know the sequence of prices is random and a fortiori being able to treat as generated by an underlying probability density function? How can we argue that the sequence is a sequence of probabilistically independent random prices? And are they really random in the sense that is most often applied in probabilistic econometrics – where X is called a random variable only if there is a sample space S with a probability measure and X is a real-valued function over the elements of S?

Bypassing the scientific challenge of going from describable randomness to calculable probability by just assuming it, is of course not an acceptable procedure. Since a probability density function is a “Gedanken” object that does not exist in a natural sense, it has to come with an export license to our real target system if it is to be considered usable.

Among those who at least honestly try to face the problem – the usual procedure is to refer to some artificial mechanism operating in some “games of chance” of the kind mentioned above and which generates the sequence. But then we still have to show that the real sequence somehow coincides with the ideal sequence that defines independence and randomness within our – to speak with science philosopher Nancy Cartwright (1999) – “nomological machine”, our chance set-up, our probabilistic model.

As the originator of the Kalman filter, Rudolf Kalman (1994:143), notes:

Not being able to test a sequence for ‘independent randomness’ (without being told how it was generated) is the same thing as accepting that reasoning about an “independent random sequence” is not operationally useful.

So why should we define randomness with probability? If we do, we have to accept that to speak of randomness we also have to presuppose the existence of nomological probability machines, since probabilities cannot be spoken of – and actually, to be strict, do not at all exist – without specifying such system-contexts (how many sides do the dice have, are the cards unmarked, etc)

If we do adhere to the Fisher-Haavelmo paradigm of probabilistic econometrics we also have to assume that all noise in our data is probabilistic and that errors are well-behaving, something that is hard to justifiably argue for as a real phenomena, and not just an operationally and pragmatically tractable assumption.

Maybe Kalman’s (1994:147) verdict that

Haavelmo’s error that randomness = (conventional) probability is just another example of scientific prejudice

is, from this perspective seen, not far-fetched.

Accepting Haavelmo’s domain of probability theory and sample space of infinite populations– just as Fisher’s (1922:311) “hypothetical infinite population, of which the actual data are regarded as constituting a random sample”, von Mises’ “collective” or Gibbs’ ”ensemble” – also implies that judgments are made on the basis of observations that are actually never made!

Infinitely repeated trials or samplings never take place in the real world. So that cannot be a sound inductive basis for science with aspirations of explaining real-world socio-economic processes, structures or events. It’s not tenable.

As David Salsburg (2001:146) notes on probability theory:

[W]e assume there is an abstract space of elementary things called ‘events’ … If a measure on the abstract space of events fulfills certain axioms, then it is a probability. To use probability in real life, we have to identify this space of events and do so with sufficient specificity to allow us to actually calculate probability measurements on that space … Unless we can identify [this] abstract space, the probability statements that emerge from statistical analyses will have many different and sometimes contrary meanings.

Just as e. g. Keynes (1921) and Georgescu-Roegen (1971), Salsburg (2001:301f) is very critical of the way social scientists – including economists and econometricians – uncritically and without arguments have come to simply assume that one can apply probability distributions from statistical theory on their own area of research:

Probability is a measure of sets in an abstract space of events. All the mathematical properties of probability can be derived from this definition. When we wish to apply probability to real life, we need to identify that abstract space of events for the particular problem at hand … It is not well established when statistical methods are used for observational studies … If we cannot identify the space of events that generate the probabilities being calculated, then one model is no more valid than another … As statistical models are used more and more for observational studies to assist in social decisions by government and advocacy groups, this fundamental failure to be able to derive probabilities without ambiguity will cast doubt on the usefulness of these methods.

Some wise words that ought to be taken seriously by probabilistic econometricians is also given by mathematical statistician Gunnar Blom (2004:389):

If the demands for randomness are not at all fulfilled, you only bring damage to your analysis using statistical methods. The analysis gets an air of science around it, that it does not at all deserve.

Richard von Mises (1957:103) noted that

Probabilities exist only in collectives … This idea, which is a deliberate restriction of the calculus of probabilities to the investigation of relations between distributions, has not been clearly carried through in any of the former theories of probability.

And obviously not in Haavelmo’s paradigm of probabilistic econometrics either. It would have been better if one had heeded von Mises warning (1957:172) that

the field of application of the theory of errors should not be extended too far.

This importantly also means that if you cannot show that data satisfies all the conditions of the probabilistic nomological machine – including randomness – then the statistical inferences used, lack sound foundations!

References

Gunnar Blom et al: Sannolikhetsteori och statistikteori med tillämpningar, Lund: Studentlitteratur.

Cartwright, Nancy (1999), The Dappled World. Cambridge: Cambridge University Press.

Fisher, Ronald (1922), On the mathematical foundations of theoretical statistics. Philosophical Transactions of The Royal Society A, 222.

Georgescu-Roegen, Nicholas (1971), The Entropy Law and the Economic Process. Harvard University Press.

Haavelmo, Trygve (1944), The probability approach in econometrics. Supplement to Econometrica 12:1-115.

Kalman, Rudolf (1994), Randomness Reexamined. Modeling, Identification and Control 3:141-151.

Keynes, John Maynard (1973 (1921)), A Treatise on Probability. Volume VIII of The Collected Writings of John Maynard Keynes, London: Macmillan.

Pålsson Syll, Lars (2007), John Maynard Keynes. Stockholm: SNS Förlag.

Salsburg, David (2001), The Lady Tasting Tea. Henry Holt.

von Mises, Richard (1957), Probability, Statistics and Truth. New York: Dover Publications.

14 Comments

1. > Just as there is no such thing as a “free lunch,” there is no such thing as a “free probability.”
.
Don’t you have to assign a “free probability” of zero to the possibility of a “free lunch”? How can you be so sure that free lunches don’t exist, without using the probability theory you are arguing against?

2. The terms probability/randomness/nomological have been invented by man to “explain” the unexplainable. They are completely abstract notions concocted by a baffled science.
.
There is no machine which can measure probability or randomness. These notions are a figment of the human imagination.
.
To say that an event is random is tantamount to saying it is acausal – that is, it is not the result of some underlying law. When we examinethe behaviour of most observable events or phenomena we see they conform to a set of fixed laws. Why is it there is a class of events which are seemingly acausal? Does not this seem strange?
.
And what is the use of probability theory if it cannot tell us what the outcome of the next event is?

• “When we examinethe behaviour of most observable events or phenomena we see they conform to a set of fixed laws.”
.
Fixed physical laws apply to maybe 4% of the universe. Dark Matter and Dark Energy are unpredicted by fixed laws. So much of experience violates fixed laws, and the assumption is that those seemingly acausal or law-violating phenomena actually conform to the fixed laws, because INSERT HANDWAVING HERE.

3. Robert,
.
“Fixed physical laws apply to maybe 4% of the universe.”
.
Where do you get the 4% from?
.
“Dark Matter and Dark Energy are unpredicted by fixed laws.”
.
So what? Our understanding of these laws is constantly changing.

• “Physicists at CERN use the world’s most powerful particle accelerators and detectors to test the predictions and limits of the Standard Model. Over the years it has explained many experimental results and precisely predicted a range of phenomena, such that today it is considered a well-tested physics theory. But the model only describes the 4% of the known universe”
.
https://home.cern/science/physics
.
“All matter except dark matter is made of molecules”
.
Dark matter is 27% of the universe. See https://home.cern/science/physics/dark-matter
.
” Dark matter seems to outweigh visible matter roughly six to one, making up about 27% of the universe. Here’s a sobering fact: The matter we know and that makes up all stars and galaxies only accounts for 5% of the content of the universe!”
.
The other 68% is Dark Energy, which has been described as “a property of space we do not fully understand.” Assuming that fixed laws, which we just haven’t discovered yet, must govern Dark Matter and Dark Energy is a matter of faith.
.
Assuming fixed laws must exist is as arrogant as epicyclists assuming planets must orbit in circles …

4. Robert,
.
“So much of experience violates fixed laws, and the assumption is that those seemingly acausal or law-violating phenomena actually conform to the fixed laws”
.
Which experience?

• There is no equation of state for water, for example. Whatever fixed rule governs water phases, math cannot express it because water’s triple points violate mathematical assumptions about non-contradiction. Once you have a single contradiction, classical explosion takes place and trivialism becomes valid. Thus the observed phase transitions and triple-points of an abundant substance like water lead to everything being true …
.
I have also personally observed rock formations that are very difficult to describe using geological assumptions that the only forces at work are temperature, pressure, and volume. Layers of rock in quartz are said to have formed in conditions where thermodynamics predicts mixing. For example see https://old.reddit.com/r/geology/comments/fhrtn0/can_someone_explain_these_black_and_white_striped/ ; a typical explanation is found in the comment: “it forms deep underground from high pressure and heat.” But why the banding? Why not uniform mixing? Why do patterns form? The fixed laws of thermodynamics do not predict such natural phenomena. It is a pure guess that some other undiscovered fixed laws do.

5. “Assuming fixed laws must exist is as arrogant as epicyclists assuming planets must orbit in circles …”
.
So is the universe one giant casino?

6. “The fixed laws of thermodynamics do not predict such natural phenomena.”
.
Really?
.
You do misconstrue all kinds of phenomena to suite your argument.
.
The confining pressures are enough to compress and layer the minerals involved but the temperatures are not high enough to have them melt into a indistinct morass.

• “The confining pressures are enough to compress and layer the minerals involved but the temperatures are not high enough to have them melt into a indistinct morass.”
.
But the prevailing story is that the temperature and pressure necessary to metamorphize rock necessarily makes it an indistinct morass, and the bands form out of the melt. But they can’t produce banding in a lab using just temperature and pressure.
.
Can you take hot magma and separate it into bands that range from millimeter or even microscopic scale to much larger (I’ve seen quartz bands meters in width)?
.
Your explanation implies the white and black layers pre-existed the metamorphism? Where in nature do black and white minerals form layers? What fixed law predicts such a phenomenon?
.
“a contention more tenable than one that says the universe operates as a casino.”
.
Strawman. I can tell many other stories other than that the universe is a casino. My favorite geological story involves electricity. Quartz has electrical properties. I’m just saying, the fixed laws you put so much faith in today will likely be looked on as very primitive approximations that miss some other kinds of forces we don’t have sensitive enough instruments to measure yet.
.
If I were to entertain a casino story, it might go something like: in an infinite universe, everything happens. So the fixed laws to which you pledge allegiance may be quite context-sensitive.

• Robert,
.
Your discussion re metamorphism is just nonsense.
.
My casino point is not a strawman. If there are no fixed laws then the universe is akin to a lawless frontier – anything goes.
.
Whilst there are fixed laws, I did not say that we understand them completely, if even at all.
.
And perhaps different laws do apply in different parts of the universe – so what? There are still laws.

• Please see https://www.reference.com/science/gneiss-formed-2182d9ca0ae13a45
.
“The rocks that form gneiss are exposed to extreme pressures and temperatures of between 600 and 700 degrees Celsius. These temperatures cause the individual minerals to migrate, forming distinct bands through the rock.”
.
But if the minerals have different specific gravities, thermodynamics predicts the heavier minerals should form bands at the bottom. However, we see interleaved bands of similar minerals.
.
My point, that you need some other force (electricity?) to account for the banding in gneiss, stands.
.
“There are still laws.”
.
The laws are psychological, fickle, arbitrary. Laws say more about human psychology than about nature. That is my hypothesis after having been exposed to some of the best scientific instruction available; the stories scientists tell remain unconvincing.

7. “Assuming that fixed laws, which we just haven’t discovered yet, must govern Dark Matter and Dark Energy is a matter of faith.”
.
Not so much a matter of faith but a contention, a contention more tenable than one that says the universe operates as a casino.

8. “..the stories scientists tell remain unconvincing..”
.
Again, I ask so what? I agree the laws of the universe may be unknown completely to man, that doesn’t mean there aren’t any. Stop sliding away from the argument.
.
“But if the minerals have different specific gravities, thermodynamics predicts the heavier minerals should form bands at the bottom. ”
.
Minerals might behave this way if they were in a free melt (e.g. igneous complexes) – this is not the case here. Please do proper homework.

Sorry, the comment form is closed at this time.