What is this thing called probability?

19 Jun, 2023 at 17:24 | Posted in Statistics & Econometrics | 2 Comments

Fitting a model that has a parameter called ‘probability’ to data does not mean that the estimated value of that parameter estimates the probability of anything in the real world. Just as the map is not the territory, the model is not the phenomenon, and calling something ‘probability’ does not make it a probability, any more than drawing a mountain on a map creates a real mountain …

In summary, the word ‘probability’ is often used with little thought about why, if at all, the term applies, and many common uses of the word are rather removed from anything in the real world that can be reasonably described or modeled as random.

Philip Stark

Modern probabilistic econometrics relies on the notion of probability. To at all be amenable to econometric analysis, economic observations allegedly have to be conceived as random events.

But is it really necessary to model the economic system as a system where uncertainty and randomness can only be analyzed and understood when based on an a priori notion of probability?

In probabilistic econometrics, events and observations are as a rule interpreted as random variables as if generated by an underlying probability density function, and, a fortiori, since probability density functions are only definable in a probability context, consistent with a probability. Attempting to convince us of the necessity of founding empirical economic analysis on probability models actually forces econometrics to (implicitly) interpret events as random variables generated by an underlying probability density function.

This is at odds with reality. Randomness and uncertainty obviously are facts of the real world. Probability, on the other hand, attaches to the world via intellectually constructed models, and a fortiori is only a fact of a probability-generating machine or a well-constructed (randomized) experimental arrangement or ‘chance set-up-‘

Just as there is no such thing as a ‘free lunch,’ there is no such thing as a ‘free probability.’ To be able at all to talk about probabilities, you have to specify a model. If there is no chance set-up or model that generates the probabilistic outcomes or events — in statistics, one refers to any process where you observe or measure as an experiment (rolling a die) and the results obtained as the outcomes or events (number of points rolled with the die, being e. g. 3 or 5) of the experiment — there strictly seen is no event at all.

Probability is a relational element. It always must come with a specification of the model from which it is calculated. And then to be of any empirical scientific value it has to be shown to coincide with (or at least converge to) real data-generating processes or structures — something seldom or never done!

And this is the basic problem with economic data. If you have a fair roulette wheel, you can arguably specify probabilities and probability density distributions. But how do you conceive of the analogous nomological machines for prices, gross domestic product, income distribution etc? Only by a leap of faith. And that does not suffice. You have to come up with some really good arguments if you want to persuade people into believing in the existence of socioeconomic structures that generate data with characteristics conceivable as stochastic events portrayed by probabilistic density distributions!

From a realistic point of view we have to admit that the socio-economic states of nature that we talk of in most social sciences — and certainly in econometrics — are not amenable to analyze as probabilities, simply because in the real-world open systems that social sciences — including econometrics — analyze, there are no probabilities to be had!

The processes that generate socio-economic data in the real world cannot just be assumed to always be adequately captured by a probability measure. And, so, it cannot be maintained — as in the paradigm of probabilistic econometrics — that it even should be mandatory to treat observations and data — whether cross-section, time series or panel data — as events generated by some probability model. The important activities of most economic agents do not usually include throwing dice or spinning roulette wheels. Data-generating processes — at least outside of nomological machines like dice and roulette wheels — are not self-evidently best modelled with probability measures.

If we agree on this, we also have to admit that probabilistic econometrics lacks sound foundations. I would even go further and argue that there really is no justifiable rationale at all for this belief that all economically relevant data can be adequately captured by a probability measure. In most real-world contexts one has to argue one’s case. And that is obviously something seldom or never done by practitioners of probabilistic econometrics.