The insufficiency of validity

14 Mar, 2023 at 11:14 | Posted in Theory of Science & Methodology | 2 Comments

Mainstream economics is at its core in the story-telling business whereby economic theorists create make-believe analogue models of the target system – usually conceived as the real economic system. This modelling activity is considered useful and essential. Since fully-fledged experiments on a societal scale as a rule are prohibitively expensive, ethically indefensible or unmanageable, economic theorists have to substitute experimenting with something else. To understand and explain relations between different entities in the real economy the predominant strategy is to build models and make things happen in these “analogue-economy models” rather than engineering things happening in real economies.

Formalistic deductive “Glasperlenspiel” can be very impressive and seductive. But in the realm of science, it ought to be considered of little or no value to simply make claims about the model and lose sight of reality. As Julian Reiss writes:

There is a difference between having evidence for some hypothesis and having evidence for the hypothesis relevant for a given purpose. The difference is important because scientific methods tend to be good at addressing hypotheses of a certain kind and not others: scientific methods come with particular applications built into them … The advantage of mathematical modelling is that its method of deriving a result is that of mathemtical proof: the conclusion is guaranteed to hold given the assumptions. However, the evidence generated in this way is valid only in abstract model worlds while we would like to evaluate hypotheses about what happens in economies in the real world … The upshot is that valid evidence does not seem to be enough. What we also need is to evaluate the relevance of the evidence in the context of a given purpose.

Proving things about thought-up worlds is not enough. To have valid evidence is not enough. A deductive argument is valid if it makes it impossible for the premises to be true and the conclusion to be false. Fine, but what we need in science is sound arguments — arguments that are both valid and whose premises are all actually true.

Theories and models being ‘coherent’ or ‘consistent’ with data do not make the theories and models success stories. We want models to somehow represent their real-world targets. This representation can not be complete in most cases because of the complexity of the target systems. That kind of incompleteness is unavoidable. But it’s a  totally different thing when models misrepresent real-world targets. Aiming only for validity, without soundness, is setting the economics aspirations level too low for developing a realist and relevant science.