I think there is an element of truth in the view that the superstition that the budget must be balanced at all times [is necessary]. Once it is debunked, [it] takes away one of the bulwarks that every society must have against expenditure out of control. There must be discipline in the allocation of resources or you will have anarchistic chaos and inefficiency. And one of the functions of old fashioned religion was to scare people by sometimes what might be regarded as myths into behaving in a way that the long-run civilized life requires. We have taken away a belief in the intrinsic necessity of balancing the budget if not in every year, [and then] in every short period of time. If Prime Minister Gladstone came back to life he would say “oh, oh what you have done” and James Buchanan argues in those terms. I have to say that I see merit in that view.
My friend Ben says that on the first day he got the following sequence of Heads and Tails when tossing a coin:
H H H H H H H H H H
And on the second day he says that he got the following sequence:
H T T H H T T H T H
Which day-report makes you suspicious?
Most people I ask this question says the first day-report looks suspicious.
But actually both days are equally probable! Every time you toss a (fair) coin there is the same probability (50 %) of getting H or T. Both days Ben makes equally many tosses and every sequence is equally probable!
And in mainstream economics one of the basic assumptions is typically — still — that people make rational choices …
How would you react if a renowned physicist, say Richard Feynman, was telling you that sometimes force is proportional to acceleration and at other times it is proportional to acceleration squared?
I guess you would be unimpressed. But actually, what Dani Rodrik does in Economics Rules amounts to the same strange thing when it comes to theory development and model modification.
In mainstream neoclassical theory preferences are standardly expressed in the form of a utility function. But although the expected utility theory has been known for a long time to be both theoretically and descriptively inadequate, neoclassical economists all over the world gladly continue to use it, as though its deficiencies were unknown or unheard of.
What Dani Rodrik and most other mainstream economists try to do in face of the obvious theoretical and behavioural inadequacies of the expected utility theory, is to marginally mend it. But that cannot be the right attitude when facing scientific anomalies. When models are plainly wrong, you’d better replace them!
As Matthew Rabin and Richard Thaler have it in Risk Aversion:
It is time for economists to recognize that expected utility is an ex-hypothesis, so that we can concentrate our energies on the important task of developing better descriptive models of choice under uncertainty.
In a similar vein, Daniel Kahneman maintains — e. g. in Thinking, Fast and Slow — that expected utility theory is seriously flawed since it doesn’t take into consideration the basic fact that people’s choices are influenced by changes in their wealth. Where standard microeconomic theory assumes that preferences are stable over time, Kahneman and other behavioural economists have forcefully again and again shown that preferences aren’t fixed, but vary with different reference points. How can a theory that doesn’t allow for people having different reference points from which they consider their options have a (typically unquestioned) axiomatic status within economic theory?
Much of what experimental and behavioural economics come up with, is really bad news for mainstream economi theory, and to just conclude, as Rodrik does, that these
insights from social psychology were subsequently applied to many areas of decision making, such as saving behavior, choice of medical insurance, and fertilizer use by poor farmers
sounds, to say the least, somewhat lame, when the works of people like Rabin, Thaler and Kahneman, in reality, show that expected utility theory is nothing but transmogrifying truth.
To Rodrik, mainstream economics is nothing but a smorgasbord of ‘thought experimental’ models. For every purpose you may have, there is always an appropriate model to pick.
But, really, there has to be some limits to the flexibility of a theory!
If you freely can substitute any part of the core and auxuiliary sets of assumptions and still consider that you deal with the same — mainstream, neoclassical or what have you — theory, well, then it’s not at theory, but a chameleon.
The big problem with Rodrik’s cherry-picking view of models is of course that the theories and models presented get totally immunized against all critique. A sure way to get rid of all kinds of ‘anomalies,’ yes, but at a far too high price. So people do not behave optimizing? No problem, we have models that assume satisficing! So people do not maximize expected utility? No problem, we have models that assume … etc., etc..
A theory that accomodates for any observed phenomena whatsoever by creating a new special model for the occasion, and a fortiori having no chance of being tested severely and found wanting, is of little real value.
Jussi — still no. 1!
In Economics Rules, Dani Rodrik maintains that ‘imaginative empirical methods’ — such as game theoretical applications, natural experiments, field experiments, lab experiments, RCTs — can help us to answer questions conerning the external validity of economic models. In Rodrik’s view they are more or less tests of ‘an underlying economic model’ and enable economists to make the right selection from the ever expanding ‘collection of potentially applicable models.’ Writes Rodrik:
Another way we can observe the transformation of the discipline is by looking at the new areas of research that have flourished in recent decades. Three of these are particularly noteworthy: behavioral economics, randomized controlled trials (RCTs), and institutions … They suggest that the view of economics as an insular, inbred discipline closed to the outside influences is more caricature than reality.
I beg to differ. When looked at carefully, there are in fact few real reasons to share Rodrik’s optimism on this ’empirical turn’ in economics.
Field studies and experiments face the same basic problem as theoretical models — they are built on rather artificial conditions and have difficulties with the ‘trade-off’ between internal and external validity. The more artificial conditions, the more internal validity, but also less external validity. The more we rig experiments/field studies/models to avoid the ‘confounding factors’, the less the conditions are reminicent of the real ‘target system.’ You could of course discuss the field vs. experiments vs. theoretical models in terms of realism — but the nodal issue is not about that, but basically about how economists using different isolation strategies in different ‘nomological machines’ attempt to learn about causal relationships. I have strong doubts on the generalizability of all three research strategies, because the probability is high that causal mechanisms are different in different contexts and that lack of homogeneity/stability/invariance doesn’t give us warranted export licenses to the ‘real’ societies or economies.
If we see experiments or field studies as theory tests or models that ultimately aspire to say something about the real ‘target system,’ then the problem of external validity is central (and was for a long time also a key reason why behavioural economists had trouble getting their research results published).
Assume that you have examined how the work performance of Chinese workers A is affected by B (‘treatment’). How can we extrapolate/generalize to new samples outside the original population (e.g. to the US)? How do we know that any replication attempt ‘succeeds’? How do we know when these replicated experimental results can be said to justify inferences made in samples from the original population? If, for example, P(A|B) is the conditional density function for the original sample, and we are interested in doing a extrapolative prediction of E [P(A|B)], how can we know that the new sample’s density function is identical with the original? Unless we can give some really good argument for this being the case, inferences built on P(A|B) is not really saying anything on that of the target system’s P'(A|B).
As I see it is this heart of the matter. External validity/extrapolation/generalization is founded on the assumption that we could make inferences based on P(A|B) that is exportable to other populations for which P'(A|B) applies. Sure, if one can convincingly show that P and P’are similar enough, the problems are perhaps surmountable. But arbitrarily just introducing functional specification restrictions of the type invariance/stability /homogeneity, is, at least for an epistemological realist far from satisfactory. And often it is – unfortunately – exactly this that I see when I take part of mainstream neoclassical economists’ models/experiments/field studies.
By this I do not mean to say that empirical methods per se are so problematic that they can never be used. On the contrary, I am basically — though not without reservations — in favour of the increased use of experiments and field studies within economics. Not least as an alternative to completely barren ‘bridge-less’ axiomatic-deductive theory models. My criticism is more about aspiration levels and what we believe that we can achieve with our mediational epistemological tools and methods in the social sciences.
Many ‘experimentalists’ claim that it is easy to replicate experiments under different conditions and therefore a fortiori easy to test the robustness of experimental results. But is it really that easy? If in the example given above, we run a test and find that our predictions were not correct – what can we conclude? The B ‘works’ in China but not in the US? Or that B ‘works’ in a backward agrarian society, but not in a post-modern service society? That B ‘worked’ in the field study conducted in year 2008 but not in year 2014? Population selection is almost never simple. Had the problem of external validity only been about inference from sample to population, this would be no critical problem. But the really interesting inferences are those we try to make from specific labs/experiments/fields to specific real world situations/institutions/ structures that we are interested in understanding or (causally) to explain. And then the population problem is more difficult to tackle.
The increasing use of natural and quasi-natural experiments in economics during the last couple of decades has led, not only Rodrik, but several other prominent economists to triumphantly declare it as a major step on a recent path toward empirics, where instead of being a deductive philosophy, economics is now increasingly becoming an inductive science.
In randomized trials the researchers try to find out the causal effects that different variables of interest may have by changing circumstances randomly — a procedure somewhat (‘on average’) equivalent to the usual ceteris paribus assumption).
Besides the fact that ‘on average’ is not always ‘good enough,’ it amounts to nothing but hand waving to simpliciter assume, without argumentation, that it is tenable to treat social agents and relations as homogeneous and interchangeable entities.
Randomization is used to basically allow the econometrician to treat the population as consisting of interchangeable and homogeneous groups (‘treatment’ and ‘control’). The regression models one arrives at by using randomized trials tell us the average effect that variations in variable X has on the outcome variable Y, without having to explicitly control for effects of other explanatory variables R, S, T, etc., etc. Everything is assumed to be essentially equal except the values taken by variable X.
In a usual regression context one would apply an ordinary least squares estimator (OLS) in trying to get an unbiased and consistent estimate:
Y = α + βX + ε,
where α is a constant intercept, β a constant ‘structural’ causal effect and ε an error term.
The problem here is that although we may get an estimate of the ‘true’ average causal effect, this may ‘mask’ important heterogeneous effects of a causal nature. Although we get the right answer of the average causal effect being 0, those who are ‘treated'( X=1) may have causal effects equal to – 100 and those ‘not treated’ (X=0) may have causal effects equal to 100. Contemplating being treated or not, most people would probably be interested in knowing about this underlying heterogeneity and would not consider the OLS average effect particularly enlightening.
Limiting model assumptions in economic science always have to be closely examined since if we are going to be able to show that the mechanisms or causes that we isolate and handle in our models are stable in the sense that they do not change when we ‘export’ them to our ‘target systems,’ we have to be able to show that they do not only hold under ceteris paribus conditions and a fortiori only are of limited value to our understanding, explanations or predictions of real economic systems.
Real world social systems are not governed by stable causal mechanisms or capacities. The kinds of ‘laws’ and relations that econometrics has established, are laws and relations about entities in models that presuppose causal mechanisms being atomistic and additive. When causal mechanisms operate in real world social target systems they only do it in ever-changing and unstable combinations where the whole is more than a mechanical sum of parts. If economic regularities obtain they do it (as a rule) only because we engineered them for that purpose. Outside man-made ‘nomological machines’ they are rare, or even non-existant.
I also think that most ‘randomistas’ really underestimate the heterogeneity problem. It does not just turn up as an external validity problem when trying to ‘export’ regression results to different times or different target populations. It is also often an internal problem to the millions of regression estimates that economists produce every year.
Just as econometrics, randomization promises more than it can deliver, basically because it requires assumptions that in practice are not possible to maintain.
Like econometrics, randomization is basically a deductive method. Given the assumptions (such as manipulability, transitivity, separability, additivity, linearity, etc.) these methods deliver deductive inferences. The problem, of course, is that we will never completely know when the assumptions are right. And although randomization may contribute to controlling for confounding, it does not guarantee it, since genuine ramdomness presupposes infinite experimentation and we know all real experimentation is finite. And even if randomization may help to establish average causal effects, it says nothing of individual effects unless homogeneity is added to the list of assumptions. Real target systems are seldom epistemically isomorphic to our axiomatic-deductive models/systems, and even if they were, we still have to argue for the external validity of the conclusions reached from within these epistemically convenient models/systems. Causal evidence generated by randomization procedures may be valid in “closed” models, but what we usually are interested in, is causal evidence in the real target system we happen to live in.
When does a conclusion established in population X hold for target population Y? Only under very restrictive conditions!
‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. “It works there” is no evidence for “it will work here”. Causes deduced in an experimental setting still have to show that they come with an export-warrant to the target population/system. The causal background assumptions made have to be justified, and without licenses to export, the value of ‘rigorous’ and ‘precise’ methods — and ‘on-average-knowledge’ — is despairingly small.
So, no, I find it hard to share Rodrik’s and others enthusiasm and optimism on the value of (quasi)natural experiments and all the statistical-econometric machinery that comes with it. Guess I’m still waiting for the export-warrant …
Taking assumptions like utility maximization or market equilibrium as a matter of course leads to the ‘standing presumption in economics that, if an empirical statement is deduced from standard assumptions then that statement is reliable’ …
The ongoing importance of these assumptions is especially evident in those areas of economic research, where empirical results are challenging standard views on economic behaviour like experimental economics or behavioural finance … From the perspective of Model-Platonism, these research-areas are still framed by the ‘superior insights’ associated with early 20th century concepts, essentially because almost all of their results are framed in terms of rational individuals, who engage in optimizing behaviour and, thereby, attain equilibrium. For instance, the attitude to explain cooperation or fair behaviour in experiments by assuming an ‘inequality aversion’ integrated in (a fraction of) the subjects’ preferences is strictly in accordance with the assumption of rational individuals, a feature which the authors are keen to report …
So, while the mere emergence of research areas like experimental economics is sometimes deemed a clear sign for the advent of a new era … a closer look at these fields allows us to illustrate the enduring relevance of the Model-Platonism-topos and, thereby, shows the pervasion of these fields with a traditional neoclassical style of thought.
Re game theory, yours truly remembers when back in 1991, earning my first Ph.D. with a dissertation on decision making and rationality in social choice theory and game theory, I concluded that
repeatedly it seems as though mathematical tractability and elegance — rather than realism and relevance — have been the most applied guidelines for the behavioural assumptions being made. On a political and social level it is doubtful if the methodological individualism, ahistoricity and formalism they are advocating are especially valid.
This, of course, was like swearing in church. My mainstream neoclassical colleagues were — to say the least — not exactly überjoyed. Listening to what one of the world’s most renowned game theorists — Ariel Rubinstein — has to say on the — rather limited — applicability of game theory in this interview (emphasis added), I basically think he confirms my doubts about how well-founded is Rodrik’s ‘optimism:’
Is game theory useful in a concrete sense or not? … I believe that game theory is very interesting. I’ve spent a lot of my life thinking about it, but I don’t respect the claims that it has direct applications.
The analogy I sometimes give is from logic. Logic is a very interesting field in philosophy, or in mathematics. But I don’t think anybody has the illusion that logic helps people to be better performers in life. A good judge does not need to know logic. It may turn out to be useful – logic was useful in the development of the computer sciences, for example – but it’s not directly practical in the sense of helping you figure out how best to behave tomorrow, say in a debate with friends, or when analysing data that you get as a judge or a citizen or as a scientist …
Game theory is about a collection of fables. Are fables useful or not? In some sense, you can say that they are useful, because good fables can give you some new insight into the world and allow you to think about a situation differently. But fables are not useful in the sense of giving you advice about what to do tomorrow, or how to reach an agreement between the West and Iran. The same is true about game theory …
In general, I would say there were too many claims made by game theoreticians about its relevance. Every book of game theory starts with “Game theory is very relevant to everything that you can imagine, and probably many things that you can’t imagine.” In my opinion that’s just a marketing device …
So — contrary to Rodrik’s optimism — I would argue that although different ’empirical’ approaches have been — more or less — integrated into mainstream economics, there is still a long way to go before economics has become a true empirical science.
According to Dani Rodrik — as argued in Economics Rules — an economic model basically consists of ‘clearly stated assumptions and behavioral mechansisms” that easily lend themselves to mathematical treatment. Furthermore, Rodrik thinks that the usual critique against the use of mathematics in economics is wrong-headed. Math only plays an instrumental role in economic models:
First, math ensures that the elements of a model … are stated clearly and are transparent …
The second virtue of mathematics is that it ensures the internal consistency of a model — simply put, that the conclusions follow from the assumptions.
What is lacking in this overly simplistic view on using mathematical modeling in economics is an ontological reflection on the conditions that have to be fullfilled for appropriately applying the methods of mathematical modeling.
Using formal mathematical modeling, mainstream economists like Rodriik sure can guarantee that the conclusion holds given the assumptions. However, there is no warrant that the validity we get in abstract model worlds automatically transfer to real world economies. Validity and consistency may be good, but it isn’t enough. From a realist perspective both relevance and soundness are sine qua non.
In their search for validity, rigour and precision, mainstream macro modellers of various ilks construct microfounded DSGE models that standardly assume rational expectations, Walrasian market clearing, unique equilibria, time invariance, linear separability and homogeneity of both inputs/outputs and technology, infinitely lived intertemporally optimizing representative household/ consumer/producer agents with homothetic and identical preferences, etc., etc. At the same time the models standardly ignore complexity, diversity, uncertainty, coordination problems, non-market clearing prices, real aggregation problems, emergence, expectations formation, etc., etc.
Behavioural and experimental economics — not to speak of psychology — show beyond any doubts that “deep parameters” — peoples’ preferences, choices and forecasts — are regularly influenced by those of other participants in the economy. And how about the homogeneity assumption? And if all actors are the same – why and with whom do they transact? And why does economics have to be exclusively teleological (concerned with intentional states of individuals)? Where are the arguments for that ontological reductionism? And what about collective intentionality and constitutive background rules?
These are all justified questions – so, in what way can one maintain that these models give workable microfoundations for macroeconomics? Science philosopher Nancy Cartwright gives a good hint at how to answer that question:
Our assessment of the probability of effectiveness is only as secure as the weakest link in our reasoning to arrive at that probability. We may have to ignore some issues to make heroic assumptions about them. But that should dramatically weaken our degree of confidence in our final assessment. Rigor isn’t contagious from link to link. If you want a relatively secure conclusion coming out, you’d better be careful that each premise is secure going on.
In all those economic models that Rodrik praise — where the conclusions follow deductively from the assumptions — mathematics is the preferred means to assure that we get what we want to establish with deductive rigour and precision. The problem, however, is that what guarantees this deductivity are as a rule the same things that make the external validity of the models wanting. The core assumptions (CA), as we have shown in previous posts, are as a rule not very many, and so, if the modellers want to establish ‘interesting’ facts about the economy, they have to make sure the set of auxiliary assumptions (AA) is large enough to enable the derivations. But then — how do we validate that large set of assumptions that gives Rodrik his ‘clarity’ and ‘consistency’ outside the model itself? How do we evaluate those assumptions that are clearly used for no other purpose than to guarantee an analytical-formalistic use of mathematics? And how do we know that our model results ‘travel’ to the real world?
On a deep level one could argue that the one-eyed focus on validity and consistency make mainstream economics irrelevant, since its insistence on deductive-axiomatic foundations doesn’t earnestly consider the fact that its formal logical reasoning, inferences and arguments show an amazingly weak relationship to their everyday real world equivalents. Although the formal logic focus may deepen our insights into the notion of validity, the rigour and precision has a devastatingly important trade-off: the higher the level of rigour and precision, the smaller is the range of real world application. So the more mainstream economists insist on formal logic validity, the less they have to say about the real world. The time is due and over-due for getting the priorities right …
The Münchhausen Trilemma is a term used in epistemology to stress the impossibility to prove any truth even in the fields of logic and mathematics. The name Münchhausen Trilemma was coined by the German philosopher Hans Albert in 1968 in reference to a Trilemma of “dogmatism vs. infinite regress vs. psychologism” used by Karl Popper; it is a reference to the problem of “bootstrapping”, after the story of Baron Münchhausen, pulling himself and the horse on which he was sitting out of a mire by his own hair. (Wikipedia)
In Dani Rodrik’s Economics Rules it is argud that ‘the multiplicity of models is economics’ strength,’ and that a science that has a different model for everything is non-problematic, since
economic models are cases that come with explicit user’s guides — teaching notes on how to apply them. That’s because they are transparent about their critical assumptions and behavioral mechanisms.
That really is at odds with yours truly’s experience from studying mainstream economic models during four decades.
When, e. g., criticizing the basic (DSGE) workhorse macroeconomic model for its inability to explain involuntary unemployment, its defenders maintain that later ‘successive approximations’ and elaborations — especially newer search models — manage to do just that. However, one of the more conspicuous problems with those ‘solutions,’ is that they — as e.g. Pissarides’ ‘Loss of Skill during Unemployment and the Persistence of Unemployment Shocks’ QJE (1992) — are as a rule constructed without seriously trying to warrant that the model immanent assumptions and results are applicable in the real world. External validity is more or less a non-existent problematique sacrificed on the altar of model derivations. This is not by chance. These theories and models do not come at all with the transparent and ‘explicit user’s guides’ that Rodrik maintains they do. And there’s a very obvious reason for that. For how could one even imagine to empirically test assumptions such as Pissarides’ ‘model 1’ assumptions of reality being adequately represented by ”two overlapping generations of fixed size”, ”wages determined by Nash bargaining”, ”actors maximizing expected utility”,”endogenous job openings”, ”jobmatching describable by a probability distribution,” without coming to the conclusion that this is — in terms of realism and relevance — far from ‘good enough’ or ‘close enough’ to real world situations?
Suck on that — and tell me if those typical mainstream neoclassical modeling assumptions in any possibly relevant way — with or without due pragmatic considerations — can be considered anything else but imagined model worlds assumptions that has nothing at all to do with the real world we happen to live in!
Here is no real transparency as to the deeper significance and role of the chosen set of axiomatic assumptions.
Here is no explicit user’s guide or indication of how we should be able to, as Rodrik puts it, ‘discriminate’ between the ‘bewildering array of possibilities’ that flow out of such outlandish and known to be false assumptions.
Theoretical models building on piles of known to be false assumptions are in no way close to being scientific explanations. On the contrary. They are untestable and a fortiori totally worthless from the point of view of scientific relevance.