LARS P. SYLL

Natural experiments in economics

Thad Dunning’s book Natural Experiments in the Social Sciences (CUP 2012) is a very useful guide for economists interested in research methodology in general and natural experiments in specific. Dunning argues that since random or as-if random assignment in natural experiments obviates the need for controlling potential confounders, this kind of “simple and transparent” design-based research method is preferable to more traditional multivariate regression analysis where the controlling only comes in ex post via statistical modelling.

But — there is always a but …

The point of making a randomized experiment is often said to be that it ‘ensures’ that any correlation between a supposed cause and effect indicates a causal relation. This is believed to hold since randomization (allegedly) ensures that a supposed causal variable does not correlate with other variables that may influence the effect.

The problem with that simplistic view on randomization is that the claims made are exaggerated and sometimes even false:

• Even if you manage to do the assignment to treatment and control groups ideally random, the sample selection certainly is — except in extremely rare cases — not random. Even if we make a proper randomized assignment, if we apply the results to a biased sample, there is always the risk that the experimental findings will not apply. What works ‘there,’ does not work ‘here.’ Randomization hence does not ‘guarantee ‘ or ‘ensure’ making the right causal claim. Although randomization may help us rule out certain possible causal claims, randomization per se does not guarantee anything!

• Even if both sampling and assignment are made in an ideal random way, performing standard randomized experiments only give you averages. The problem here is that although we may get an estimate of the ‘true’ average causal effect, this may ‘mask’ important heterogeneous effects of a causal nature. Although we get the right answer of the average causal effect being 0, those who are ‘treated’  may have causal effects equal to -100 and those ‘not treated’ may have causal effects equal to 100. Contemplating being treated or not, most people would probably be interested in knowing about this underlying heterogeneity and would not consider the average effect particularly enlightening.

• There is almost always a trade-off between bias and precision. In real-world settings, a little bias often does not overtrump greater precision. And — most importantly — in case we have a population with sizeable heterogeneity, the average treatment effect of the sample may differ substantially from the average treatment effect in the population. If so, the value of any extrapolating inferences made from trial samples to other populations is highly questionable.

• Since most real-world experiments and trials build on performing a single randomization, what would happen if you kept on randomizing forever, does not help you to ‘ensure’ or ‘guarantee’ that you do not make false causal conclusions in the one particular randomized experiment you actually do perform. It is indeed difficult to see why thinking about what you know you will never do, would make you happy about what you actually do.

• And then there is also the problem that ‘Nature’ may not always supply us with the random experiments we are most interested in. If we are interested in X, why should we study Y only because design dictates that? Method should never be prioritized over substance!

Randomization is not a panacea. It is not the best method for all questions and circumstances. Proponents of randomization make claims about its ability to deliver causal knowledge that are simply wrong. There are good reasons to be sceptical of the now popular — and ill-informed — view that randomization is the only valid and the best method on the market. It is not.

The tools economists use

In their quest for statistical “identification” of a causal effect, economists often have to resort to techniques that answer either a narrower or a somewhat different version of the question that motivated the research.

Results from randomized social experiments carried out in particular regions of, say, India or Kenya may not apply to other regions or countries. A research design exploiting variation across space may not yield the correct answer to a question that is essentially about changes over time …

Economists’ research can rarely substitute for more complete works of synthesis, which consider a multitude of causes, weigh likely effects, and address spatial and temporal variation of causal mechanisms. Work of this kind is more likely to be undertaken by historians and non-quantitatively oriented social scientists …

Economists would not even know where to start without the work of historians, ethnographers, and other social scientists who provide rich narratives of phenomena and hypothesize about possible causes, but do not claim causal certainty.

Economists can be justifiably proud of the power of their statistical and analytical methods. But they need to be more self-conscious about these tools’ limitations. Ultimately, our understanding of the social world is enriched by both styles of research. Economists and other scholars should embrace the diversity of their approaches instead of dismissing or taking umbrage at work done in adjacent disciplines.

Dani Rodrik

As Rodrik notes, ‘ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. Causes deduced in an experimental setting still have to show that they come with an export-warrant to their target populations.

The almost religious belief with which its propagators — like 2019’s ‘Nobel prize’ winners Duflo, Banerjee and Kremer — portray it, cannot hide the fact that randomized controlled trials, RCTs, cannot be taken for granted to give generalisable results. That something works somewhere is no warranty for us to believe it to work for us here or even that it works generally.

Believing there is only one really good evidence-based method on the market — and that randomisation is the only way to achieve scientific validity — blinds people to searching for and using other methods that in many contexts are better. Insisting on using only one tool often means using the wrong tool.

‘Randomistas’ like Duflo et consortes think that economics should be based on evidence from randomised experiments and field studies. They want to give up on ‘big ideas’ like political economy and institutional reform and instead go for solving more manageable problems the way plumbers do. But that modern time ‘marginalist’ approach sure can’t be the right way to move economics forward and make it a relevant and realist science. A plumber can fix minor leaks in your system, but if the whole system is rotten, something more than good old fashion plumbing is needed. The big social and economic problems we face today is not going to be solved by plumbers performing RCTs.

Tony Lawson on economics and social ontology

.

Modern economics has become increasingly irrelevant to the understanding of the real world. In his seminal book Economics and Reality (1997), Tony Lawson traced this irrelevance to the failure of economists to match their deductive-axiomatic methods with their subject

It is — sad to say — as relevant today as it was twenty-five years ago.

It is still a fact that within mainstream economics internal validity is everything and external validity nothing. Why anyone should be interested in that kind of theories and models is beyond imagination. As long as mainstream economists do not come up with any export-licenses for their theories and models to the real world in which we live, they really should not be surprised if people say that this is not science!

In pure mathematics and logic, we do not have to worry about external validity. But economics is not pure mathematics or logic. It’s about society. The real world.

Economics and Reality was a great inspiration to yours truly twenty-five years ago. It still is.

Regeringens sparpolitik bygger på en rad missuppfattningar

När Magdalena Andersson varnar för att Sverige kan hamna i samma läge som de krisande euroländerna visar hon bristande insikt om finanspolitikens grundläggande villkor. Så länge Sverige lånar i sin egen valuta utan fast växelkurs kan Riksbanken avvärja sådana kriser. Det finns ingen osynlig gräns som Sverige plötsligt kan korsa och bli som Grekland bara för att vi ökar vår statsskuld. Skillnaden mellan vår valutaregim och euroländernas är inte kvantitativ utan kvalitativ. Det är inte en gradskillnad utan en utan en artskillnad.

Finansministern varnade också för att en hög statsskuld kunde leda till en repris på den svenska 1990-talskrisen. Men den krisen berodde inte på statsskulden utan på en bostadsbubbla som brast, följd av drastiska räntehöjningar för att försöka behålla kronans fasta växelkurs. Som Magdalena Andersson påpekade i P1 slog 1990-talskrisen hårdast mot de svaga i samhället, men inte för att statsskulden var för hög utan tvärtom för att den på grund av svångremspolitiken minskade alltför fort, och därmed inte kunde agera stötdämpare åt den privata sektorns kollapsande köpkraft …

Det finanspolitiska ramverket sätter ramarna för all politik i Sverige. Frågan om huruvida vi borde behålla det eller inte förtjänar att diskuteras utifrån fakta, inte missförstånd.

Max Jerneck / Dagens Industri

Pourquoi la dette publique n’est pas un problème

.

Who’s afraid of MMT?

As anyone who has ever been responsible for legislative oversight of central bankers knows, they do not like to have their authority challenged. Most of all, they will defend their mystique – that magical aura that hovers over their words, shrouding a slushy mix of banality and baloney in a mist of power and jargon …

In our day, the voices of Modern Monetary Theory perturb the sleep not only of present central bankers, but even of those retired from the role. They prowl the corridors like Lady Macbeth, shouting “Out damn spot!”

Two fresh cases are Raghuram G. Rajan, a former governor of the Reserve Bank of India, and Mervyn King, a former governor of the BOE. In recently published commentaries, each combines bluster and condescension (in roughly equal measure) in a statement of trite truths with which one can, for the most part, hardly disagree.

But Rajan and King each confront MMT only in the abstract. Neither cites or quotes from a single source, and neither names a single person associated with MMT …

What, then, is MMT? Contrary to the claims of King and Rajan, it is not a policy slogan. Rather, it is a body of theory in Keynes’s monetary tradition, which includes such eminent thinkers as the American economist Hyman Minsky and Wynne Godley of the UK Treasury and the University of Cambridge. MMT describes how “modern” governments and central banks actually work, and how changes in their balance sheets are mirrored by changes in the balance sheets of the public – an application of double-entry bookkeeping to economic thought. Thus, as Kelton writes in the plainest English, the deficit of the government is the surplus of the private sector, and vice versa.

MMT shares Keynes’s view that a proper goal of economic policy in a sovereign and developed country is to achieve full employment, buttressed by a guarantee of jobs to all who may need them. This is a goal that I helped write into law in the US under the Humphrey-Hawkins Full Employment and Balanced Growth Act of 1978, along with balanced growth and reasonable price stability. With occasional successes in practice, this policy objective, known as the “dual mandate,” has been the law of the land in the US ever since.

In short, as an example of good economics made popular, accessible, and democratic, MMT represents what central bankers have always feared – as well they might.

James K. Galbraith

Many countries today have deficits. That’s true. But the problem is not the budget deficit. The real deficits are in the climate, healthcare and infrastructure. How do we tackle those deficits? By spending!

MMT rejects the traditional Phillips curve inflation-unemployment trade-off and has a less positive evaluation of traditional policy measures to reach full employment. Instead of a general increase in aggregate demand, it usually prefers more ‘structural’ and directed demand measures with less risk of producing increased inflation. At full employment deficit spendings will often be inflationary, but that is not what should decide the fiscal position of the government. The size of public debt and deficits is not — as already Abba Lerner argued with his ‘functional finance’ theory in the 1940s — a policy objective. The size of public debt and deficits are what they are when we try to fulfil our basic economic objectives — full employment and price stability.

Governments can spend whatever amount of money they want. That does not mean that MMT says they ought to — that’s something our politicians have to decide. No MMTer denies that too much of government spendings can be inflationary. What is questioned is that government deficits necessarily is inflationary.

What is “forgotten” in mainstream macro modelling, is the insight that finance — in all its different shapes — has its own dimension, and if taken seriously, its effect on an analysis must modify the whole theoretical system and not just be added as an unsystematic appendage. Finance is fundamental to our understanding of modern economies and acting like the baker’s apprentice who, having forgotten to add yeast to the dough, throws it into the oven afterward, simply isn’t enough.

All real economic activities nowadays depend on a functioning financial machinery. But institutional arrangements, states of confidence, fundamental uncertainties, asymmetric expectations, the banking system, financial intermediation, loan granting processes, default risks, liquidity constraints, aggregate debt, cash flow fluctuations, etc., etc. — things that play decisive roles in channeling​ money/savings/credit — are more or less left in the dark in modern macro theoretical formalizations.

Är hög statsskuld — verkligen — problemet?

.

När depressionen drabbade 1930-talets industrivärld visade sig den ekonomiska teorin inte vara till någon större hjälp att komma ur situationen. Den engelske nationalekonomen John Maynard Keynes såg behovet av att utveckla en ny teori som på bröt mot den etablerade sanningen. I The General Theory of Employment, Interest and Money (1936) presenterade han sitt alternativ.

Vad som behövs nu är upplyst handling grundad på relevant och realistisk ekonomisk teori av det slag som Keynes står för.

Den överhängande faran är att vi inte får fart på konsumtion och kreditgivning. Förtroende och effektiv efterfrågan måste återupprättas.

Ett av de grundläggande feltänken i dagens diskussion om statsskuld och budgetunderskott är att man inte skiljer på skuld och skuld. Även om det på makroplanet av nödvändighet är så att skulder och tillgångar balanserar varandra, så är det inte oväsentligt vem som har tillgångarna och vem som har skulderna.

Länge har man varit motvillig att öka de offentliga skulderna eftersom ekonomiska kriser i mångt och mycket fortfarande uppfattas som förorsakad av för mycket skulder. Men det är här fördelningen av skulder kommer in. Om staten i ett läge med risk för skulddeflation och likviditetsfällor ‘lånar’ pengar för att bygga ut järnvägar, skola och hälsovård, så är ju de samhälleliga kostnaderna för detta minimala eftersom resurserna annars legat oanvända. När hjulen väl börjar snurra kan både de offentliga och de privata skulderna betalas av. Och även om detta inte skulle uppnås fullt ut så förbättras det ekonomiska läget därför att låntagare med dåliga balansräkningar ersätts med de som har bättre.

I stället för att ”värna om statsfinanserna” bör man se till att värna om samhällets framtid. Problemet med en statsskuld i en situation med nästintill negativa räntor inte är att den är för stor, utan för liten.