The experimental dilemma

31 Oct, 2019 at 15:43 | Posted in Economics | 1 Comment

resissWe can either let theory guide us in our attempt to estimate causal relationships from data … or we don’t let theory guide us. If we let theory guide us, our causal inferences will be ‘incredible’ because our theoretical knowledge is itself not certain … If we do not let theory guide us, we have no good reason to believe that our causal conclusions are true either of the experimental population or of other populations because we have no understanding of the mechanisms that are responsible for a causal relationship to hold in the first place, and it is difficult to see how we could generalize an experimental result to other settings if this understanding doesn’t exist. Either way, then, causal inference seems to be a cul-de-sac.

Nowadays many mainstream economists maintain that ‘imaginative empirical methods’ — especially randomized experiments (RCTs) — can help us to answer questions concerning the external validity of economic models. In their view, they are, more or less, tests of ‘an underlying economic model’ and enable economists to make the right selection from the ever-expanding ‘collection of potentially applicable models.’

It is widely believed among economists that the scientific value of randomization — contrary to other methods — is totally uncontroversial and that randomized experiments are free from bias. When looked at carefully, however, there are in fact few real reasons to share this optimism on the alleged ’experimental turn’ in economics. Strictly seen, randomization does not guarantee anything.

‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. ‘It works there’ is no evidence for ‘it will work here’. Causes deduced in an experimental setting still have to show that they come with an export-warrant to the target population. The causal background assumptions made have to be justified, and without licenses to export, the value of ‘rigorous’ and ‘precise’ methods — and ‘on-average-knowledge’ — is despairingly small.

the-right-toolThe almost religious belief with which its propagators — this year’s ‘Nobel prize’ winners Duflo, Banerjee and Kremer included — portray it, cannot hide the fact that RCTs cannot be taken for granted to give generalizable results. That something works somewhere is no warranty for us to believe it to work for us here or even that it works generally.

The present RCT idolatry is dangerous. Believing there is only one really good evidence-based method on the market — and that randomization is the only way to achieve scientific validity — blinds people to searching for and using other methods that in many contexts are better. RCTs are simply not the best method for all questions and in all circumstances. Insisting on using only one tool often means using the wrong tool.

Robert Lucas coming out as a closet Keynesian

31 Oct, 2019 at 11:24 | Posted in Economics | 11 Comments

In his Keynote Address to the 2003 History of Political Economy Conference, Nobel laureate Robert Lucas said:

Well, I’m not here to tell people in this group about the history of
monetary thought. I guess I’m here as a kind of witness from a vanished
culture, the heyday of Keynesian economics. It’s like historians rushing
to interview the last former slaves before they died, or the last of the
people who remembered growing up in a Polish shtetl. I am going to tell
you what it was like growing up in a day when Keynesian economics
was taught as a solid basis on which macroeconomics could proceed.

keynesdanceMy credentials? Was I a Keynesian myself? Absolutely. And does my Chicago training disqualify me for that? No, not at all. David Laidler
[who was present at the conference] will agree with me on this, and I will explain in some detail when I talk about my education. Our Keynesian
credentials, if we wanted to claim them, were as good as could be obtained in any graduate school in the country in 1963.

I thought when I was trying to prepare some notes for this talk that people attending the conference might be arguing about Axel Leijonhufvud’s thesis that IS-LM was a distortion of Keynes, but I didn’t
really hear any of this in the discussions this afternoon. So I’m going to think about IS-LM and Keynesian economics as being synonyms. I remember when Leijonhufvud’s book2 came out and I asked my colleague Gary Becker if he thought Hicks had got the General Theory right with
his IS-LM diagram. Gary said, “Well, I don’t know, but I hope he did, because if it wasn’t for Hicks I never would have made any sense out of that damn book.” That’s kind of the way I feel, too, so I’m hoping Hicks got it right.

Mirabile dictu! I’m a Keynesian — although I haven’t understood anything of what Keynes wrote, but I’ve read another guy who said he had read his book, so I hope for the best and assume he got it right (which Hicks actually didn’t, and was intellectually honest to admit in at least three scientific publications published about twenty years before Lucas statement). In truth a very scientific attitude. No wonder the guy after having deluded himself into believing (?) being a Keynesian — although actually only elaborating upon a model developed and then disowned by John Hicks — got the ‘Nobel prize’ in economics …

Si j’essaie ici de le décrire, c’est afin de ne pas l’oublier

31 Oct, 2019 at 11:18 | Posted in Varia | Comments Off on Si j’essaie ici de le décrire, c’est afin de ne pas l’oublier

12123684298Mais, bien sûr, nous qui comprenons la vie, nous nous moquons bien des numéros ! J’aurais aimé commencer cette histoire à la façon des contes de fées. J’aurais aimé dire:

“Il était une fois un petit prince qui habitait une planète à peine plus grande que lui, et qui avait besoin d’un ami…” Pour ceux qui comprennent la vie, ça aurait eu l’air beaucoup plus vrai.

Car je n’aime pas qu’on lise mon livre à la légère. J’éprouve tant de chagrin à raconter ces souvenirs. Il y a six ans déjà que mon ami s’en est allé avec son mouton. Si j’essaie ici de le décrire, c’est afin de ne pas l’oublier. C’est triste d’oublier un ami. Tout le monde n’a pas eu un ami. Et je puis devenir comme les grandes personnes qui ne s’intéressent plus qu’aux chiffres.

L’incroyable voyage de Shackleton

29 Oct, 2019 at 20:12 | Posted in Varia | Comments Off on L’incroyable voyage de Shackleton

In science, courage is to follow the motto of enlightenment and Kant’s dictum — Sapere Aude!  To use your own understanding, having the ​courage to think for yourself and question ‘received opinion,’ authority or orthodoxy.

In our daily lives, courage is a capability to confront fear, as when in front of the powerful and mighty, not to step back, but stand up for one’s rights not to be humiliated or abused.

Courage is to do the right thing in spite of danger and fear.

As when Ernest Shackleton, in April 1916, aboard the small boat ‘James Caird’, spent 16 days crossing 1,300 km of ocean to reach South Georgia, then trekked across the island to a whaling station, and finally could rescue the remaining men from the crew of ‘Endurance’ left on the Elephant Island. Not a single member of the expedition died.

What we do in life echoes in eternity.


29 Oct, 2019 at 16:32 | Posted in Varia | Comments Off on Feel


Paolo Conte – Via Con Me

29 Oct, 2019 at 16:18 | Posted in Varia | Comments Off on Paolo Conte – Via Con Me


Where economics went wrong

29 Oct, 2019 at 11:27 | Posted in Economics | 3 Comments

colDavid Colander and Craig Freedman’s Where Economics Went Wrong is a provocative book designed to inspire economists to serious reflection on the nature of economics and how it is practiced. It is a book to that seeks to stimulate discussion about the current state of the discipline; it should be read by anyone who categorizes what they do as applied policy work. I agree with much – though not all – of what Colander and Freedman’s write … Reliance on mathematics has obscured much of the assumed structure that economists work from, leaving us unable to clearly articulate assumptions or identify our often normative precepts. Adoption of the scientific method has resulted in the belief that economic theory can deliver useful, practical knowledge. However, this belief has not been tempered by a corresponding understanding of the limits of theory in a complex world where people do not always behave as rational actors, but are influenced by culture, society, history, and government structure. In this review essay, I explore some aspects of the Chicago-School story to illustrate why shifting the profession to Colander and Freedman’s vision of a Classical liberal attitude is likely to be a difficult task – and why the effort is valuable.

Marianne Johnson

A book well worth reading, although yours truly have to confess of not being convinced that it really is possible — or even desirable — to separate ‘positive economics’ from ‘normative economics’. With the background of this year’s ‘Nobel prize’ in economics, it would certainly also have been interesting to evaluate what the ‘randomistas’ revolution with its (alleged) abandonment of theory for experiments means for the authors’ thesis on the separation between ‘science’ (theory) and its ‘application’ (policy).

What do RCTs reveal about causality?

28 Oct, 2019 at 12:26 | Posted in Economics | 4 Comments

rct-gold-standardThe insight critique contested the proposition that RCTs had revealed significant new facts or provided new understanding of development processes … But closer inspection reveals that they most often merely provide a validation of common sense. Whereas at times randomization seemed to reveal something surprising … in other instances it simply told us what had been long expected … One such finding—that providing preventative public health treatments at low or no cost, or better yet with incentives, leads to an increase in the number of people willing to accept them—is cited by the prize committee as having led to a change in the received wisdom in favor of user fees in primary health. This gets the history quite wrong, since such fees had long before that lost favor …

RCTs cannot reveal very much about causal processes since at their core they are designed to determine whether something has an effect, not how. The randomistas have attempted to deal with this charge by designing studies to interpret whether variations in the treatment have different effects, but this requires a prior conception of what the causal mechanisms are. The lack of understanding of causation can limit the value of any insights derived from RCTs in understanding economic life or in designing further policies and interventions. Ultimately, the randomistas tested what they thought was worth testing, and this revealed their own preoccupations and suppositions, contrary to the notion that they spent countless hours listening to and in close contact with the poor …

If RCTs now “entirely dominate” development economics, or worse, provide the basis for development policymaking, that is no cause for celebration. The roaring success of the randomistas tells us most of all about the historical moment in which they came to prominence: one in which defeatism or cynicism about public initiatives on a larger scale has been replaced by a focus on what works at the level of individuals and communities. But even there, what does work, really, remains an open question. The difficult question of how to fix broken institutions and help societies function better requires going beyond a biomedical metaphor of taking the right pill. Nobel or not, the debate must continue.

Sanjay Reddy

The problem many ‘randomistas’ — like this year’s ‘Nobel prize’ winners in economics; Duflo, Banerjee and Kremer — end up with when underestimating heterogeneity and interaction is not only an external validity problem when trying to ‘export’ regression results to different times or different target populations. It is also often an internal problem to the millions of regression estimates that economists produce every year.

‘Ideally controlled experiments’ tell us with certainty what causes what effects — but only given the right ‘closures.’ Making appropriate extrapolations from (ideal, accidental, natural or quasi) experiments to different settings, populations or target systems, is not easy. ‘It works there’ is no evidence for ‘it will work here.’ Causes deduced in an experimental setting still have to show that they come with an export-warrant to the target population/system. The causal background assumptions made have to be justified, and without licenses to export, the value of ‘rigorous’ and ‘precise’ methods — and ‘on-average-knowledge’ — is despairingly small.

RCTs have very little reach beyond giving descriptions of what has happened in the past. From the perspective of the future and for policy purposes they are as a rule of limited value since they cannot tell us what background factors were held constant when the trial intervention was being made.

RCTs usually do not provide evidence that the results are exportable to other target systems. RCTs cannot be taken for granted to give generalizable results. That something works somewhere for someone is no warranty for us to believe it to work for us here or even that it works generally.

Les limites de la méthode Duflo

27 Oct, 2019 at 23:45 | Posted in Economics | Comments Off on Les limites de la méthode Duflo

maComme le note Martin Ravallion, «pour le J-PAL, les expérimentations randomisées ne sont pas simplement au summum du menu des méthodes autorisées, rien d’autre n’est au menu.» Il serait regrettable que le tsunami expérimental auquel on assiste aujourd’hui en économie du développement emporte avec lui des méthodes éprouvées en sciences sociales et fasse fi de décennies de travaux consacrées à ces questions, sous prétexte que ces productions seraient toutes, par définition, moins rigoureuses. Des thématiques centrales qui ne se prêtent pas aux expérimentations resteraient en friche, d’autres ne seraient abordées que par le petit bout de la lorgnette randomisable. « Si le seul outil dont vous disposez est un marteau, tout finit par ressembler à un clou » avertissait Abraham Maslow. Et cela peut faire mal. En économie, comme dans toutes les disciplines, il n’existe pas de panacée méthodologique.

Arthur Jatteau & Agnès Labrousse

Sourire — c’est pour les ploucs …

27 Oct, 2019 at 20:41 | Posted in Varia | Comments Off on Sourire — c’est pour les ploucs …


I’m a believer

27 Oct, 2019 at 20:03 | Posted in Varia | Comments Off on I’m a believer


A whiter shade of pale

27 Oct, 2019 at 18:11 | Posted in Varia | Comments Off on A whiter shade of pale


On tour

24 Oct, 2019 at 14:53 | Posted in Varia | Comments Off on On tour


Guest appearances in Denmark and Germany. Regular blogging will be resumed next week.

The pitfalls of econometrics

24 Oct, 2019 at 14:47 | Posted in Statistics & Econometrics | 3 Comments

Ed Leamer’s Tantalus on the Road to Asymptopia is one of my favourite critiques of econometrics, and for the benefit of those who are not versed in the econometric jargon, this handy summary gives the gist of it in plain English:


Most work in econometrics and regression analysis is made on the assumption that the researcher has a theoretical model that is ‘true.’ Based on this belief of having a correct specification for an econometric model or running a regression, one proceeds as if the only problem remaining to solve have to do with measurement and observation.

economWhen things sound to good to be true, they usually aren’t. And that goes for econometric wet dreams too. The snag is, as Leamer convincingly argues, that there is pretty little to support the perfect specification assumption. Looking around in social science and economics we don’t find a single regression or econometric model that lives up to the standards set by the ‘true’ theoretical model — and there is pretty little that gives us reason to believe things will be different in the future.

To think that we are being able to construct a model where all relevant variables are included and correctly specify the functional relationships that exist between them, is  not only a belief without support, but a belief impossible to support. The theories we work with when building our econometric models are insufficient. No matter what we study, there are always some variables missing, and we don’t know the correct way to functionally specify the relationships between the variables we choose to put into our models.

Every econometric model constructed is misspecified. There are always an endless list of possible variables to include, and endless possible ways to specify the relationships between them. So every applied econometrician comes up with his own specification and parameter estimates. The econometric Holy Grail of consistent and stable parameter-values is nothing but a dream.

A rigorous application of econometric methods presupposes that the phenomena of our real world economies are ruled by stable causal relations between variables.  Parameter-values estimated in specific spatio-temporal contexts are presupposed to be exportable to totally different contexts. To warrant this assumption one, however, has to convincingly establish that the targeted acting causes are stable and invariant so that they maintain their parametric status after the bridging. The endemic lack of predictive success of the econometric project indicates that this hope of finding fixed parameters is a hope for which there really is no other ground than hope itself.

The theoretical conditions that have to be fulfilled for regression analysis and econometrics to really work are nowhere even closely met in reality. Making outlandish statistical assumptions does not provide a solid ground for doing relevant social science and economics. Although regression analysis and econometrics have become the most used quantitative methods in social sciences and economics today, it’s still a fact that the inferences made from them are, strictly seen, invalid.

Econometrics is basically a deductive method. Given the assumptions (such as manipulability, transitivity, separability, additivity, linearity, etc) it delivers deductive inferences. The problem, of course, is that we will never completely know when the assumptions are right. Conclusions can only be as certain as their premises. That also applies to econometrics.

The Wall

23 Oct, 2019 at 17:51 | Posted in Politics & Society | Comments Off on The Wall


Next Page »

Blog at
Entries and Comments feeds.