Keynes on the ‘devastating inconsistencies’ of econometrics

30 November, 2016 at 11:05 | Posted in Statistics & Econometrics | 6 Comments

In practice Prof. Tinbergen seems to be entirely indifferent whether or not his basic factors are independent of one another … But my mind goes back to the days when Mr. Yule sprang a mine under the contraptions of optimistic statisticians by his discovery of spurious correlation. In plain terms, it is evident that if what is really the same factor is appearing in several places under various disguises, a free choice of regression coefficients can lead to strange results. It becomes like those puzzles for children where you write down your age, multiply, add this and that, subtract something else, and eventually end up with the number of the Beast in Revelation.

deb6e811f2b49ceda8cc2a2981e309f39e3629d8ae801a7088bf80467303077bProf. Tinbergen explains that, generally speaking, he assumes that the correlations under investigation are linear … I have not discovered any example of curvilinear correlation in this book, and he does not tell us what kind of evidence would lead him to introduce it. If, as he suggests above, he were in such cases to use the method of changing his linear coefficients from time to time, it would certainly seem that quite easy manipulation on these lines would make it possible to fit any explanation to any facts. Am I right in thinking that the uniqueness of his results depends on his knowing beforehand that the correlation curve must be a particular kind of function, whether linear or some other kind ?

Apart from this, one would have liked to be told emphatically what is involved in the assumption of linearity. It means that the quantitative effect of any causal factor on the phenomenon under investigation is directly proportional to the factor’s own magnitude … But it is a very drastic and usually improbable postulate to suppose that all economic forces are of this character, producing independent changes in the phenomenon under investigation which are directly proportional to the changes in themselves ; indeed, it is ridiculous. Yet this is what Prof. Tinbergen is throughout assuming …

J M Keynes

Keynes’ comprehensive critique of econometrics and the assumptions it is built around — completeness, measurability, indepencence, homogeneity, and linearity — is still valid today.

Most work in econometrics is made on the assumption that the researcher has a theoretical model that is ‘true.’ But — to think that we are being able to construct a model where all relevant variables are included and correctly specify the functional relationships that exist between them, is  not only a belief without support, it is a belief impossible to support.

The theories we work with when building our econometric regression models are insufficient. No matter what we study, there are always some variables missing, and we don’t know the correct way to functionally specify the relationships between the variables.

Every econometric model constructed is misspecified. There are always an endless list of possible variables to include, and endless possible ways to specify the relationships between them. So every applied econometrician comes up with his own specification and ‘parameter’ estimates. The econometric Holy Grail of consistent and stable parameter-values is nothing but a dream.

A rigorous application of econometric methods in economics really presupposes that the phenomena of our real world economies are ruled by stable causal relations between variables.  Parameter-values estimated in specific spatio-temporal contexts are presupposed to be exportable to totally different contexts. To warrant this assumption one, however, has to convincingly establish that the targeted acting causes are stable and invariant so that they maintain their parametric status after the bridging. The endemic lack of predictive success of the econometric project indicates that this hope of finding fixed parameters is a hope for which there really is no other ground than hope itself.

The theoretical conditions that have to be fulfilled for econometrics to really work are nowhere even closely met in reality. Making outlandish statistical assumptions does not provide a solid ground for doing relevant social science and economics. Although econometrics have become the most used quantitative methods in economics today, it’s still a fact that the inferences made from them are as a rule invalid.

Econometrics is basically a deductive method. Given the assumptions it delivers deductive inferences. The problem, of course, is that we will never completely know when the assumptions are right. Conclusions can only be as certain as their premises — and that also applies to econometrics.

6 Comments »

RSS feed for comments on this post. TrackBack URI

  1. The point about the deductive method being based on past results is lost. Deductive methods based on technical relationships between entities that are generally true and are not based on past performance are more likely to forecast the future responses.

  2. “A rigorous application of econometric methods in economics really presupposes that the phenomena of our real world economies are ruled by stable causal relations between variables.”

    However, the very social phenomena which might result in some relatively stable causal relations between variables, technology and social institutions, are subject to longer term evolution which the mainstream microeconomic theory is incapable of explaining, so that if relatively stable relationships do exist for some period of time, the mainstream theory is incapable of explaining why they are stable over that period nor of predicting the conditions under which they will lose that stability.

  3. Keynes’ concerns about Tinbergen’s econometrics were largely valid. However, in correspondence Keynes recognized that some applications of econometrics were valid (e.g. a 1939 study of automobile demand). Moreover, Keynes expressed the belief that if data were available it would be possible to make empirical estimates of the consumption function and multiplier (General Theory, chapter 10 part V).
    .
    Prof. Syll’s difficulties with econometrics appear to be more fundamental. They may stem from his “celestial weldanshauung”, a philosophy of life which regards “reality” as being separate from the data and understanding available to humans.
    For example, Prof. Syll writes above “The theoretical conditions that have to be fulfilled for econometrics to really work are nowhere even closely met in reality.” And in a previous blog he wrote” “reality has no “correct” representation in an economic or econometric model. There is no such thing as a “true” model that can capture an open, complex and contextual system…”
    Thus Prof. Syll is a not nihilist – he believes that “reality” exists. But he never explains the source of his purported knowledge of “reality”. Perhaps his knowledge comes from intuition or divine revelation. Philosopher John Locke commented: “spirits of a higher rank than those immersed in flesh, may have as clear ideas of the radical constitution of substances … but the manner how they come by that knowledge exceeds our conceptions.”
    John Locke – An Essay concerning Human Understanding [1689] Book III, chapter XI, §23
    .
    Locke, like other empiricists, argued that “The whole extent of our knowledge or imagination reaches not beyond our own ideas limited to our ways of perception.”
    On this view, and in contrast to Prof. Syll, reality is NOT something which exists separate from observations and data. Rather “reality” IS our best understanding of available data.
    .
    Perhaps some of the confusion here stems from contrasting dictionary definitions of “reality”.
    1 the state of things as they actually exist, as opposed to an idealistic or notional idea of them.
    2 a thing that is actually experienced or seen.
    3 the state or quality of having existence or substance. In Philosophy: existence that is absolute or objective and not subject to human decisions or conventions.

    Prof. Syll focusses on definition 3.
    He contributes little regarding definition 2.

  4. “On this view, and in contrast to Prof. Syll, reality is NOT something which exists separate from observations and data. Rather “reality” IS our best understanding of available data.”
    .
    If you toss a coin, and it comes up heads, then in “reality” it is a double-headed coin, because every time you’ve tossed it, it has come up heads.
    .
    Or, alternately, one could get metaphysical and conjecture that, actually, the other side of the coin is tails, but that would be creating a reality not found in the data, wouldn’t it?
    .
    That’s the inescapable conclusion when reality is defined not to extend beyond the limits of the data available at a given point in time.

    • Michael,

      …If you toss a coin, and it comes up heads, then in “reality” it is a double-headed coin, because every time you’ve tossed it, it has come up heads.

      .
      It is OK if it refers to naturalthings such as our universe. This is because human beings are not creator of this natural worlds and get to know the reality only via observations and data. However, economics is social science, not natural science. Economies are artifacts. Human beings are the designer with rules of the game and also participate the game with our free-will to make choices. We know all potential instances from our game, but hard to know the choices from other persons at particular time periods.
      .
      I will rephrase your example more like economy version. The rules of a game: we have a coin with head and tail on each side. 5 persons use majority voting rule to decide if it is head or tail. We can know the reality, all possible results independent of observations, right? We would not think it is a double-headed coin even if voting results are heads for first 10000… times.
      .
      Another example is your previous debt-to-GDP ratio. I can easily explain it when this over 90% debt ratio rule breaks from NIPA accounting aspects. Government deficits/debt ratio could either increase GDP growth or decrease GDP growth heavily depending on how the government spends the money.
      .
      In Q3 2016, US government deficit (5% GDP) = Tax(28.7% GDP) – G(17.6% GDP) – Non-discretionary spending(16.1% GDP). Gov non-discretionary spending would not be counted in GDP estimation (C+I + G + X – M). Thus, you can see the following properties:

      (a) The more discretionary spending(i.e. G part), the higher GDP contributed from government even it may increase deficits and debt/GDP ratio.

      (b) If just moving 5% from 16.1% non-discretionary spending into G, then we will have additional 5% GDP growth with same deficit and debt ratio.

  5. “Am I right in thinking that the uniqueness of his results depends on his knowing beforehand that the correlation curve must be a particular kind of function, whether linear or some other kind?”

    Straight to the heart of the matter.

    … we don’t know the correct way to functionally specify the relationships between the variables …

    Every econometric model constructed is misspecified.

    To me, Keynes’s implicit point seems to suggest something much stronger. There may be no “correct way to functionally specify” causal relationships in the social world, because they may have no determinate, quantitative law of composition at all. There may be no “correct specification” whatsoever. This is not an epistemic point about our scientific ignorance. It is an ontological point about the nature of social reality.

    And the whole damn project of “statistical causal inference” depends utterly on the empirical correctness of this never-ever-argued-for assumption that Keynes exposes. You can’t even begin to interpret “results” without it.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.
Entries and comments feeds.