Data without theory is always treacherous

18 May, 2019 at 15:50 | Posted in Statistics & Econometrics | Comments Off on Data without theory is always treacherous

Data without theory can lead to bogus inferences …

Before being comforted or alarmed, consider whether it makes sense to extrapolate. Is there a persuasive reason why the future can be predicted simply by looking at the past? Or is that wishful thinking? Or nothing at all? …

Remember that even random flips can yield striking, even stunning, patterns that mean nothing at all …

A statistical comparison of two things is similarly unpersuasive unless there is a logical reason why they should be related … Ask yourself whether the people who did the study thought before calculating.

The central problem with the present ‘machine learning’ and ‘big data’ hype is that so many — falsely — think that they can get away with analysing real-world phenomena without any (commitment to) theory. But — data never speaks for itself. Without a prior statistical set-up, there actually are no data at all to process. And — using a machine learning algorithm will only produce what you are looking for.

Machine learning algorithms always express a view of what constitutes a pattern or regularity. They are never theory-neutral.

Clever data-mining tricks are not enough to answer important scientific questions. Theory matters.