I read an interesting story in the New York Times on Tuesday – interesting, but perplexing. It seems that a Czech ornithologist (more specifically, an avian evolutionary biologist and behavioral ecologist) surveyed other Czech ornithologists (more specifically, other avian evolutionary biologists and behavioral ecologists) on their beer drinking habits. He then correlated their scientific output (as measured by publications/year and citations/paper) with their annual beer consumption. The result was counterintuitive – higher beer consumption led to lower scientific output.
My first thought was to scoff at the study – after all, I drink a lot of beer, and my scientific output has been pretty good. Further, I hang out with quite of few other prolific scientists who also drink their fair share of man’s greatest beverage. There must be something strange about those Czech bird watchers.
But as I began to think further on the subject (and enjoy a fine Pale Ale to settle me down), I realized I was making two cardinal mistakes in my approach to this startling scientific development: 1) I trusted my limited anecdotal evidence over a statistically valid scientific study, and 2) I based my understanding of the science on a journalist’s description of a technical paper. Recognizing my initial flaws, I moved on to a smooth and especially bitter IPA and got on the internet. After a few minutes I had located the original paper in the biology journal Oikos. Here is the citation:
Tomáš Grim, “A possible role of social activity to explain differences in publication output among ecologists”, Oikos, OnlineEarly Articles, 8-Feb-2008.
The paper is only three pages long, so it was a quick read. It was also fairly easy to find the defects in the work. First, there was the common mistake of confusing correlation with causation. The author implied that increased beer drinking caused reduced scientific output. An equally likely explanation is that poor performance in one’s chosen career (in this case ornithology) led to increased beer drinking (and after all, the subjects live in a country with the world’s highest per capita beer consumption). Alternatively, a third, unmeasured factor could be leading to both poor job performance and higher beer consumption (a nagging spouse, for example).
As I looked more carefully at the data, I found a much more significant problem. The total number of data points (as these bird-watching scientists had been reduced to) was 34. This is not an exceptionally large number of subjects when one wishes to draw conclusions about all beer-drinking scientists. The discovered linear relationship between beer consumption and scientific output had a correlation coefficient (R-squared) of only about 0.5 – not very high by my standards, though I suspect many biologists would be happy to get one that high in their work.
But it was while I was switching to a magnificent Pacific Northwest microbrew porter that I saw the real problem. Looking at the graph of the 34 data points, it was clear that the entire correlation was caused by the five lowest-output scientists. Without those five data points, the remaining 29 – showing a wide range of scientific output and beer consumption habits – exhibited absolutely no correlation. Thus, the entire study came down to only one conclusion: the five worst ornithologists in the Czech Republic drank a lot of beer.
Other significant problems were also evident. Standard linear regression, with all the fanciest statistics one can muster, still makes the assumption that each data point is independent. But this study was specifically looking for the impact of social habits on scientific output. Isn’t it likely that some, or many, of these scientists socialized together? After all, the Czech avian evolutionary biology community is not that large. I know that much (possibly most) of my beer drinking is done with fellow lithographers. For all we know, the five lowest-output scientists that created this whole controversy were all part of a drinking club – they’re probably enjoying a fine pilsner and having a fine joke at our expense right now!
In the end, though, I was pleased to see that careful reading and analysis of the original published work led to an easy debunking of the silly notion reported in the press that somehow beer drinking was bad for scientific performance. With the reputation of beer-loving scientists restored to its rightful glory, I sat back and sipped my double-chocolate stout. Ah, the life of a Gentleman Scientist.