Today I have two anniversaries to think about. First, it is one year ago today that my second daughter was born, Anna Sophia Mack. She is complete joy – exceptionally beautiful, and she even looks like me (I know, it is an enigma). This last year has gone by so fast, just like the cliché that every parent I’ve ever met has told me.

But today is also another one-year anniversary. On the evening of that same day, my good friend Jeff Byers was in a car accident. After two months in a coma, Jeff died. These two events will always be linked for me, but that’s not such a bad thing. I am happy to be reminded of Jeff. As I sit at my desk writing this, I’m remembering of a good Jeff story.

Six years ago I started remodeling my house. On the third floor was a very small bedroom that I knew would make the perfect wood-paneled office that I had always dreamed of having. In fact, I had already bought the prefect desk, one of those giant walnut monsters that oozes class and substance. A quick check, however, showed that the desk was too big to make it up the stairs and into the soon-to-be office. Talking with my contractor, we decided to get a crane to lift the thing in as we added a gabled roof.

Telling Jeff this story, he was appalled by the inelegance of my brute-force solution to the oversized desk problem. Quickly, he organized a contest at work to see who could come up with a better (or at least more imaginative) way of getting my desk into my office. When the deadline for entries arrived, over a dozen solutions were submitted, involving things like operas, dolphins, weather balloons, and leaf-cutting ants. My personal favorite had Susan (now my wife, but then my girlfriend) convincing me that it was more manly to leave the desk downstairs (I’m quite sure that one would have worked). Jeff judged the entries, with winners paid in beer.

In the end, the process of remodeling the room into an office removed a few walls and opened the stairs sufficiently to allow the desk to go up them after all. The office was finished around the desk, so that it is now a permanent part of the room.

Thanks for that memory, Jeff. We all still miss you.

Warren and his wife April were good friends of Susan and I. He will be greatly missed.

Dr. Warren D. Grobman, who died on Wednesday July 9, 2008, had a distinguished career as an x-ray spectroscopist and a semiconductor physicist. He was born in Philadelphia, Pennsylvania in 1942, received his undergraduate degree from the University of Pennsylvania in 1964, and his PhD from Princeton University in 1967. He worked for many years at the IBM Thomas J. Watson Research Center in New York, notably as the founder of the x-ray lithography program, before relocating with his wife, April to Texas where he worked at SEMATECH and Motorola Semiconductor. He made many contributions, both as a research scientist and in product development and design, and was a fellow of both the American Physical Society and of IEEE. He also found time to pursue his love of admiring and creating art. He was thoroughly enjoying his retirement with his beloved wife, April, traveling, music, food and wine, computers, and time with the rest of their family, along with expanding his work as an artist. Warren had a rare combination of a keen intellect and an irreverent sense of humor and fun, coupled with a gift for caring, working with and teaching others. He was as likely to be taking a reflective walk with April, building a computer or a robot just for fun, admiring or creating art, studying physics lectors and following the progress of the Large Hadron Collider or sitting on the floor playing and giggling with his grandchildren. He leaves a legacy to his family and friends of love and caring for others and of love of knowledge, enjoyment of life, honesty, integrity and hard work. He was a wonderful husband, father; grandfather, brother, uncle and friend.

He is survived by his loving wife, April Schweighart; sons and daughters-in-law, Jeffrey and Norma Grobman and Steven and Ashlyn Grobman; grandchildren, Tyler and Lauren Grobman; sister and niece, Rita Grobman Howard and Rachel Howard; brother-in-law and sister-in-law, Charles and Cindy Schweighart and their daughters, Allison and Elizabeth Schweighart; and father-in-law, Ed Schweighart.

I just ran across a random tidbit of information that got me thinking. Proctor and Gamble has a product called FreBreze. People spray this chemical around their house to make it seem as if a fresh breeze had been blowing through (because, of course, we would never want to actually open our windows). This is a product that absolutely no one needs. It’s sold in the classic way – through massive advertising guided by people with PhDs in psychology. So here is the tidbit: annual sales of FreBreze were greater than $800M last year. Here is some perspective: according the World Bank, there are twenty countries with 2007 GDPs less than this amount. This is what I call inconspicuous consumption – the little things we spend our money on that we hardly notice – and if we didn’t spend it, we would hardly notice that, too.

Yesterday, my 10 month old daughter became an “infant scientist”. At least, that what they called her at the University of Texas Infant Cognition Laboratory. In fact, she was the subject of an experiment (though I refuse the lab rat comparison). We volunteered Anna for the test, which took about 45 minutes, just as we had volunteered her older sister Sarah for a different experiment two years ago. The experiment was quite interesting. Anna was outfitted with an electrode-laden cap so that brain activity could be monitored. Then she was shown pictures of female faces of varying degrees of beauty (the faces had been rated for their attractiveness earlier by adults). The question was, is the baby brain hardwired to recognize beauty? Anna’s left brain definitely lit up more when pretty faces were shown to her. Interesting.

I’ve been doing some work lately as an expert witness, so I found the following factoid quite interesting. The most commonly quoted song lyric in judicial opinions is from Bob Dylan’s Subterranean Homesick Blues: “You don’t need a weatherman to know which way the wind blows.” How do I know this? The law professor Alex B. Long is described as the leading expert on music citations in judicial opinions, and his analysis can be found in an article in yesterday’s New York Times. Couple this with Arthur C. Clarke’s fourth law (“For every expert there is an equal and opposite expert”) and we see that, especially in a court of law, two expert weathermen will generally tell you the wind is blowing in opposite directions.

I was befuddled (rank: 53,829) by my recent experience with wordcount.org (see my previous post). It seems that the word ‘lithography’ is ranked appallingly low in frequency of use, relegating me and my life’s work to the denizens of the perennially unpopular. But something smelled funny. I began to think that WordCount was not very good at counting. Since I have spent a lot of time thinking about how to measure things over the years, I decided to do what I always do when I see a data point I don’t like: blame the measurement.

I began by looking into the website’s counting method. From the wordcount.org site:

“WordCount™ is an artistic experiment in the way we use language. It presents the 86,800 most frequently used English words, ranked in order of commonness… WordCount data currently comes from the British National Corpus, a 100 million word collection of samples of written and spoken language from a wide range of sources, designed to represent an accurate cross-section of current English usage.”

So WordCount is an art project. I suppose that doesn’t mean it couldn’t be accurate, though I suspect that accuracy is low on the list of success criteria for most artists. But what is the British National Corpus? I found the official BNC website, and this is what they said:

“The British National Corpus (BNC) is a 100 million word collection of samples of written and spoken language from a wide range of sources, designed to represent a wide cross-section of current British English, both spoken and written.”

British English! That explains a lot. I thought the word count would relate to real English. But since lithography was a European invention, and was certainly practiced in England, I’m not sure that this could explain lithography’s unexpected lack of popularity. True, England doesn’t have a semiconductor industry to speak of, so talk of semiconductor lithography over dinner is probably unlikely. But still, the frequency of use seemed too low, especially compared to ‘sciorto’.

I did a little more digging. Of the 100 million words in the collection, the word ‘lithography’ is used 47 times. That’s a pretty small count, even if the sample appears to be large. 100 million words is obviously not enough if you want good statistics at the tail of the distribution. The other words near ‘lithography’ on the list – luqa, calculi, tiverton, kaysone, sciorto, and bullingdon – were all tied with lithography. Digging further in the BNC website, I could even find the sources for those 47 word uses. This is where the fun begins.

Yes, Sciorto is an Italian family name, but Count Roman di Sciorto is a character from a romance novel called Calypso’s Island, the source of all 47 occurrences in the BNC. Talk about skewing the sample. Here is one example: “How ludicrous, after all, to have imagined that the great Count Romano de Sciorto, of Casa Sciorto, of the Città Notabile, the Noble City, could fall seriously in love with her.” Riveting. Tiverton, while certainly a city in England, is also a character from another romance novel, Hidden Flame, from which 19 of its 47 word-use references came. It seems that romance novels make up a fair part of the 100 million word collection. Almost every use of Bullingdon occurred on television news and refered to the prison of that name in Oxfordshire, England. What we have here is a phenomenon called ‘the sampling sucks’, caused by the lumpiness of an abysmally low sample size for these words. 100 million words seems large, but when you think about all of the words that are written and spoken in English each day, that number starts looking very small.

The bottom line is this: WordCount is art, and while it definitely has words, it doesn’t do a very good job of counting. You shouldn’t expect artists to count – that’s what nerds are for.

By the way, ‘recount’ is number 29,409 on the list. I think the wordcount.org folks need to move it a little higher up.

II discovered an interesting website recently: wordcount.org. The site claims to have taken the 86,800 most common words in the English language and ordered them by frequency of use. It’s no surprise that the top ten words are, in order, ‘the’, ‘of’, ‘and’, ‘to’, ‘a’, ‘in’, ‘that’, ‘it’, ‘is’, ‘was’. Overcome by curiosity, I began typing in words to see how they ranked. ‘Mack’ is number 26,453. ‘Beer’ is 2,927. ‘Wine’ is more common at 1,634 (though I would have thought beer, the common man’s drink, would be more popular).

Then I typed in ‘lithography’ (the subject of my profession). I was shocked at the result: #42,832. There are 42,831 words in English that are more commonly used than ‘lithography’! Obviously, that is not true in my household, but could I be so out of touch with the rest of the world that I didn’t realize that lithography is in the bottom half of word popularity?

I decided to look at the few words just above ‘lithography’ on the list. Here they are.

42,826 luqa
42,827 calculi
42,828 tiverton
42,829 kaysone
42,830 sciorto
42,831 bullingdon
42,832 lithography

Are they kidding me? A word with a ‘q’ not followed by a ‘u’ is more popular than ‘lithography’? The only word I even remotely recognize from this list is the plural of calculus, and I can truthfully say that I believe that one calculus is enough. Is my field of study and work, the field that I have devoted the last 25 years of my life to, really this obscure? I guess so.

By the way, Luqa is a small village in Malta. Tiverton is an English town in Devon (as well as a New English town in Rhode Island). Kaysone is the first name of Kaysone Phomvihane, a former prime minister of Loas. Sciorto is an Italian family name. Bullingdon is an area of land in Oxfordshire, England, known as a “hundred”. The Bullingdon Club (or Bullingdon Dining Club) is a top-secret Oxford student drinking club for the super-rich. (Thank you, Google.) I look forward to using these words in conversation soon. I suspect there are a lot of words above ‘lithography’ on the list that I’ll need to learn.

But wait, there is more on this subject that must be told. Could there have been a measurement error? Stay tuned…

I would be a happy man if I could get by day to day without a car.

Currently, I manage to go sometimes several days without using my car. I walk to my teaching job at the University of Texas (1.6 miles each way), and there are a few errands that I can do by walking. But many things are just too far away.

I discovered an interesting website: www.walkscore.com. Type in your address, then it rates your neighborhood based on walkability on a scale of 0 – 100 (using an undisclosed algorithm). Here’s what the scores mean:

90 – 100 = Walkers’ Paradise: Most errands can be accomplished on foot and many people get by without owning a car.
70 – 90 = Very Walkable: It’s possible to get by without owning a car.
50 – 70 = Some Walkable Locations: Some stores and amenities are within walking distance, but many everyday trips still require a bike, public transportation, or car.
25 – 50 = Not Walkable: Only a few destinations are within easy walking range. For most errands, driving or public transportation is a must.
0 – 25 = Driving Only: Virtually no neighborhood destinations within walking range. You can walk from your house to your car!

My address scored a 43, which is not great. I’m willing to walk a bit farther than most people, I suspect, so I’m willing to call my neighborhood “somewhat walkable”. And by walking much more than I have been in the past, I’m also driving less often to the gym.

In my last two posts on this blog, I commented on a recent scientific publication:

Tomáš Grim, “A possible role of social activity to explain differences in publication output among ecologists”, Oikos, OnlineEarly Articles, 8-Feb-2008.

The article found a correlation between the weekly drinking habits of Czech avian ecologists and their scientific publication output. I was pleased to see that Dr. Grim, the author of this study, commented on both of my posts. Here, I will address most of his comments and give a further critique of his paper.

I complained in my first post that correlation does not imply causation. Dr. Grim correctly pointed out that the word “‘causes’ or ‘causation’ does not appear in my paper. I am not that stupid.” He then suggested that I “read more carefully, please.” But I have read his paper very carefully. And while Dr. Grim is most definitely smart and refrained from using the term ‘causation’, his theory of causation is evident throughout the paper. He states in his paper that “human cognitive performance during and after drinking is decreased”, that it “significantly decrease[s] cooperativeness” and the “effects of alcohol use are well known to decrease mental and working performance in general”. Dr. Grim offers no other theory to explain the observed correlation (even though there are many), and in fact his idea of causation is evident from the title of his paper. His recent comments on this blog also confirm his adherence to one causation theory, adding that hangovers impede productivity (even though his survey provided no evidence of drunkenness or hangovers on the part of the participants). Thus, my complaint that Dr. Grim’s observed correlation is insufficient evidence to support his theory of causation is entirely relevant, despite Dr. Grim’s protests.

Dr. Grim criticized my speculations about the nature of the raw data for his study, saying that I “just do not know” and that my speculations were wrong. I accept these criticisms. A bigger complaint, however, is that I must speculate at all. Dr. Grim does not provide any of the raw data for his study, instead showing only transformed data (without publishing transformation parameters). He provides virtually no descriptions of the data either – no means or ranges of either publication rates or drinking habits. As a result, it is impossible to determine the importance of the resulting correlation: does drinking at twice the mean rate correlate with a 5% decrease in scientific output, or a 50% decrease? In other words, how big is the effect? To my mind, that would seem important – and in fact the whole point. Unfortunately, readers are given no choice but to speculate (and possibly to hear Dr. Grim tell them how much they don’t know). Dr. Grim’s defense to this criticism – that a promise of confidentially requires that he keep the data secret – simply points out a basic flaw in the design of his study. His choice to survey only his fellow Czech avian ecologists may have made for good barroom banter among his buddies, but did nothing to benefit science.

I thank Dr. Grim for clearing up my misunderstanding about the number of participants in the 2006 study (34 rather than 16). Figure 1 in his paper labels 18 data points as “included in the first survey 2002” and as “past”, while 16 data points are labeled “included in 2006” and “present”. From such labeling it is easy to misinterpret the relationship between the data points in the two surveys. In fact, I’m still confused. Are the 34 data points in Figure 1 all from the 2006 survey? What does “past” with n = 18 and “present” with n = 16 refer to? Are the 34 total data points for 34 different people, but some surveyed in 2002 and others in 2006? I’m trying, but I just can’t “[s]ee the bloody Fig. 1 properly, man!” Perhaps a more complete explanation would have made the paper clearer.

I described the correlation coefficients for his 2002 and 2006 survey results as “barely large enough to claim with 95% confidence that the results are statistically different from an R^2 of 0 (no correlation)”, though my confusion as to the number of data points in the 2006 study would seem to invalidate my analysis for that correlation coefficient. In somewhat colorful language, Dr. Grim called into question my statistical abilities. His advice to me: “PLEASE, read some statistical textbook first”. While I certainly don’t consider myself a leading expert in statistical analysis, I have read a statistical textbook or two over the years. In fact, I read quite a few in 2006 when I taught a graduate-level statistics course at the University of Notre Dame. I recall teaching my students about the Student’s t-test to compare the means of two normally distributed populations each having the same estimated variance, and about Fisher’s z-transform to transform a distribution of correlation coefficients into something approximately normal, so that the Student’s t-test can be used. For a sample size of 18, the R^2 must be greater than 0.25 to be different from 0 with 95% confidence. The 2002 study showed values of R^2 just barely larger than this, as I had stated previously.

I also recall teaching my students that statistics are often the last refuge of the mediocre scientist. When systematic error (bias in the data) is greater than random error, the use of statistics is a waste of time. Worse, in that situation the use of statistical analysis can give a false sense that one’s conclusions are scientifically rigorous. So while an R^2 of 0.5 and a sample size of 34 may be statistically significant in the absence of bias in the data, one must do more than enter numbers into JMP if one wishes to understand whether the correlation is scientifically significant. So let’s look at Dr. Grim’s study a little more closely to see whether it is likely to be free of significant bias.

For a study such as Dr. Grim’s, there are a number of opportunities to introduce bias into the data. Here I’ll address the three most important:

1. Survey bias. As anyone who has seen political parties trot out competing polls showing significantly different results knows, it is easy to bias survey results by the way surveys are written or administered. Does Dr. Grim have experience in the proper design and administration of human behavior surveys? Was his survey vetted by other scientists with such experience? Considering the fact that many (if not most) of the survey participants were likely friends or colleagues of Dr. Grim, and that the subject matter of the survey (social drinking) is a sensitive topic on both personal and professional levels, the possibility for bias in the responses is extreme. It doesn’t take an expert in human behavioral research to see that his survey does not pass the smell test. (As an aside, a typical ethical requirement for human behavioral studies would prohibit the use of subjects with any significant personal or professional ties to the researchers, especially if those ties are not disclosed.)
2. Sampling bias. When one cannot measure an entire population, one must sample that population and draw inferences about the population from the sample. If the sample is not representative of the population, sampling bias can invalidate any conclusions drawn from the study. So what was the population under study by Dr. Grim? In his comments on this blog, Dr. Grim claims that the population under study were Czech avian ecologists (of which, according to Dr. Grim, there are 38). If this were the case, then Dr. Grim’s study would be interesting to, oh, about 38 people. Hardly the kind of thing that gets coverage in the New York Times. In fact, Dr. Grim’s paper implies a much greater population is at play. The title of the paper claims application to all ecologists, and the text of the paper contains more than a dozen references to science, scientists, and scientific productivity unqualified by the Czech avian ecologist subgroup. The abstract of the paper discusses science and scientists in general, without reference to a narrower population. His hypothesis under test is clearly stated in the abstract: “I predicted negative correlations between beer consumption and several measures of scientific performance.” The penultimate paragraph of the paper is filled with wide-ranging conjectures about the implications of this study to the lives and careers of scientists (including the bizarre speculation that social drinking may impact the “biological success” of scientists as well). Dr. Grim’s statement that the population under study is limited to Czech researchers studying avian evolutionary biology and behavioral ecology is simply disingenuous. His sample was limited to Czech avian ecologists, and is thus horrendously biased compared to the population of all ecologists or all scientists. Simply put, if his population is all Czech avian ecologists, then his study is essentially worthless to the greater population of scientists, and if his population is the greater population of scientists, then his study is essentially worthless due to extreme sampling bias. Either way, the conclusion is the same.
3. Confounding factors. When factors outside of the control of the researcher have significant influence on the resulting correlation, even a statistically justified correlation may lead to no insight as to cause and effect (and in fact may mislead). Did Dr. Grim’s study identify all of the significant confounding factors? This is always one of the most difficult questions facing the researcher of human behavior.

There are other criticisms of this study as well, such as the lack of independence of the data points (thus invalidating essentially all of Dr. Grim’s statistical analysis), and of course the use of a very small sample size to draw sweeping conclusions about human behavior. Dr. Grim’s defense that a “sample of that magnitude (34) is quite good for an ecological study” is of little importance since this was not an ecological study but a human behavior study, involving extremely complex human behavior at that. His final fallback, that “[e]ven just stating the logic of a hypothesis without any empirical data is worth publishing”, is more than a stretch. I think a strategy of trying to publish hypotheses without any empirical data would have a bigger impact on a scientist’s journal publication rate than beer drinking ever could.

But all of these criticisms boil down to one simple fact: Dr. Grim surveyed a small number of fellow Czech ornithologists in what was probably great fun, but definitely bad science. Dr. Grim himself described his paper as “half-joke-half-study”. Now, I’m the first person to appreciate a good joke, especially at the expense of us scientists. However, Dr. Grim failed to mention the “half-joke” part of the equation in his published paper, preferring instead to pass off his bit of fun as real science. As such, publication of his paper in a scientific journal borders on the unethical. Fortunately for Dr. Grim, he chose to publish his paper in an ecology journal, rather than one specializing in human behavioral studies, where it is likely that his reviewers would have been less kind. Unfortunately for Dr. Grim, he must now own up to his joke or face digging an even deeper hole for himself.

My March 21 post on this topic generated an amazing amount of interest. I guess it didn’t hurt that a blurb about that post made the front page of Slashdot – and thus conferring near-god-like status on me according to my most geeky friends (“Chris – you’ve been slashdotted!”). The Slashdot post generated over 130 comments, and the post on my blog hit a record 35 comments as of today. Wow. Who’d of thought that a challenge to the beer-drinking habits of scientists would generate such interest?

To review, Czech ornithologist Tomas Grim published a paper showing a negative correlation between social beer drinking and scientific output. I then posted a critique of that study explaining why I thought that the data did not support his conclusions.

Many of the comments I received were fun, some were silly, and still others were just plain weird. Some comments made suggestions for future work (yes, I am considering carefully whether I should conduct my own study), while others criticized my analysis. I think some of these critiques are worth responding to.

Some people pointed out the obvious cause-and-effect relationship between drunkenness and poor thinking skills. Isn’t it obvious that drinking would lead to bad science? However, the paper gives enough information to conclude that the average rate of beer drinking among studied subjects is not much different from the Czech average – less than one pint (0.5 liters) per day. Although the raw data is not presented (to protect the identities of the subjects, according to the author), it seems unlikely that the heaviest drinkers in the study rose to the level of alcohol abusers. And since the survey only asked about average weekly beers consumed, there is no data on binging or even if any of the heaviest beer drinkers got a buzz!

Several people caught my error where I referred to R-squared as the “correlation coefficient”. I definitely deserve a reduction in my beer rations for that bit of sloppiness. R-squared (or R^2) is often called the “coefficient of determination” and, for the case of least-squares linear regression, R-squared is equal to the square of the correlation coefficient (r).

Several people jumped on my case of claiming that the five lowest scientific output data points were outliers that should be removed from the analysis. I never used the term outlier, and I never said they should be removed from the analysis. Instead, I claimed that these data points were extremely influential (without them the correlation nearly disappears). Properly dealing with outliers is extremely tricky and I’d be surprised if 1 in a 100 scientists could do it well. But understanding how to handle highly influential data points (and what that means for the interpretation of the fitting results) is even harder, and is almost always ignored. Pointing out that five highly influential beer guzzlers controlled the fit is very important when trying to draw conclusions that apply to tens of thousands of scientists.

Which brings me back to the main point of my critique of Dr. Grim’s study – the low number of subjects. I mentioned that there were 34 data points – but this is actually an exaggeration. Grim made two surveys of beer drinking habits, one in 2002 with 18 responses and a second one 2006 with 16 responses. Thus, there were 34 data points. But the people being surveyed were the same! Thus, the total number of people involved in the study looks to be 18. (I wonder which data point(s) belong to Dr. Grim.) The analysis of the 2002 data showed an R^2 of 0.34, while the 2006 survey showed an R^2 of 0.52. The correlation coefficients are just barely large enough to claim with 95% confidence that the results are statistically different from an R^2 of 0 (no correlation) – assuming all confounding factors have been properly eliminated.

But that’s the rub. There is no way to know whether all confounding factors (read systematic errors) have been eliminated. And it seems unlikely, as I mentioned before, that one can plausibly claim that the data points are independent, since it seems likely that at least some of this small group of Czech ornithologists socialized together. Thus, a low R^2 for a study with a small number of data points on a subject of great complexity is unlikely to be very revealing despite the most rigorous of statistical treatments.

Interestingly, the most revealing thing I learned in the past week about this study came from Dr. Grim’s website. The page on his beer study is labeled “Beer vs science – first laugh, then think (what to drink:-)”, which makes me suspect he is angling for an Ig Noble award. He posts his now-infamous paper for download, as well as giving links to many of the press reports about his work. Then come a collection of pictures showing that Dr. Grim’s data point would not lie on the low beer consumption side of the graph. If nothing else, it seems that Dr. Grim is having fun. Good for him!

By the way, I did not drink a single beer while I wrote this post. Only scotch.