Virulent Word of Mouse

September 12, 2010

Two politicians, a research Proposal, and a cheap shot.

Filed under: Considering topical questions,Editorial — Shane Greenstein @ 4:21 pm

The last time I checked in on John McCain he was giving a very classy concession speech after he lost the presidential election.  There was something very inspiring in the grace with which he handled losing.

So I reacted with disappointment when I heard of McCain’s latest actions. As I will explain below, these are not classy.

Specifically, along with Senator Tom Coburn, in August McCain issued a report that took a cheap shot at a professor’s research at Northwestern, the university where I work, calling his most recent research proposal a waste of money. This post has one primary purpose: to explain why the charges lack merit.

It is possible to go further, so I will. The research is in a technically complex area of frontier computer science. This post argues that the critique of it involved a deliberate and juvenile misreading of the research.

It is possible to have a cynical reaction to Coburn’s and McCain’s report — to excuse it as one of the games politicians play. But McCain’s classiness actually gets under my skin in this case. It does not seem to be unrealistically idealistic to want him to show some class in his actions. Is it too much to expect a classy Senator not to debase public discussion with cheap shots that lack merit?

There is, in addition, a deeper and more subtle point behind these events. This example illustrates one of the strengths of the US research system, namely, the extensive use of peer-review. As I will point out below, peer-review insulates much of US research from the shenanigans of politicians, even formerly classy ones who gave in to the temptation to take a cheap shot. This example gives some clues as to why such insulation keeps the US research system free from such interference, and maintains its status as one of the best in the world.

What happened?

Let’s not be naive about US politics. It can involve deliberate misquoting of opponent’s statements out of context.  But most of the time these cheap shots seem like a side show, lacking any true policy substance. Consider the following scenario, however. What would you do if a politician aimed a cheap shot at someone you know, or a colleague who worked for the same employer?

As it happened, a cheap shot was aimed at someone who is a professor at Northwestern University, where I teach. This colleague and I have met, but I do not know him well. I read about these events in the newspaper (more below), and that is what made me curious to learn more. The more I learned (more below) the more I learned to despise what Coburn and McCain said and did.

Specifically, in August Senators John McCain and Tom Coburn issued a report that called many uses of stimulus money a waste of money. The report is called  “Summertime Blues: 100 Stimulus Projects that Give Taxpayers the Blues.” The title gives away the purpose: the Senators are trying to uncover waste in the spending of stimulus money.

To be sure, on the surface, there is nothing wrong with politicians performing oversight. On the surface this might seem like a reasonable thing for Senators to do.

Except that in the normal course of things the US government actually does a lot of auditing of government programs. Why do the Senators need to do more?

Specifically, auditing is the job of the Government Accounting Office (GAO), which has the expertise and staff to do an audit on just about anything, including some pretty obscure topics. I have read many GAO reports over the years, and usually I come away with a positive impression of their work. The staff are thorough, careful, and even-handed — just like, um, good auditors ought to be. The GAO can find wastes of money and bring it in the open.

Coburn and McCain might respond that they object to what the money was spent on, namely, to the goals of the project. The auditors might verify that the accounting was proper and legal, but Coburn and McCain would still call something a waste if the project did not deserve any funding.

That type of objections would seem reasonable too, though it begs further questions too. The report covers one hundred projects. Could the two Senators and their staff possibly have thoroughly investigated all one hundred projects?

Anyway, I decided to investigate only one of those projects, the one involving my colleague. It was not my purpose to assess the entire report.

Here is what I found.

How to criticize frontier computer science

I looked up the report issued by Coburn and McCain. The Coburn/McCain report is not a GAO report. It is not even close to the same thoroughness and quality. It is just a list of one hundred projects the Senators do not like. Sometimes their criticisms are two or three paragraphs. Sometimes one paragraph.

I focused on the task at hand. The criticism of my colleague arises in the 36th project listed in the report. This involves only one paragraph. Here is the criticism, reproduced exactly:


That is it. One paragraph of ridicule and five footnotes. Nothing more, nothing less.

This one paragraph might have passed without notice, lost among the one hundred. Let me explain how I heard about it. That will illustrate something about the US political system and US media, as well as a bit about the report itself.

Apparently, some political web sites picked up on the headline of the 36th entry. Like this one. Or this one. Or this silly one. The headline about joke-machine seems to have struck some site masters as an attention-grabbing-headline. A few of them made more ridicule with the material. (Of course, this is what the staff who wrote the report wanted…)

There is something disturbing and insidious about the way this headline spread. First, all these political web sites repeat the charges, and embellish them, but nobody seems to question their credibility. Second, none of these sites actually seem to have read the proposal by the computer scientist. Most of them quote directly from the Coburn-McCain report. Third, every site plays to the existing prejudices of their audience. There is no attempt to illuminate a large point or raise a bigger debate.

It still might have stayed in a small community, but then the Chicago Sun Times picked up on the same story. Despite the age of the story (over a month old), the Times ran a front page story in the print edition of an early September paper. The Sun Times story had the following headline:

NU prof: Computer research no joke

Reviled by McCain but defended as ‘hard-core computer engineering’

If you want to read the whole story, click here.

By the way, the other major Chicago paper, the Tribune, has not run any similar story. (Hurrah for Zell’s Tribune for having a bit of restraint!)

This is when I heard about the matter. A friend showed me the article. (They showed me the headline and said “Do you know this guy?”)

I read the article. To the credit of the reporter at the Sun Times, this story does not look like the other online articles I referred to above. This story includes an interview with the computer scientist, Professor Hammond, who makes his case reasonably well. The reporter seems to conclude that this is all a misunderstanding.

At the same time, the editors at the Sun Times deserve about as much disdain as I can heap on them. The headline is misleading. The story is unimportant. The placement of the article on the front page is sensational journalism at its worse.

Of course, that made me even more curious about the original report by Coburn and McCain. So I looked into it. I went to the footnotes.

Looking closely

Let’s look a bit more closely at the actual research. This will belabor the key point a bit, but the examination has a purpose. It will show why this is a cheap shot (in great detail) and why the charges lacks merit (in great detail) and why peer review is so essential to the US research system.

Interestingly, the authors of the Coburn-McCain report used the Northwestern Site and listed it in one of footnotes. I went to it and learned about the project.

The Northwestern site explains that the project was funded by the National Science Foundation. This makes sense. The National Institute of Health has been the best funded research in the US in the last decade. In contrast, the National Science Foundation (NSF) has been comparatively underfunded.

In other words, some stimulus money went to NSF, which has been the poor step-sister of the US research system under the prior administration. (The Bush administration always promised to bring NSF up to NIH funding levels, but actually never did). NSF then used it to fund a few more proposals than they could on their normal budget.

I have served on many panels at NSF (though, not any in of the divisions of computer science that would have examined this proposal). Usually there are scores of proposals at NSF. They are ranked for quality of the research team, expected impact, and scientific merit. Usually, there are far more proposals with scientific merit than the budget at NSF permits.

I will bet that the division within NSF that considered Professor Hammond’s proposal did the following. Instead of funding (say) 20 out of 130 proposals, as it usually does with its limited budget, it funded a few more. With the extra money, the same division funded an additional (say) dozen out of 130, and designated these as recipients of stimulus money. (I do not know the actual percentages, but these type numbers would not surprise me).

In other words, if someone is going to criticize the choice of the research and call it a waste of money, criticizing the researcher is the wrong way to do it. Wouldn’t it make more sense to criticize NSF’s panel, which selected the project, and rated it higher than others?

That observation leads to a big problem for Coburm and McCain, however. NSF funds a proposal only after every proposal passed the scrutiny of many other computer scientists and engineers. Professor Hammond’s proposal had to be rated near or among the best in its category. That undermines the credibility of  Coburn’s and McCain’s criticism. Do they think they know better than a panel of experts in computer science where best to invest in frontier computer science?

Coburn and McCain and their staff surely would have known all this. NSF’s procedures and its budget woes are not news. And, yet, the staff went ahead and selectively interpreted the proposal anyway, presumably in order to get a headline.

Why fund frontier computer science?

This entire discussion begs another question: why does the federal government fund frontier computer science? Perhaps Coburn and McCain do not want to fund any computer science research at all.

If that is what they are arguing, then they are really in for a fight. The short answer to that question is well known: US society gets a very high rate of return on investment in computer science, and that rate of return far exceeds the rate of return society gets on just about any other type of federally funded science. A classy Senator would know this.

More to the point, Professor Hammond’s proposal is good frontier research and potentially valuable to the economy. It is one of hundreds of such projects the federal government is currently funding. Criticizing this one is just like criticizing any other. Frankly, one of them — perhaps this one by Hammond, or perhaps another — deserves funding because one of them will lead to extraordinarily large and revolutionary changes. Perhaps more than one will.

In this case, the researchers are trying to make a better search engine by trying to better understand one aspect of human thinking, namely, humorous queries. It is not news, but computers have a very bad record providing results to human queries that deviate from a narrowly constricted set of rules for grammar and sentence construction. Exploring humor is no worse than scores of other deviations we all wish search engines could accommodate.

<I, for one, wish that search engines could tell when my queries are made with a tinge of sarcasm. Sarcastic questions ought to be answered with sarcastic questions, don’t you think? But I digress.>

More to the point, this project explores one of the biggest puzzles in research into human-computer interaction. It uses frontier lexical tools. Any major progress is a big deal. Even incremental progress is a big deal.

Now, to be sure, it seems unlikely the researchers will crack this nut with $700K of research funding. Even $7 million would probably not do it. So, perhaps the worse thing one can say  about the proposal is this: One might criticize the proposal for over-promising. That said,  there are no grounds for criticizing the proposal for its goals.

Maybe you are still skeptical about the value of this research. Well, consider this story.  It was a proposal in computer science — not unlike Professor Hammond’s — that funded two graduate students at Stanford, Larry Page and Sergey Brin, who went on to found Google.

Think about it. If the same crass and juvenile sensibility as Coburn and McCain’s report had been applied to Page and Brin, then they never would have been funded. Their research about page-rank would have been ridiculed and (mis)characterized as an attempt to automate popularity contests among web pages.

In Brin and Page’s case, their firm now makes far more than twenty billion in revenue a year. A reductionist characterizations of fundamental research would not have been informative or helpful.

Back to the main point. Bottom line. Peer-review insulates this proposal from outside criticism, and makes the Coburn-McCain criticism look foolish.

Summary

This example illustrates one of the reasons why many of the most successful examples of federally funded research — such as the Internet (another frontier computer science project for its time) — were insulated from political shenanigans and other cheap shots.  Political sensibilities provide a poor filter for assessing progress at an early stage of research, when the technology is quite complex.

I just looked at one example. I did not have to do much work to figure out that the criticisms lacked merit. This is a cheap shot from a two politicians. At least one of them ought to have known better.

<Of course, it makes me wonder about the other ninety-nine projects they criticized. But I digress once again.>

I read somewhere that McCain is up for re-election in Arizona this year, and he is in a tough fight. Perhaps that motivated him to issue this report. That still does not justify a cheap shot.

Don’t classy politicians have better things to do with their time than beat up on a research proposal they cannot possibly understand? Don’t classy politicians have the ability to hire staff who are grownups and do thoughtful homework before issuing reports?

1 Comment »

  1. Fact: Peer-review insulates.

    For someone outside that review process, this shielding is annoying.

    For someone inside that shield, it’s quite useful.

    That anyone would love to expand their group’s influence into other regions is tragic common sense. We should not expect too much of politicians; they are human, like the rest of us and respond to incentives. We can create institutions and policies that protect us from our darker selves. A good first step is keeping them out of the review process.

    Comment by Corban — December 17, 2010 @ 10:45 am | Reply


RSS feed for comments on this post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: