Most of the time, when someone gives a post a title that’s a question, they have an answer in mind that they’re trying to convince you of. This one is a sincere question, to which I don’t know the answer.
Science bloggers are erupting at the moment over a draft bill proposed by Lamar Smith about National Science Foundation funding. Here’s Phil Plait, for example:
To start, Rep. Lamar Smith (R-Texas), who is a global warming denier, by the way, is the head of the House Committee on Science, Space, and Technology. He has recently decided that the National Science Foundation—a globally respected agency of scientific research and investigation—should no longer use peer review to fund grants. Instead it should essentially get political permission for which research to fund.
(Incidentally, if you’re not reading Plait’s Bad Astronomy blog, you should be.)
This story, sadly, is extremely credible. Many Congressional Republicans, very definitely including Smith, are pushing an anti-science agenda and would love to take science funding decisions out of the hands of scientists. But when I went looking for the actual language in which Smith advocates removing peer review, I couldn’t find it.
The journal Science‘s piece on this would seem like a good place to go, especially as it links to a draft of Smith’s proposed legislation. The legislation would require the NSF director to certify that all grants are
1) “… in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science;
2) “… the finest quality, is groundbreaking, and answers questions or solves problems that are of utmost importance to society at large; and
3) “… not duplicative of other research projects being funded by the Foundation or other Federal science agencies.”
I can’t see anything in here to justify the claim that the legislation goes after peer review. If the legislation passed, the NSF director would presumably have to change the criteria given to the peer reviewers, not eliminate peer review or even alter it all that much.
That’s not to say that the legislation is a good idea — it definitely isn’t — but the response to it seems quite overblown. The first two criteria actually seem mostly harmless: I don’t think that the NSF director would have any trouble certifying that funded projects “advance the national … welfare … by promoting the progress of science” or that the various superlatives in item 2 apply. NSF could rephrase the current “intellectual merit” criterion to include both of those without changing much of anything about the review process. The “not duplicative” language in item 3 is arguably the worst part — some duplication is good — but I suspect that very few purely (or even mostly) duplicative grants get funded anyway, so as long as there’s a bit of leeway in interpreting the language I don’t think that this would make much difference either.
I’ve looked at a bunch of other pieces on the subject, and they generally repeat the “attacking peer review” claim without citing specific evidence. Hence the question in the title, to which I would sincerely like to know the answer.
I feel a bit bad about even raising the question, as it feels like I’m giving aid and comfort to the enemy. And I use the term “enemy” advisedly: there is a strong anti-science contingent in the US Congress, housed almost entirely in one political party, who do want to do very damaging things. But I’d still like to know.
I’ll give xkcd the last word:
I think the claims about peer review mostly come from the accompanying letter to the NSF asking to see the internal and external reviews of several specific proposals. Going from there to “political permission” is a slippery slope argument that I don’t find very convincing. Congress have always felt free to criticize particular grants (remember Proxmire’s “Golden Fleece” award?); while I disapprove of Congress trying to peek at the peer reviews, I don’t expect it to change that much.
eliminating duplication is not a problem until you replace the term, ‘duplicate’ with, ‘replicate’.
Does that term sound familiar? Exactly where would you place it within the scientific method, and how integral do you think it is to generating good, robust data, theory, and methods?
I agree with both of these comments. Let me expand a bit on the replication issue.
In principle, I agree that replication is important (see http://blog.richmond.edu/physicsbunn/2011/06/27/error-correcting-in-science/ and http://blog.richmond.edu/physicsbunn/2011/04/29/science-is-all-about-replication-of-results-right/ in the unlikely event you want more detail on this). But if you changed NSF policy tomorrow to forbid funding for replications, it’d make essentially no difference, because people already don’t get funding for replication. From my experience writing and evaluating NSF grant proposals, I can assure you that in order to get funded, a proposal has to promise to do new stuff. A proposal that doesn’t make such claims is going nowhere.
As a matter of principle, putting no-replication language into law is a bad thing. In practice, it’d make no difference.
(Just to be clear, I feel the need to repeat one thing yet again: I am not saying that the proposed law is a good idea. It’s decidedly not a good idea. I’m just suggesting that some specific criticisms of it are inaccurate or exaggerated.)