(I pitched this feature to Nature after hearing about Vinay Deolalikar, a well-known researcher who claimed he’d solved one of the biggest puzzles in computer science: the P/NP problem. Nature had covered it as news, but I thought there was more to the idea of science being reviewed on social media, particularly in biology–as opposed to math or computer science, where online review is an accepted tradition. You can also read the online version, or download a pdf of the article.)
Blogs and tweets are ripping papers apart within days of publication, leaving researchers unsure how to react.
“Scientists discover keys to long life,” proclaimed The Wall Street Journal headline on 1 July last year. “Who will live to be 100? Genetic test might tell,” said National Public Radio a day later.
These and hundreds of similarly enthusiastic headlines were touting a paper in Science1 in which researchers claimed to have identified a set of genes that could predict human longevity with 77% accuracy — a finding with potentially huge implications for medicine, health policy and the economy.
But even as the popular media was trumpeting the finding, other researchers were taking to the web to criticize the paper’s methodology. “We expect that most of the results of this study will not have the same longevity as its participants,” sniped a blog posted by researchers at the personal genomics company 23andMe, based in Mountain View, California.
Critics were particularly perturbed by the genome-wide association study (GWAS) that the authors had used to identify their longevity genes: the centenarians and the controls in the study had been tested with different kinds of DNA chips, which potentially skewed the results.
“Basically anybody that does a lot of GWAS knows this [pitfall], which is why we all said it so fast,” says David Goldstein, director of Duke University’s Center for Human Genome Variation, who voiced his concerns to a Newsweek blogger the day the study appeared.
This critical onslaught was striking — but not exceptional. Papers are increasingly being taken apart in blogs, on Twitter and on other social media within hours rather than years, and in public, rather than at small conferences or in private conversation.