Cleaning up the scientific literature and DORA

There is a great guest post by David L Vaux at Retraction Watch on ‘what it’s like to retract one of your own papers in an attempt to clean up the literature‘. The post is followed by some interesting discussion, in particular this comment from Peer007 which I reproduce in full here:

There’s a very simple fact that is often ignored by publishers: scientific papers receive much more scrutiny after they are published than before. The recent somatic cell nuclear transfer human ES cell paper in Cell highlights this brilliantly–problems with the images were flagged almost immediately after the paper appeared on line, yet apparently these were not detected during the review process. We could call this sloppy review, but we are humans after all, and a few thousand pairs of eyes looking at something is bound to produce better results than the three or so pairs that reviewed the manuscript in the first place.

In cases like Vaux’s example, it took years of experimentation to get to the heart of the problem. This is not something that reviewers of the original paper could have picked up on. Unless we are going to require that data be independently reproduced before publication, it is inevitable that wrong conclusions will be published.

In other words, the published literature is fallible. Putting fraud aside, mistakes will be made, controls omitted, variables unaccounted for, and incorrect conclusions will be drawn. It is inevitable. Journals, in my opinion, need to adjust their posture to account for the fact that all papers are a work in progress, and be more receptive to publishing corrections, correspondence, and (worse case scenario) retractions.

But there is one crucial factor that scientists need to take to heart as we complain about the current situation and press for change. Journals like Cell, Science, and Nature are money-making entities there to serve the best interests of their publishers. They are not impartial keepers of the scientific record nor are they the scientific police. It’s much easier and more profitable for them to publish cool stories that may be incorrect and move on than it is to set the record straight. And we, the scientists, support this strategy by valuing publishing in these journals more highly than we do, say, society journals. The value of a well-timed Nature paper for a promotion or grant decision, for example, cannot be underestimated. We are the ones that make these journals sexy and are thus complicit in supporting the notion that, in terms of scientific success, it’s better to be first than to be correct.

I believe that the sort of change many of us recognize as being sorely needed begins with us. We need to change how we value different journals and how we measure scientific success. I am very pleased to see that almost 10,000 of us have signed the San Francisco Declaration on Research Assessment (http://am.ascb.org/dora/) which calls for a total revision of how the output of scientific research is evaluated and offers a very thoughtful set of guidelines for how this can be achieved. I encourage everyone to visit the site, sign on if so inclined, and begin implementing change.

In the comments, David L Vaux also points at this commentary published in 1992 in Nature about the difficulty of correcting the scientific record… related to an article published in JACS! Now that is really an extraordinary piece and I think I’ll have more to say about this…

If you need further ‘argument’ for signing up to DORA, you can read (and hopefully get annoyed by) this post at the Scholarly Kitchen.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s