The last thing you probably want to read is one more article about anonymity in (post-publication) peer review. The topic has been covered recently by Bastian, Blatt, Lawrence, Oransky, Moriarty & PubPeer, Neuroskeptic, Schneider to name just a few. I am sorry. I’ll keep it short.
I have decided to sign the peer review reports I write as a referee. Yet, I insist that attacks against anonymity in post-publication peer review are unfair, misguided and counterproductive. These two positions might seem contradictory. Bear with me to the end of the post and, hopefully, you might agree they are not.
Social media (and in particular PubPeer) have played a role in pretty much all recent scientific controversies, in part because the traditional channels are at best inefficient and at worst useless. Journal editors and some authors do not know how to react when criticism of articles appear on these platforms. One possible reaction is to shoot the messenger. If it is anonymous, call them anonymous cowards and question the motives. If it is not, call it cyber bullying.
Philip Moriarty, a strong supporter of PubPeer, has nevertheless titled his blog contribution to this debate “We are anonymous. We are legion. We are (mostly) harmful.” With friends like this, who needs enemies… I, and others, have responded in the comments section.
We can argue all of 2016, but the key practical question are the following. Your colleague/student has read a paper and has some interesting comments that she would like to share with the world;
Say you have chosen the first option and your colleague/student has now posted her critique on PubPeer (anonymously). You happen to also know the authors.
These are very simple questions. How we answer them has implications. Peer review is central to our practice, yet publicly engaging in scientific discussion on someone else’s work is often seen as not nice. This is the cultural barrier that we need to break. Many authors make the choice of not responding to carefully crafted criticism of their work (whether the criticism is anonymous or not). A colleague recently contacted me querying my opinion on a paper. We exchanged a few emails. We agreed there were several problems and possible errors of interpretation. I suggested to share our critique on PubPeer thereby giving the original authors the opportunity of a reply (and the rest of the community the opportunity to contribute to the discussion). He replied (SIC, smilies included): “Sorry, not for me 🙂 I do not like sharing 🙂“. This is the culture we have to change. We did not even get to discuss the possibility of anonymity.
Leonid Schneider argues that most of PubPeer is not really post publication peer review. Instead, it is calling out fraud. While anonymity is OK when you call out fraud, it would problematic in cases of scientific arguments. Even if we accepted the latter (which I do not), the distinction is artificial. There is a continuum of practices from outright fabrication to cherry picking of data and extremely optimistic interpretation of results (Twitter convo on this point here), e.g. it is common that extensive statistical analysis of data is necessary to demonstrate fraud. Any analysis of published data is post-publication peer review, whether it results in new hypotheses, questions and clarifications, or suspicions of fraud.
I disagree that anonymity is a problem and I can see plenty of valid, honest, reasonable (and even anodyne) reasons why you might choose to comment anonymously. If you insist it is a problem, fine, but I hope you will agree that it is secondary to getting valuable critiques of published work in the open. And, most importantly, do not side with Blatt who is calling for authors not to respond to criticism of their work.
It is sometimes argued that anonymity in peer review is fine because there is someone, the editor, who knows the identity of the reviewer. The power asymmetry, and therefore the potential for abuse, is however much larger in formal peer review than in post-publication peer review. At PubPeer, an anonymous comment stands purely on its merits; a scientifically strong response from the authors will bring credit to the authors. In formal peer review, the anonymous referee comes with the prestigious vetting of the editor. Furthermore, editors who are often not experts and always stretched with time, rely largely on these reports for their decisions; those have an impact on career progression, eventually grant funding, etc. The accountability of reviewers is little to none. I have started to sign my reviews during the course of 2015 with some hesitations, but my commitment is now firm. I have had some positive feedback and in one case (where I had rejected the paper), an email from an author asking me for further advice on their revised ms before submitting elsewhere. For more reasons to sign your reviews, check this.
So, to conclude, here is my advice for 2016: contribute to PubPeer (anonymously, or not, I care little), sign your peer reviews… and publish them when possible.