We are anonymous (or not). We need you to join. We are (mostly) making scientific discussion in the open possible and easy.

The last thing you probably want to read is one more article about anonymity in (post-publication) peer review. The topic has been covered recently by Bastian, Blatt, Lawrence, Oransky, Moriarty & PubPeerNeuroskeptic, Schneider to name just a few. I am sorry. I’ll keep it short.

I have decided to sign the peer review reports I write as a referee. Yet, I insist that attacks against anonymity in post-publication peer review are unfair, misguided and counterproductive. These two positions might seem contradictory. Bear with me to the end of the post and, hopefully, you might agree they are not.

Social media (and in particular PubPeer) have played a role in pretty much all recent scientific controversies, in part because the traditional channels are at best inefficient and at worst useless. Journal editors and some authors do not know how to react when criticism of articles appear on these platforms. One possible reaction is to shoot the messenger. If it is anonymous, call them anonymous cowards and question the motives. If it is not, call it cyber bullying.

Philip Moriarty, a strong supporter of PubPeer, has nevertheless titled his blog contribution to this debate “We are anonymous. We are legion. We are (mostly) harmful.” With friends like this, who needs enemies… I, and others, have responded in the comments section.

We can argue all of 2016, but the key practical question are the following. Your colleague/student has read a paper and has some interesting comments that she would like to share with the world;

Say you have chosen the first option and your colleague/student has now posted her critique on PubPeer (anonymously). You happen to also know the authors.

These are very simple questions. How we answer them has implications. Peer review is central to our practice, yet publicly engaging in scientific discussion on someone else’s work is often seen as not nice. This is the cultural barrier that we need to break. Many authors make the choice of not responding to carefully crafted criticism of their work (whether the criticism is anonymous or not). A colleague recently contacted me querying my opinion on a paper. We exchanged a few emails. We agreed there were several problems and possible errors of interpretation. I suggested to share our critique on PubPeer thereby giving the original authors the opportunity of a reply (and the rest of the community the opportunity to contribute to the discussion). He replied (SIC, smilies included): “Sorry, not for me 🙂 I do not like sharing 🙂“. This is the culture we have to change. We did not even get to discuss the possibility of anonymity.

Leonid Schneider argues that most of PubPeer is not really post publication peer review. Instead, it is calling out fraud. While anonymity is OK when you call out fraud, it would problematic in cases of scientific arguments. Even if we accepted the latter (which I do not), the distinction is artificial. There is a continuum of practices from outright fabrication to cherry picking of data and extremely optimistic interpretation of results (Twitter convo on this point here), e.g. it is common that extensive statistical analysis of data is necessary to demonstrate fraud. Any analysis of published data is post-publication peer review, whether it results in new hypotheses, questions and clarifications, or suspicions of fraud.

I disagree that anonymity is a problem and I can see plenty of valid, honest, reasonable (and even anodyne) reasons why you might choose to comment anonymously. If you insist it is a problem, fine, but I hope you will agree that it is secondary to getting valuable critiques of published work in the open. And, most importantly, do not side with Blatt who is calling for authors not to respond to criticism of their work.

It is sometimes argued that anonymity in peer review is fine because there is someone, the editor, who knows the identity of the reviewer. The power asymmetry, and therefore the potential for abuse, is however much larger in formal peer review than in post-publication peer review. At PubPeer, an anonymous comment stands purely on its merits; a scientifically strong response from the authors will bring credit to the authors. In formal peer review, the anonymous referee comes with the prestigious vetting of the editor. Furthermore, editors who are often not experts and always stretched with time, rely largely on these reports for their decisions; those have an impact on career progression, eventually grant funding, etc. The accountability of reviewers is little to none. I have started to sign my reviews during the course of 2015 with some hesitations, but my commitment is now firm. I have had some positive feedback and in one case (where I had rejected the paper), an email from an author asking me for further advice on their revised ms before submitting elsewhere. For more reasons to sign your reviews, check this.

So, to conclude, here is my advice for 2016: contribute to PubPeer (anonymously, or not, I care little), sign your peer reviews… and publish them when possible.

 

 

 

 

 

Advertisements

5 comments

  1. Hi Raphael, many thanks for your reference! I made the distinction between “whistle-blowing” of data manipulations on the one hand and the scientific PPPR on the other for two main reasons:
    1. Accusing people of fraud or misconduct is a legally punishable offence. An author accused in this way can easily win in court, thus such comments should be formulated very carefully, and best of all, the identity of the whistle-blower should be protected (by anonymity or otherwise). However, noone can ever sue you in court or threaten you with lawyers for scientifically discussing their paper, however critically (provided you live in a democratic country, that is).
    2. I distinguish between PPPR and whistle-blowing because for the former, you need to be at least to some degree an expert in the relevant field to understand what the discussion is about. By this I do not mean a stellar publication record or a faculty position. Being a PhD student in the relevant field can suffice as expertise. This is why it helps to know who wrote that PPPR before investing as a field outsider your valuable time to try and make sense of it. As for the latter, anyone with a minimum of interest in the matter can grasp that digitally manipulating images or inventing numbers or patients is not OK at all.

    Liked by 2 people

  2. Hi, Raphael.

    Sorry for taking some time to respond to your important post.

    We agree on many things but the question of anonymity is, I guess, always going to be a bone of contention between us!

    Your first poll is hardly a good representation of my position…

    “If you do not have the courage and honesty to put your name to it, then you have to ask yourself why you are a scientist. Sign it or else”

    “Or else”? Really?!

    Even PubPeer, in their THE article, state that they would much prefer if comments online were made openly but that they see anonymity as being required (at the moment). My point has always been that we need to change the culture so that researchers will comment without hiding behind anonymity. But that doesn’t mean anything so draconian as “banning” anonymous comments (as some have misrepresented my position), nor does it entail an “Or else” clause!

    There are some good examples of where anonymity goes astray not only in the threads for our stripy nanoparticle paper (pre-print and the PLOS ONE article itself) but in the thread for Blatt’s editorial. What I found deeply amusing was one of the anonymous commenter asking Leonid Schneider to provide more information on his credentials. The irony there is rich.

    You can, of course, say that I’ve cherry picked an example. But if you look through that thread you will see a number of examples of where comments have been made that I am very confident would not have been made if the author knew they had to put their name to them. (One commenter even states explictly that one reason they comment under cover of anonymity is to provide cover in case they get things wrong. The author(s) of the work that is being critiqued don’t have that opportunity…).

    I have voted in the polls above. And in each case I have selected the first response. That’s hardly surprising! But can we not agree that we need to move towards a culture where we openly comment/critique science. That openness and intellectual courage is preferable to anonymity? Because if we agree on this, I cannot see how we move towards that goal without challenging the ubiquity of anonymity in online criticism.

    Philip

    Liked by 1 person

  3. Hi Philip

    Thanks for the reply. The second option in the first poll was not supposed to be a representation of your position. I accept it could be understood this way though. Happy to clarify in the post if you think it is necessary.

    I agree with you that openness is nicer. There is irony indeed in the request to Leonid to provide credentials, coming from an anon comment. But it matters zilch. Everybody can enjoy this irony. It is transparent and has zero impact. Furthermore, if such request was to come from a named Nobel Prize winner, it would be maybe not ironical, but just as stupid. A scientist recently told me he must be right because he has 5y h-index of 56 (I congratulated him and he missed the irony). Sorry, I diverge.

    I have a problem with your call to intellectual courage. I think (and we in fact agree on this too) that there should be zero need for courage because scientific critique is valuable, even when it is wrong, both for the person making the critique and for the person who has to respond. But this is not the society in which we live. In the society in which we live, the first author of the first stripy paper is in charge of a budget of ~ 350 million dollars and US scientists who might think of applying to this pot of money may prefer to make their contributions to the debate anonymously. I prefer if they do it anonymously that if they don’t do it at all. This is but one example. We live in a world with myriads of such perceived or real risks. For scientists like us who have a secure position, we are risking bad reviews in our next grant applications or papers. For younger scientists, the risk, real or perceived, is larger. I think it is unfair to criticize them for lack of courage, which is what you do, by implication, through your call to intellectual courage. I want them to contribute their intelligence to scientific discussion even if they don’t feel comfortable doing so under their name.

    My main concern – largely addressed by PubPeer – is lowering as much as possible the barrier to sharing critiques. Anonymity does that. The risk of unfair anonymous critiques seems to me infinitely lower than the risk of not benefiting from our collective analysis of papers. There is an enormous need for more shared discussion of papers. When there will be no need to be brave, then let’s insist on signing critiques. Until then, a lower profile encouragement is enough, especially when some use this as an excuse to dismiss post-publication peer review entirely (a la Blatt).

    Raphael

    Like

    1. You are quite right. They were more a rhetorical device rather than real polls and I am actually surprised by the number of people who did answer them.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s