The stripy controversy as a window into the scientific process*

* h/t Retraction Watch for the title

Stripy Nanoparticles Revisited was published a year ago.

Science is self correcting? [part one: doing the right thing]

At the core of the scientific process as we imagine it, there is the principle that science is self-correcting. Errors are made, but then, if they have any bearing on future studies, they get corrected. That principle failed (and continues to fail) dramatically. Consider the following.

The first “stripy nanoparticles” article was published in 2004 in Nature Materials. The flaws in that article were many and obvious but colleagues most qualified to identify those flaws, i.e. SPM experts, were not interested enough to react: they privately laughed but publicly did nothing. Correcting the flaws in the literature was “somebody else’s problem“.

In 2005, a graduate student in Francesco Stellacci’s lab at MIT also recognized that the observed stripes were a scanning artifact. He did control experiments and showed that the same stripes occur in the absence of particles on the same instrument with the same settings. He also did simulations that explain how those patterns can be generated by the STM feedback system.  At that point, retracting the Nature Materials paper would have been “doing the right thing“. It was not retracted.

In 2006, the first article was followed  by a second article (in J Am Chem Soc), which acknowledges the presence of the artifact, but desperately tries to demonstrate that the features on the particles are the “reality” (with no less than 18 figures). The article is surreal – it makes the case that the artifact is present everywhere except on the particles, where those stripes which look just like the artifact, are instead, the “reality”. An extraordinary case of confirmation bias.

What happened in 2006 is the very exact opposite of self correction. Predrag’s demonstration was simple and clear but remained confined within MIT [it only became public when Predrag contacted me after the publication of our article last year]. Instead, in the official scientific record, no doubt were raised and an impressive pile of stripy figures and articles started to accumulate. That pile continues to grow at an impressive rate, even after the publication of our article. The count is now at about 35 articles from the same group, with more striking examples of confirmation bias.

Science is self-correcting? [part two: crossing the line]

Universities, journals and funding agencies have proclaimed ethical rules which thankfully generally align. One of those shared rules is that re-use of text or figures (even your own text or figures) is not allowed without proper attribution. If this rule is broken, you would therefore expect rapid action to correct the scientific record.

The stripy controversy includes several examples of data re-use and it is informative to look at how editors and institutions have responded to those cases and what have been the final (?) outcome.

Nature Materials was informed as early as 2009 of this issue since my letter to the editor accompanying our submission did mention that “The same image of the same nanoparticle is used in both the ChemComm and the Nature Materials articles.” That did not lead to any action (and our submission to Nature Materials was rejected). At the end of 2012, with the increased scrutiny brought by the controversy, several other data re-use cases were discovered. Editors of the corresponding journals were contacted by Dave Fernig. MIT and EPFL were also informed of these concerns as well as of other concerns regarding data sharing. The initial responses from Editors are reported by Dave here. Eventually, corrections were issued at both PNAS and Nature Materials. Both of these journals are members of the Committee on Publication Ethics (COPE).

The Journal of Scanning Probe Microscopy however is not a member of COPE and its Editor clearly indicated (within minutes of Dave’s email) that he had no intention to enforce any ethical policies. More worryingly maybe, the EPFL investigation has closed and while it has been successful in ensuring compliance of Francesco Stellacci with his obligation of sharing data, it has not led to the correction of the scientific record in the case of the Journal of Scanning Probe Microscopy article. Three figures of that article contain image reuse (from two different articles) without attribution and the analyses presented in other figures correspond to data from yet a third article. The extent of data re-use in this article would probably warrant a full retraction in a journal that follows COPE guidelines. However, the paper is not retracted; instead is still cited in all recent publications as evidence of thorough and rigorous statistical analysis of the stripes [more on this here for the interested reader].

I guess that the conclusion for this part is somewhat nuanced: the system kind of worked for PNAS and Nature Materials; the editors eventually did the right thing and followed COPE guidelines. The system did not work for the more obscure Journal of Scanning Probe Microscopy and EPFL/MIT/COPE rules were not enforced (that may still happen, or maybe there is an explication/justification for these instances of data re-use in which case I will happily publish a correction to that post and other relevant posts as soon as it is communicated to me). As an aside, I note that the Royal Society of Chemistry is a member of COPE and that therefore Nanoscale Editor-in-Chief (European office) is in charge of applying COPE guidelines to Nanoscale authors.

Science is self-correcting? [part 3: critiquing is not nice]

For science to be self-correcting, there should be an incentive for authors to publish failed replication studies and disseminate analyses which identify fundamental flaws in the peer reviewed literature. Quite clearly there is currently no such incentive. I do sometimes get  asked why I engaged in this controversy. I have no personal conflict with Francesco Stellacci. I had not met him when I submitted a technical comment to Science, and, I had still not met him when I submitted the first version of Stripy Nanoparticles Revisited to Nature Materials. My reason to engage was simple: I had identified flaws in articles which had a significant impact on my field of research and I concluded that it was important to correct the record so that we could build on more solid foundations. I was conscious it would be difficult but I certainly had not predicted it would take three and a half years to publish our manuscript. I also had not predicted the extravagant events of the past 12 months, e.g. Predrag contacting me, a Nature Editor trolling my blog anonymously, the data re-use mentioned above, etc.

We need a system where confrontation of ideas is encouraged and is part of the norm, not one where it takes enormous amounts of effort and unreasonable length of time to publish a “revisited” paper. We need a system where engagement in scientific discussions is rewarded, not one where criticism (online or through the peer-reviewed literature) is seen as not nice “because we are all humans“. The latter quote is a reference to the now famous ACS Nano editorial (excellent responses at Chembark, Nature and Pubpeer). Various reasons for this specific editorial at this specific time have been proposed, from ongoing discussions at PubPeer of papers from one of the ACS Nano Editors to the uncovering by bloggers of a recent cases of fraud. Paul Weiss chief-editor of ACS Nano has denied that any single blogger were targeted but, like Paul Bracher, I certainly took it as a personal attack, especially since the latest stripy article was published in the same journal issue.

The “critiquing is not nice” line is not  very different from the initial response to Monica Byrne naming Boraz as her harasser.  This is inappropriate, not nice. A standard response to expressions of concerns about sexism or racism: we can’t talk about this, it is too serious and too damaging (to the reputation of the offender). There is one additional reason to stay silent. This is from Priya Shetty’s (excellent) piece about sexual harassment and the “deafening silence” that surrounds it:

Other than in whispered conversations in coffee shops or quiet meeting rooms, I’ve never known any woman to name the man involved, for the same reason that most women don’t speak up about sexual harassment in the workplace in general – science and journalism are both fields in which funding and job opportunities are increasingly precarious, and women stay silent for fear that their careers will suffer.

The same peer pressure applies in science around cases of bad science or misconduct and the grey area in between.

Conclusion?

None. You write it.