There is a fascinating open peer review happening on the post-publication peer review platform PubPeer. In six days, over 70 comments have been left on the thread of our submitted article (which is already available as a preprint on the ArXiv, the article is submitted to PloS One).
The discussion started with an optimistic and supportive comment by Peer 1 stating that our paper, Stirling et al, “should finally lay to rest the whole striped nanoparticles controversy“. That comment also included the suggestion that the article should have been submitted to Nature Materials rather than PloS One which prompted an interesting discussion on the choice of journal; excerpts: Peer 2: “PLOS is open access and not for profit. I do not see that PLOSone is a “worse” journal than Nature Materials or one of the “glam” journals.” Julian: “openness should be at the heart of science, especially when things become disputed“. Philip then quoted a correspondence with an Editor who indicated that his/her journal does not publish “papers that correct, correlate, reinterpret, or in any way use existing published literature data.” That prompted further discussions as well as an excellent post entitled “Science is interpretation“at blogs.discovermagazine.com by Neuroskeptic.
The discussion then took a different turn:
— PubPeer (@PubPeer) January 5, 2014
I won’t attempt to summarize the scientific arguments here. The discussion continues as of this morning. Philip and Julian went to some extreme efforts to address all the long, convoluted (and repetitive) points made by this defender, including preparing a slide show which I have already shared here in the previous post.
The most recent discussions have centered on the contrast between the clearly visible stripes in the early papers and their absence in the most recent articles, e.g. this (from another Unregistered user): “The proof that the 2004 experiment are artifacts is that Stellacci himself can not reproduce the STM images with clear stripes. […] Things are pretty clear to me. All these papers are just overselling bad experiments into a fancy story with nice cartoons. Period.” and this from Peer 7 “In their recent papers (ACS Nano and Langmuir 2013) when proper imaging conditions are used with their particles, the stripes are not visible at all and the authors must use complicated (and in my opinion wrong) image treatment procedures to extract a vague correlation length and do not admit that they were wrong in the first place.”
Even the “stripey defender” admits that artifacts are present in the seminal 2004 paper, but we are unlikely to see a retraction of this article. I have elaborated on that point at PubPeer and reproduce here my comment in full:
Philip above suggests that there is a “tacit admission” in the 2013 stripy papers that the 2004 results can’t be reproduced.
There is no admission in the 2013 Stellacci papers, not even a ‘tacit’ one, that the 2004 stripes may be an artifact, nor even that they can’t be reproduced, this even though:
* that was already demonstrated (but not published) by Predrag Djuranovic, Stellacci’s student at MIT, in 2005: https://raphazlab.wordpress.com/2012/12/11/seven-years-of-imaging-artifacts/
* we have already published strong arguments showing that the stripes were an artifact end of 2012 (and Francesco had read our paper already in 2009):https://raphazlab.wordpress.com/2012/11/23/stripy-nanoparticles-revisited/
* as noted by us and multiple peers above, the STM images in the 2013 papers clearly show that the 2004 stripes cannot be reproduced,
BUT, critically, the 2013 articles text explicitly says exactly the contrary, i.e. it says that the data prove the existence of the stripes observed in 2004, e.g.:
Biscarini et al: “The analysis of STM images has shown that mixed-ligand NPs exhibit a spatially correlated architecture with a periodicity of 1 nm that is independent of the imaging conditions and can be reproduced in four different laboratories using three different STM microscopes.”
“We have used the method to compare data across di?ent laboratories and validate the data acquired earlier.29, 35”
ref 29 is the 2004 Nature Materials article while ref 35 is its first follow-up, Jackson et al 2006 (JACS)
Ong et al:
“The shape of these ?d?omains has a striking resemblance to the results of simulations published in ref 28.”
“In the case of NP2 the spacing that we observe is the same spacing reported for these types of particles in the past in ref 19. We believe that we image molecular features and not features related to the gold core, as high-resolution transmission electron microscopy images of these particles do not show any evidence of protrusions with these spacings (see Figure S12)”
ref 28 is Singh et al (simulations)
ref 19 is Hu et al; Hu et al 2009 which contains multiple instances of data re-use in the figures, and use the data from 2004 for its *rigorous* statistical analysis (the latter only became apparent when the data were released last year).
The glaring contradiction between the what-the-data-show and what-the-authors-say-the-data-show is one among many of the extraordinary aspects of the saga. What is even more difficult to understand is that these articles passed peer review in 2013 when many of the arguments were already in the public domain.
The 2004 paper should have been retracted in 2005. Unregistered 4:42am says “If the 2004 images cannot be reproduced, that makes it clear for just about anyone to see that all of the follow on work should be reviewed by very critical eyes.” That is one reason why we are very unlikely to see an admission that the 2004 paper is based on an artifact, let alone retracted. Immediate and voluntary admission of an error would probably not having carried much of a penalty* but ten years and 30 articles later, the stakes are possibly higher. *http://www.slate.com/articles/health_and_science/science/2012/10/scientists_make_mistakes_how_astronomers_and_biologists_correct_the_record.html
I truly and genuinely do not know what will happen now in this field. Will Stirling et al -with the help of the discussion happening here and elsewhere- be more efficient at correcting the scientific record than Cesbron et al 2012, or, will we see more stripy articles in 2014 than in 2013? The experiment about science continues… Theories and predictions welcome although unfortunately there will be no possibility of reproducing the experiment.