Stirling

Towards the end of the stripy controversy?

Last week saw the publication in PloS One of Quy Khac Ong and Francesco Stellacci’s response to Stirling et al “Critical Assessment of the Evidence for Striped Nanoparticles” published a year earlier (November 2014, I am one of the co-authors).

The controversy had started with our publication of Stripy Nanoparticles Revisited after a three year editorial process (2009-2012) and was followed by a large number of events at this blog, on PubPeer and a few other places.

Here is a short statement in response to Ong and Stellacci. Since theirs  was a response to Stirling et al, Julian Stirling was invited to referee their submission (report).

We are pleased that Ong and Stellacci have responded to our paper, Critical assessment of the evidence for striped nanoparticles, PLoS ONE 9 e108482 (2014). Each of their rebuttals of our critique has, however, already been addressed quite some time ago either in our original paper, in the extensive PubPeer threads associated with that paper (and its preprint arXiv version), and/or in a variety of blog posts. Indeed, arguably the strongest evidence against the claim that highly ordered stripes form in the ligand shell of suitably-functionalised nanoparticles comes from Stellacci and co-authors’ own recent work, published shortly after we submitted our PLOS ONE critique. This short and simple document compares the images acquired from ostensibly striped nanoparticles with control particles where, for the latter (and as claimed throughout the work of Stellacci et al.), stripes should not be present. We leave it to the reader to draw their own conclusions. At this point, we believe that little is to be gained from continuing our debate with Stellacci et al. We remain firmly of the opinion that the experimental data to date show no evidence for formation of the “highly ordered” striped morphology that has been claimed throughout the work of Stellacci and co-workers, and, for the reasons we have detailed at considerable length previously, do not find the counter-claims in Ong and Stellacci in any way compelling. We have therefore clearly reached an impasse. It is thus now up to the nanoscience community to come to its own judgement regarding the viability of the striped nanoparticle hypothesis. As such, we would very much welcome STM studies from independent groups not associated with any of the research teams involved in the controversy to date. For completeness, we append below the referee reports which JS submitted on Ong and Stellacci’s manuscript.

Julian Stirling, Raphaël Lévy, and Philip Moriarty November 16 2015

 

 

Identity theft: a new low in the stripy nanoparticles controversy

If you are familiar with the controversy and wonder why a PloS One comment thread disappeared today, skip directly to recent events.


Jackson et al, Nature Materials 2004

Jackson et al, Nature Materials, published 18 April 2004

In 2004, Alicia Jackson (now deputy director of the Biological Technologies office at DARPA), Jacob Myerson and Francesco Stellacci (now Constellium Professor at EPFL) mis-interpreted a common scanning probe microscopy artefact for a real structure. They submitted their results to Nature Materials, which published the work. This error was to become the starting point of a controversy which, more than 10 years later, is still unraveling.

Predrag Djuranovic

Predrag Djuranovic was a PhD student in Stellacci’s group at MIT

In 2005, concerns were raised by Predrag Djuranovic, then a student in Francesco Stellacci’s group. According to Lauren Wolf writing for Chemical & Engineering News last year:

he discussed his concerns with his department head at MIT […]. A few months later, an official investigation of Stellacci launched at MIT and Djuranovic left Stellacci’s research group. MIT closed its review three years later, in 2008, finding Stellacci not guilty of academic misconduct. The university did, however, suggest that Stellacci should do additional work to substantiate his 2004 findings.”

MIT's Building 10 and Great Dome; adapted from this  pic

MIT’s Building 10 and Great Dome; adapted from this pic

MIT was not convinced by the published data but instead of sharing those concerns with the scientific community, they kept them secret and Stellacci was asked to “substantiate” what he had published. This –trying to substantiate a structure which had been postulated to explain an experimental artefact– is what he has now been doing for more than 10 years… with over 30 articles published.

stripy nanoparticles revisited

Stripy Nanoparticles Revisited, published 23 Nov 2012

The controversy entered the public domain in 2012 with the publication of our article entitled Stripy Nanoparticles Revisited in Small: I was not aware of Predrag’s work but had come to the same conclusion. Francesco Stellacci published his response alongside our paper. He, and co-authors, also published several other articles around the same period. At the end of last year, Julian Stirling, and co-authors, published in PloS One an extensive Critical Assessment of the Evidence for Striped Nanoparticles which demonstrates that there is no evidence for the existence of the stripy nanoparticles. A response from Francesco Stellacci is still expected at PloS One.

Scientists interested in the details of the scientific arguments can refer to the articles linked to above (and references therein) as well as the blog posts and PubPeer discussions.

I wish this was all about science and differences of interpretation. I have touched previously on the re-use of figures in different articles. This eventually led to two corrections, one at Nature Materials and one at PNAS.  An EPFL investigation was triggered, opened, and, eventually, following the report of an international independent panel, closed. A third article, pretty much entirely based on data re-use, has not been corrected.

I decided to start blogging about the controversy after the publication of our article in 2012. I used the blog to present our paper, discuss the response (without having to wait three years), publish an invited post from Predrad Djuranovic, other experts (Philip Moriarty, Mathias Brust, and Quanmin Guo), and many other things . Other blogs and the post-publication peer review platform PubPeer also became important fora.

Unfortunately, just as the traditional literature has its flaws and can be gamed by scientists who do not respect the rules, so do these new modes of scientific communication. I have argued earlier that the stripy controversy was a window into the scientific process revealing the serious failures of self correction mechanisms. It is also a window into the frightening flaws of online post-publication peer review – I say this as a strong and determined supporter of post-publication peer review.

Those who believe in stripes have never engaged openly in the online debate (they have even vigorously complained that it was taking place, likening it to cyberbullying). Instead, we’ve had, in chronological order:

  • a troll posting over a period of two days a large number of comments on this blog… he eventually was identified as an Editor of the journal.
  • A fakerapha blog (that still exists) and a fakerapha twitter (the account has been removed) who impersonated me on the Chembark blog.
  • A text book case of Gish Gallop by “unreg” (a debating technique named after, and often used by, creationists) at PubPeer, It is to their immense credit that Julian Stirling, Philip Moriarty and several anonymous peers have engaged, repeatedly and honestly with “unreg”… until it became absolutely beyond doubt that this was no ordinary – if a little heated – discussion. Maybe best to let Nanonymous, an anonymous but insightful and regular commentator of the controversy describe that thread:

    Wow, the same thing again and again, in spite of being addressed again and again.

    and later

    […] it is interesting to reflect on the motivations for comments like these. They seem to come from an identity that either: 1)Does not understand basic scientific reasoning.2)Is vested in obfuscating some fairly straightforward arguments against clear and obvious errors in scientific work.Perhaps both. It could be just some random troll, but clear evidence that there are people acting in very bad faith out there.   Nanonymous

  • Possibly that thread also includes sockpuppetry with “unreg” and “ProfSTM/BioNanoChair” being the same person.

These things happened on blogs or at PubPeer, i.e. platforms which accept anonymous writing. It is hard to think of how they could be completely avoided while retaining anonymous comments.  The discussions of the costs/benefits of anonymous post-publication peer review is ongoing (I agree with Peer 1 in that thread).

In the last few weeks, the stripy nanoparticles discussion has moved to PloS One which enables comments directly on the article. Plos One has a clear policy requiring contributors to “unambiguously identify themselves with their first and last names, their geographic location, […] Any registered user who is found to have provided false name or location information will have their account suspended and any postings deleted.” In spite of this:

  • A “Gustav Dhror” posted over 10 comments (Gish Gallop again) about “Problems with Fig 4”. There is simply no doubt that Gustav is not a real person but instead a sockpuppet. There is no person with that name who has ever published any article on anything, let alone STM, nanoparticles or surface science. Even first year PhD students get their names listed on some website in academic institutions but  “Gustav Dhror” draws zero hits on Google, or more precisely two: the PloS One thread and a baby name website. After I posted a comment (and contacted PloS One) raising the issue of false identity, that thread stopped… but a few days later…
  • A “Dr Wei Chen” started another thread entitled “Multiple flaws with this paper“. The user profile linked to a real and existing Wei Chen, Professor at Suzhou Institute of Nano-Tech and Nano-Bionics in China. Given the context and previous events, I eventually took the liberty of calling Wei Chen to check whether he was “Wei Chen”, commenter at PloS One. And he was not. He had no idea. He had never posted anything at PloS One. Someone had stolen his identity to post these comments. Someone who wanted to discredit our paper (with flawed pseudo-scientific arguments), was not prepared to do it under their own name, but was prepare to steal someone else’s identity to do it.

PloS One has been contacted and has now removed the “Dr Wei Chen” thread. They have been in contact with “Gustav Dhror” and should post a statement regarding that thread tomorrow. In the paragraphs above, I have used links to WebArchives where you can see how the pages looked like this morning.

New paper published today: a major turn in the stripy controversy… or a non-event?

The article by Stirling et al is published today in PloS One [I have the privilege of being a co-author]. The publication occurs after several months of delay due to copyright issues (see here, here, here and here); the negotiations between publishers have been hard and the resolution is hailed as a victory for open access.

The paper is undoubtedly an important piece of work. It analyses in exquisite detail the stripy nanoparticle evidence (from the abstract):

Through a combination of an exhaustive re-analysis of the original data with new experimental measurements of a simple control sample comprising entirely unfunctionalised particles, we conclusively show that all of the STM evidence for striped nanoparticles published to date can instead be explained by a combination of well-known instrumental artefacts, strong observer bias, and/or improper data acquisition/analysis protocols.

Why do I ask whether its publication might be a “non-event”?

A preprint version of this article was uploaded to the arXiv nearly a year ago. This arXiv version was the subject of post-publication peer review. This arXiv article remains the article with the highest number of comments on the post-publication peer review website PubPeer (276 comments at the time of writing). The PubPeer thread has over 20 000 individual IP views. Two comprehensive and very supportive reports from PloS One referees reports were shared two months ago.

Given that the article has been in the public domain for nearly a year, validated through super-extensive-peer-review-beyond-anything-that-will-ever-happen-in-a-journal, what difference does it make that it is now officially published in a scientific journal?

First, the final version is not identical to the arXiv version. The most important difference is in the small angle neutron scattering section. This has been improved thanks to the inclusion of a new author, Brian R Pauw… [see how he became involved in this in his latest blog post].

Second, papers remain a career currency as discussed in a previous post (Scientific journals no longer necessary?). I am extremely pleased that Julian’s enormous work is recognized through the publication of this article. I am also very pleased that Predrag Djuranovic is a co-author. He was the first to question the existence of the stripes when he was a student in Stellacci’s group at MIT, and his criticisms, from almost a decade ago, are finally vindicated ‘in print'”.

Third, although it’s been two years since Stripy Nanoparticles Revisited (Cesbron et al) and one year since the release of Stirling et al on the arXiv — and that therefore everyone who needs to know should know that there is no solid evidence for the existence of stripes — articles based on the stripy concept continue to be accepted and published as if none of this had ever happened.  For example, Francesco Stellacci’s group published in ChemComm this year an article where the first sentence of the abstract reads “Scanning tunnelling microscopy studies have found stripe-like domains on gold nanoparticles coated with certain binary mixtures of ligand molecules.” The article was submitted in May 2014 and published in July (PubPeer); it does not cite Cesbron et al nor Stirling et al. This is not an isolated slip of the peer review system: there is also this one in ACS Nano with a nice stripes cartoon in the TOC graphic, accepted in May 2014… and this one in Nanoscale, accepted in April 2014 (PubPeer). Maybe the PloS One publication will carry more weight than the arXiv preprint?

I do hope that this will be not a major turn, but the beginning of the closing of the stripy controversy, 10 years, 35+ papers, and significant public funding, after the publication of Spontaneous assembly of subnanometre-ordered domains in the ligand shell of monolayer-protected nanoparticles. Francesco Stellacci has been given the opportunity to provide a referee’s report on Stirling et al and while he has not allowed us to reproduce his report, we understand that he has submitted a ms to PloS One too. I am today uploading comments and links onto PubMed Commons so that scientists interested by these 35+ articles can easily find relevant post-publication peer review information.

How can we trust scientific publishers with our work if they won’t play fair?

Julian Stirling:

I am angry. Very, very angry. Personally I have never liked how scientific journals charge us to read the research that we produce, and that we review for them free of charge. But that is another debate for another day. What I really hate is how they abuse this power to stifle debate in the name of their business interests. This is now going to dramatically affect the quality of a paper into which I poured a huge amount of effort – a critique of the (lack of) evidence for striped nanoparticles. (More information can be found here and here.)

The oft-repeated mantra is that science is inherently self-correcting, as all science is up for debate. In theory this is true.

Read it all here.