stripy revisited

15 years of PLOS ONE and some stripes

PloS One has an interesting blog post celebrating 15 years since that adventure began with interviews of former member of the editorial team, Damian Pattinson, Ginny Barbour, Matt Hodgkinson, Iratxe Puebla and Joerg Heber. During that period PloS One published a quarter of millions of articles containing probably ~ one million figures so I was amazed to see that the one figure included in that blog post comes from our paper on the interpretation of stripy nanoparticles images (Stirling et al, Critical Assessment of the Evidence for Striped Nanoparticles ). To be fair, the reason for this choice has nothing to do with the merits of our article. Here is the relevant excerpt from the blog post

In some cases, difficult editorial situations led to innovative solutions that further advanced PLOS’ mission. Iratxe Puebla, Associate Director at ASAPbio, remembers a time where PLOS ONE’s publication criteria and Open Access publishing model drove knowledge forward:

PLOS ONE was created to remove barriers: for authors to publish their work (related to scope or perceived impact) and for readers to access and reuse scientific content. Looking back at the many initiatives and papers I was involved with during my time at PLOS, there is one article that exemplifies this goal of facilitating openness.

In 2014 PLOS ONE handled a paper that reported a re-analysis of previous publications reporting the creation of “striped nanoparticles”. The authors completed a re-analysis and critique of those findings and wished to publish their work in a journal so that it would be part of the scientific record, on the same ground as the original articles. The authors had had trouble getting earlier critiques published in journals, and decided to submit the paper to PLOS ONE. This is where the first barrier went down: PLOS ONE would not reject the manuscript because it reported a re-analysis or because it relied on previously available data, the evaluation would focus on the rigor of the methodology and the validity of the conclusions.

The paper underwent a thorough peer review process and was accepted. But then we encountered a dilemma: the re-analysis required comparisons to images in the original publications where the journals owned copyright. Should we ask for permission to publish the images under a single-use license or request to republish them under the CC BY license used by PLOS ONE? While the former would have been the traditional (and easier) approach, we chose to pursue the latter. Why? Because the journal wanted to make all its content be available for reuse, for both humans and machines, without having to check individual figures in individual articles for the permitted uses. The PLOS ONE team worked with the authors and the publishers of the original articles, and we were pleased that they agreed to have the images republished under the CC BY license. As a result, the full article, including all images, is available for reuse without license-related barriers.

Finding that resolution was not so easy and we were quite frustrated at the time as this post (How can we trust scientific publishers with our work if they won’t play fair? Julian Stirling) illustrates .

And the stripy nanoparticles saga tested PLOS ONE’s publishing platforms in other ways too. As the bottom of this post (Identity theft: a new low in the stripy nanoparticles controversy) recalls, it was too easy to create profiles & comment on papers with a false identity. A fake “Dr Wei Chen” and a fake “Dr Gustav Dhror” left 10s of comments on our article.

Do striped nanoparticles exist? Figure 3 from Stirling J, Lekkas I, Sweetman A, Djuranovic P, Guo Q, Pauw B, et al. (2014) Critical Assessment of the Evidence for Striped Nanoparticles. *PLOS ONE* 9(11): e108482. https://doi.org/10.1371/journal.pone.0108482

A welcome Nature Editorial

I reproduce below a comment I have left on this Nature editorial entitled “Go forth and replicate!“.

Nature Publishing Group encouragement of replications and discussions of their own published studies is a very welcome move. Seven years ago, I wrote a letter (accompanying a submission) to the Editor of Nature Materials. The last paragraph of that letter read: “The possibility of refuting existing data and theories is an important condition of progress of scientific knowledge. The high-impact publication of wrong results can have a real impact on research activities and funding priorities. There is no doubt that the series of papers revisited in this Report contribute to shape the current scientific landscape in this area of science and that their refutation will have a large impact.” [1]

The submission was “Stripy Nanoparticles Revisited” and it took three more years to publish it… in another journal; meanwhile Nature Materials continued to publish findings based on the original flawed paper [2]. The ensuing, finally public (after three years in the secret of peer review), discussions on blogs, news commentary and follow up articles were certainly very informative on the absolute necessity of changing the ways we do science to ensure a more rapid discussion of research results [3].

One of the lessons I draw from this adventure is that the traditional publishing system is, at best ill suited (e.g. Small: three years delay), or at worst (e.g. Nature Materials) completely reluctant at considering replications or challenges to their published findings. Therefore, I am now using PrePrints (e.g. to publish a letter PNAS won’t share with their readers [4]), PubPeer and journals such as ScienceOpen where publication happens immediately and peer review follows [5].

So whilst I warmly welcome this editorial, it will need a little more to convince me that it is not a complete waste of time to use the traditional channels to open discussions of published results.

[1] The rest of letter can be found at https://raphazlab.wordpress.com/2012/12/17/letter-to-naturematerials/
[2] The article was eventually published in Small (DOI:10.1002/smll.201001465

2 comments on PubPeer

); timeline: https://raphazlab.wordpress.com/2012/12/20/stripy-timeline/
[3] https://raphazlab.wordpress.com/stripy-outside/
[4] https://raphazlab.wordpress.com/2015/11/16/pnas-your-letter-does-not-contribute-significantly-to-the-discussion-of-this-paper/
[5] https://raphazlab.wordpress.com/2015/11/17/the-spherical-nucleic-acids-mrna-detection-paradox/

Nanoparticles & cell membranes: history of a (science) fiction?

UPDATE (13/05/2022): A careful reader, Wytske Hepkema (PhD student in the NanoBubbles project) has discovered an error in this post. I wrote, about a claim in the Nel review that “This claim is not supported by a reference, but later in the article Nel et al refer to an earlier paper entitled “Ultrafine particles cross cellular membranes by nonphagocytic mechanisms in lungs and in cultured cells” by Marianne Geiser and colleagues“. In fact, the Geiser et al paper is not cited by Nel and I do not remember why I made a connection between those two articles. Correction of (a) science (blog) in action… Thank you Wytske!

One of the reason scientists, journalists and the general public are excited about nanoparticles is their supposed ability to cross biological barriers, including, the cell membrane. This could do wonders for drug delivery by bringing active molecules to the interior of the cell where they could interact with key components of the cell machinery to restore function or kill cancer cells. On the opposite side of the coin, if nanoparticles can do this, then there are enormous implications in terms of their potential toxicity and it is very urgent to investigate. But is it true? What is the evidence? How did this idea come into the scientific literature in the first place? I have been intrigued by this question for some time. It is the publication of an article about stripy nanoparticles magically crossing the cell membrane that led me to engage in what became the stripy nanoparticles controversy. It is this same vexing question that led me to question Merck/Mirkin claims about smartflare/nanoflare/stickyflare.

In the introduction of our article “The spherical nucleic acids mRNA detection paradox“, we describe the long history of the use of gold nanoparticles (“gold colloids”) in cell biology and conclude that

…, more than five decades of work has clearly established that nanoparticles enter cells by endocytotic mechanisms that result in their entrapment inside intracellular vesicles unless those nanoparticles are biological in nature and have acquired through evolution, advanced molecular tools which enable them to escape.

In the paragraph that followed, we were trying to make the point, in part using citation data of one of these 1950s pioneering articles, that this solid knowledge has been ignored in some of the thousands of recent articles on interactions of nanoparticles with membranes and cells that have appeared in the past 15 years. In his review of the first version of our article, Steve Royle criticises that latter paragraph (in his word, a “very minor” point):

I’m not a big fan of using number of Web of Science search results as an argument (Introduction). The number of papers on Gold Nanoparticles may be increasing since 2007, but then so are the number of papers on anything. It needs to be normalised to be meaningful. It’s also a shame that only 5 papers have cited Harford et al., but it’s an old paper, maybe people are citing reviews that cover this paper instead?

This is a fair point. While normalisation as well as more detailed and systematic searches might shed some light, it is rather difficult to quantify an absence of citation. Instead, I have tried to discover where the idea that nanoparticles can diffuse through membranes comes from. Here are my prime suspects (but I would be more than happy to update this post to better reflect the history of science and ideas so please leave comment, tweet, email), Andre Nel and colleagues, in Science, 3rd of February 2006, “Toxic Potential of Materials at the Nanolevel” :

“ Moreover, some nanoparticles readily travel throughout the body, deposit in target organs, penetrate cell membranes, lodge in mitochondria, and may trigger injurious responses.”

This claim is not supported by a reference, but later in the article Nel et al refer to an earlier paper entitled “Ultrafine particles cross cellular membranes by nonphagocytic mechanisms in lungs and in cultured cells” by Marianne Geiser and colleagues. These two papers, Nel et al, and, Geiser et al, have been cited respectively 5000 times and 850 times according to PubMed.

As early as 2007, Shayla Banerji and Mark Hayes had already challenged this idea of transport of nanoparticles across membranes in an elegant experimental and theoretical study which was a direct response to the two papers cited above “Examination of Nonendocytotic Bulk Transport of Nanoparticles Across Phospholipid Membranes“:

In accordance with these health concerns, Nel et al. have described some phenomena that can only potentiate fear of the negative health risks associated with nanotechnology.

[…]

Non-endocytotic transmembrane transport of large macromolecules is a significant exception to what is presently known about cell membrane permeability. Most early studies show that lipid bilayers are essentially impenetrable by molecules larger than water under physiological conditions: transport of most molecules across cell membranes is specifically cell-mediated by endocytosis.34 Endocytosis, unlike proposed passive, non-endocytotic transport, is an active cell-mediated process by which a substance gains entry into a cell. Specifically, a cell’s plasma membrane continuously invaginates to form vesicles around materials that originated outside the membrane: as the invagination continuously folds inward, the cell membrane constituents simultaneously reorganize in such a way that the material being transported into the cell is completely enclosed in a lipid bilayer, forming an endosome.35,36

[…]

The results suggest that a diffusive process of transport is not likely.

Figure 8 is particularly telling (!).

The article by Shayla Banerji and Mark Hayes has been cited 44 times.

Do nanoparticles deliver? Merck’s SmartFlares and other controversies

Leonid Schneider’s article starts with a summary of the stripy controversy and then moves on to the SmartFlare. Of particular interest is the quote from Luke Armstrong, formerly at EMD Millipore, which demonstrates that the company ought to be well aware that the probes detect nucleases rather than mRNAs. This begs the question of why they are still selling and advertising this product. Unfortunately, they did not provide a statement to Leonid. [Picture above is from Leonid’s post]

For Better Science

A large body of scientific nanotechnology literature is dedicated to the biomedical aspect of nanoparticle delivery into cells and tissues. The functionalization of the nanoparticle surface is designed to insure their specificity at targeting only a certain type of cells, such as cancers cells. Other technological approaches aim at the cargo design, in order to ensure the targeted release of various biologically active agents: small pharmacological substances, peptides or entire enzymes, or nucleotides such as regulatory small RNAs or even genes. There is however a main limitation to this approach: though cells do readily take up nanoparticles through specific membrane-bound receptor interaction (endocytosis) or randomly (pinocytosis), these nanoparticles hardly ever truly reach the inside of the cell, namely its nucleocytoplasmic space. Solid nanoparticles are namely continuously surrounded by the very same membrane barrier they first interacted with when entering the cell. These outer-cell membrane compartments mature into endosomal and then…

View original post 2,353 more words

Towards the end of the stripy controversy?

Last week saw the publication in PloS One of Quy Khac Ong and Francesco Stellacci’s response to Stirling et al “Critical Assessment of the Evidence for Striped Nanoparticles” published a year earlier (November 2014, I am one of the co-authors).

The controversy had started with our publication of Stripy Nanoparticles Revisited after a three year editorial process (2009-2012) and was followed by a large number of events at this blog, on PubPeer and a few other places.

Here is a short statement in response to Ong and Stellacci. Since theirs was a response to Stirling et al, Julian Stirling was invited to referee their submission (report).

We are pleased that Ong and Stellacci have responded to our paper, Critical assessment of the evidence for striped nanoparticles, PLoS ONE 9 e108482 (2014). Each of their rebuttals of our critique has, however, already been addressed quite some time ago either in our original paper, in the extensive PubPeer threads associated with that paper (and its preprint arXiv version), and/or in a variety of blog posts. Indeed, arguably the strongest evidence against the claim that highly ordered stripes form in the ligand shell of suitably-functionalised nanoparticles comes from Stellacci and co-authors’ own recent work, published shortly after we submitted our PLOS ONE critique. This short and simple document compares the images acquired from ostensibly striped nanoparticles with control particles where, for the latter (and as claimed throughout the work of Stellacci et al.), stripes should not be present. We leave it to the reader to draw their own conclusions. At this point, we believe that little is to be gained from continuing our debate with Stellacci et al. We remain firmly of the opinion that the experimental data to date show no evidence for formation of the “highly ordered” striped morphology that has been claimed throughout the work of Stellacci and co-workers, and, for the reasons we have detailed at considerable length previously, do not find the counter-claims in Ong and Stellacci in any way compelling. We have therefore clearly reached an impasse. It is thus now up to the nanoscience community to come to its own judgement regarding the viability of the striped nanoparticle hypothesis. As such, we would very much welcome STM studies from independent groups not associated with any of the research teams involved in the controversy to date. For completeness, we append below the referee reports which JS submitted on Ong and Stellacci’s manuscript.

Julian Stirling, Raphaël Lévy, and Philip Moriarty November 16 2015

The death of Nanonymous

Raphael,
The latest post on pubpeer:

https://pubpeer.com/publications/58199FBEA31FB5755C750144088886#fb23615

falsely attributes authorship to my handle. It is certainly the same troll who has been spamming recently. My response isn’t showing up on pubpeer yet, I post it below. My, this is getting quite silly. I’m afraid the “encyclopedia dramatica” aspects will detract from the serious nature of the debate. I will probably sign up as registered peer to avoid this.
——————————————————————————————

The above post (Unreg February 1st, 2015 6:04pm UTC ) was not written by the real nanonymous (me). The poster (who is quite clearly associated with all the gish galloping) uses a deliberately sacchrin tone, perhaps for the purposes provoking a “spot the impostor” cliche:

http://tvtropes.org/pmwiki/pmwiki.php/Main/SpotTheImposter

The persistence of anonymous handles is a measure of how untrustworthy a forum can be, the fact that the above Unreg has approriated a handle that is not his/her own (this was already done once on the fakerapha site, there is no doubt that it is the unreg above is the same identity that is responsible for that site).

This is an attack against the utility of PubPeer since we must presume that the vast majority of comments that do not mention Russian brides, Nigerean banks or viagara are written in good faith and that no one would appropriate an established handle in good faith. This clearly demonstrates that we need to figure out ways around these attacks, we all knew this was inevitable. While the PubPeer admins certainly don’t have time, one could envision some sort of digital signature type scheme in PubPeer 2.0. (there are probably a few good reasons we don’t want mere registered usernames).

Suppose I don’t have much choice but to retire the moniker, any further posts labelled “nanonymous” are not written by me, the identity that assumed the (relatively) novel name during the course of learning about the stripy saga. I may consider joining as a regular Peer with my academic address, but will need to consider carefully for a bit.

I always thought that no one could possibly vest so much time and effort into trying to obfuscate facts about (what are to that vast majority of people) obscure subjects like nanoparticles but fakerapha/unreg/bionanochair/ProfSTM/gustav/weichen/etc. has proven me wrong. Learning that one is wrong anout something is perhaps the part that is most fun about internet discussions.

Nanostripe controversy in new twist

Simon Hadlington, writing for Chemistry World, reports on our new paper:

A simmering controversy over whether certain nanoparticle structures are merely instrument artefacts has boiled over into a bitter dispute, with a senior scientist alleging that he has become the victim of a personal vendetta, something that is strongly denied by researchers on the other side of the argument.

A significant section of the report deals with such accusations. These are not new. More interestingly, given the very small number of scientists who have expressed a public view on the controversy, it was notable to read the following from Paolo Samori, from the University of Strasbourg:

In my laboratory we have imaged Professor Stellacci’s particles and found that these particles indeed have stripes on them. The images show clear features that are invariant with imaging parameters (scan rate, scan angle, feedback loop, etc) and hence they can ascribe to true tip sample interactions […] (he then goes to discuss power spectral analysis, this is thoroughly address in Stirling et al, e.g. see fig 10)

I would like to invite Prof Samori to share these images “that indeed have stripes on them”, e.g. using FigShare. In the meantime, we can only relie on the published evidence. To my knowledge, the only image of stripy (?) nanoparticles published coming from Samori’s lab is from Biscarini et al and is reproduced below next to stripy nanoparticles from the original 2004 Jackson et al paper.

left from Fig 1a of Jackson et al (2004), right from Biscarini et al (2013), fig 1d

Reactions to this 2013 Biscarini et al were somewhat incredulous:

Mathias Brust: “And yet there are stripes”

Quanmin Guo: “Where are the stripes”

Philip Moriarty: “The Emperor’s new stripes“

The Rights Stuff: Copyright, Scientific Debate, and Reuse; By Damian Pattinson and Cameron Neylon

Damian Pattinson, Editorial Director of Plos One, and Cameron Neylon, Advocacy Director for PloS

We’ve all monkeyed around trying to sort out the ownership of published content. In the scientific community, copyright and its (mis)application in publishing has authors, publishers, and readers grappling with questions of what is legally possible, what is desirable, and what is “allowable” by any particular party.

The most recent example of these challenges can be found in a PLOS ONE article published yesterday by Stirling et al., which focuses on a re-analysis of images used by other research groups as evidence for the creation of “striped nanoparticles”. The scientific controversy is fascinating and has been covered on blogs and in social media, but the copyright issues that cropped up during the paper’s publication process are also noteworthy.

In the study, the authors re-analyzed key results from previous work written up in the literature by other researchers and wanted to share their findings by publishing them, formally adding to the scientific literature on nanoparticles. To provide context and more effectively discuss the data, the authors of the re-analysis included figures (images) from previous studies in their own paper. However, the original figures were published in journals that owned the copyright of all the written content. And here is where we ran into a problem, and one that was far from simple.

Looking at Nothing, Seeing a Lot. [Brian Pauw]

Brian Pauw

Today is a day of relief for Dr. Julian Stirling and his eight co-authors (with many looking forward to the response, including Raphaël Lévy). The paper released today opposes ten years of prolific work from a group claiming to have made and observed stripes on the surface of nanoparticles (c.f. Figure 0, Figure 1 in this post). While most of the work revolves around scanning probe microscopy (SPM), small-angle scattering also played a minor role (c.f. Figure 2 and this paper). This, coupled with modern approaches to publication, led to my inclusion in the (otherwise amazing) list of authors. Here is how this came to be.

Find out here

New paper published today: a major turn in the stripy controversy… or a non-event?

The article by Stirling et al is published today in PloS One [I have the privilege of being a co-author]. The publication occurs after several months of delay due to copyright issues (see here, here, here and here); the negotiations between publishers have been hard and the resolution is hailed as a victory for open access.

The paper is undoubtedly an important piece of work. It analyses in exquisite detail the stripy nanoparticle evidence (from the abstract):

Through a combination of an exhaustive re-analysis of the original data with new experimental measurements of a simple control sample comprising entirely unfunctionalised particles, we conclusively show that all of the STM evidence for striped nanoparticles published to date can instead be explained by a combination of well-known instrumental artefacts, strong observer bias, and/or improper data acquisition/analysis protocols.

Why do I ask whether its publication might be a “non-event”?

A preprint version of this article was uploaded to the arXiv nearly a year ago. ~~This arXiv version was the subject of post-publication peer review.~~ This arXiv article remains the article with the highest number of comments on the post-publication peer review website PubPeer (276 comments at the time of writing). The PubPeer thread has over 20 000 individual IP views. Two comprehensive and very supportive reports from PloS One referees reports were shared two months ago.

Given that the article has been in the public domain for nearly a year, validated through super-extensive-peer-review-beyond-anything-that-will-ever-happen-in-a-journal, what difference does it make that it is now officially published in a scientific journal?

First, the final version is not identical to the arXiv version. The most important difference is in the small angle neutron scattering section. This has been improved thanks to the inclusion of a new author, Brian R Pauw… [see how he became involved in this in his latest blog post].

Second, papers remain a career currency as discussed in a previous post (Scientific journals no longer necessary?). I am extremely pleased that Julian’s enormous work is recognized through the publication of this article. I am also very pleased that Predrag Djuranovic is a co-author. He was the first to question the existence of the stripes when he was a student in Stellacci’s group at MIT, and his criticisms, from almost a decade ago, are finally vindicated ‘in print'”.

Third, although it’s been two years since Stripy Nanoparticles Revisited (Cesbron et al) and one year since the release of Stirling et al on the arXiv — and that therefore everyone who needs to know should know that there is no solid evidence for the existence of stripes — articles based on the stripy concept continue to be accepted and published as if none of this had ever happened. For example, Francesco Stellacci’s group published in ChemComm this year an article where the first sentence of the abstract reads “Scanning tunnelling microscopy studies have found stripe-like domains on gold nanoparticles coated with certain binary mixtures of ligand molecules.” The article was submitted in May 2014 and published in July (PubPeer); it does not cite Cesbron et al nor Stirling et al. This is not an isolated slip of the peer review system: there is also this one in ACS Nano with a nice stripes cartoon in the TOC graphic, accepted in May 2014… and this one in Nanoscale, accepted in April 2014 (PubPeer). Maybe the PloS One publication will carry more weight than the arXiv preprint?

I do hope that this will be not a major turn, but the beginning of the closing of the stripy controversy, 10 years, 35+ papers, and significant public funding, after the publication of Spontaneous assembly of subnanometre-ordered domains in the ligand shell of monolayer-protected nanoparticles. Francesco Stellacci has been given the opportunity to provide a referee’s report on Stirling et al and while he has not allowed us to reproduce his report, we understand that he has submitted a ms to PloS One too. I am today uploading comments and links onto PubMed Commons so that scientists interested by these 35+ articles can easily find relevant post-publication peer review information.

Rapha-z-lab

The blog of Raphaël Lévy at Université Sorbonne Paris Nord