A response from Chad Mirkin’s group

Well nearly. Possibly as close as we might get.

If you have followed the Spherical Nucleic Acid / SmartFlare / StickyFlare story on this blog, you will know that we have raised doubts about the endosomal escape of these nanoparticles which are supposed to reach the cytosol of cells where they could detect mRNAs. We have even published an article on this topic. The Mirkin group has developed the technology and it has been commercialized four years ago for the live cell detection of mRNAs by EMD Millipore.

One PNAS paper on the topic was Briley et al (see here for letter to PNAS and what happened to that letter). In his PhD dissertation (Briley, W. E. (2016), William Edward Briley respond to our criticism. I reproduce the relevant section below. One might note that there is an incredibly simple way to determine the localisation of these particles inside the cells: electron microscopy, a technique which has been used for this exact purpose for over 50 years. We have used it. The results were unambiguous.

2.3 Commentary on the Endosomal Escape of SNAs
Though the endosomal escape of SNA nanostructures such as the Nanoflare and stickyflare is evident based upon their ability to provide sequence-specific information regarding RNA levels and locations within cells, one researcher has concluded that SNAs cannot escape from endosomes.[75] That researcher [That’s me!] is ignoring the many papers now that use such architectures for sequence-specific cell-sorting experiments. Indeed, if these architectures, which are taken up by scavenger-receptor mediated endosomal pathways,[68a] do not escape the endosome, then it is difficult to understand the reports by the many groups who have documented the sequencespecific function of SNAs (all of which require endosomal escape), in antisense gene regulation,[12, 44-45, 53] siRNA gene regulation,[20, 68b, 76] nanoflare gene detection,[47, 67a, 67c, 77] and sticky-flare gene detection.[78] Perhaps the best demonstration of this sequence specificity is in the function of the multiplexed Nanoflare.[67a] This nanoconjugate, discussed in chapter 1, contains two targeting sequences specific to two different genes (actin and survivin). When cells treated with multiplexed nanoflares were subjected to siRNA that specifically knocked down the expression of survivin, a corresponding decrease of fluorescence was observed in the nanoflare’s survivin-associated fluorescence (cy5), but not the actin-associated fluorescence (cy3) compared to control.[67a] Likewise, when cells were subjected to actin-targeting siRNA, a decrease in actin-associated fluorescence was observed, with no decrease of survivin-associated fluorescence.
These results indicate that the detection by nanoflares is unique to the targeted mRNA transcript. To rule out any possible effects of the fluorophores, the Cy5 and Cy3 dyes were switched to the other gene (actin-cy5, survivin-cy3), and again the corresponding sequence-specific responses were observed. Importantly, since the development of the multiplexed nanoflare, other research groups have independently developed nanoflare-like structures capable of sequence-specific detection of 3,[57] and even 4[79] genes simultaneously. Additionally, since the commercialization and sale of the nanoflare platform under the trade name Smartflare (Millipore), dozens of researchers around the world have participated in successful sequence-specific gene detection.[80]
Further evidence of SNA endosome escape can be seen visually through analysis of the sticky-flare construct. If sticky-flares were incapable of escaping endosomes, one would expect consistent colocalization with endosomes. However, this is not the case. The patterns of β-actin targeted sticky-flares, when used in HeLa cells, instead exhibit localization around mitochondria. If such structures were limited to endosomes, it is inconceivable that they would co-localize with specific organelles. This is consistent with mitochondrial localization of multiple RNAs which
has also been identified by other groups, using multiple techniques, in HeLa cells.[74] Sticky-flare release from the endosome was further confirmed by designing sticky-flares targeting a second sequence, a U1 short nuclear RNA (snRNA). U1 snRNA is known to traffic from the cytoplasm to the nucleus. Indeed, cells treated with U1-targeting sticky-flares exhibited specific fluorescence within the nucleus. The pattern observed indicates nuclear localization through endosomal escape and sequence-specific tagging of an RNA that is actively transported into the nucleus. Again, this would not be possible if such structures were confined exclusively to endosomes. Speculation that SNAs do not escape endosomes has been fueled by the observation that the fluorescence pattern of β-actin in MEFs is punctate, which has been interpreted as an indication that the sticky-flares are simply trapped in endosomes. However, β-actin is well known to exhibit punctate fluorescence in many cases, an observation made by others in multiple cell lines, including MEFs.[81] Punctate fluorescence is very common in RNA-labeling studies and well known by researchers familiar with FISH.82 This is due to the fact that RNA is often packaged into large RNA-containing granules, which facilitates transport and translational control of the included transcripts.[82b, 83] Such packaging into granules has been extensively studied using β-actin mRNA. Thus, the fact that sticky-flares targeting β-actin were packaged into granules as previously observed, while U1-targeting sticky-flares were specific to the nucleus in the same cell line, demonstrates the functionality of the construct.
Taken together, the success of the many groups who use flare architectures for the detection and knockdown of RNA in cells, and the work of dozens of labs studying related nanoparticle constructs, provide unambiguous evidence of the ability for such architectures to escape the endosome and participate in reactions exclusive to the cytosol. The mechanism of endosomal escape for nanoparticle-based vehicles is currently unknown and is an interesting and important question that is actively being investigated by many in the field.[84]

Briley, W. E. (2016). Investigation and manipulation of the local microenvironment of spherical nucleic acid nanoconjugates (Order No. 10117274). Available from ProQuest Dissertations & Theses Global. (1795522748). Retrieved from https://search.proquest.com/docview/1795522748?accountid=12117

Opening up peer review: the peculiar case of PNAS contributed papers

Proceedings of the National Academy of Sciences (PNAS) has two paths for submission of research articles, one standard and one less so, the famous contributed track where the submitting author has to be a member of the National Academy of Sciences of the United States of America. Peter Aldhous reviewed in 2014 this inside track and those who use it more often. He describes the contributed track as follows: “This unusual process allows authors to choose who will review their paper and how to respond to those reviewers’ comments“.

There are two elements of transparency and accountability to counterbalance this conflict of interest of having an author acting as the editor of their own article: it is recorded on the paper that it is a contributed paper, and, the names of the referees (chosen by the author) are also published. It is interesting (maybe) to note that this ancient house of PNAS has a system there which is pretty similar to what has been recently proposed as a disruptive innovation in scientific publishing by Jan Velterop and implemented by ScienceOpen, i.e. peer review by endorsement (PRO). There are a couple of differences. The first one is that PRO at ScienceOpen is open to everyone, not just National Academy members. The second one is that not just the names of the reviewers, but also the content of the reviews that is shared in PRO.

As readers of this blog will know, David Mason and I have recently challenged a particular contributed PNAS paper by the Mirkin group on StickyFlares. The discussion can be found at PubPeer (the authors did not engage). We requested some data which (after some efforts) we eventually obtained. We wrote a letter to the Editor. Which was eventually rejected by Inder Verma editor of PNAS. The letter is available at BiorXiv.

Given that neither PubPeer nor the PNAS letter to editor enabled to get any answers from the authors to our substantial criticism, we were curious to know if any of the referees had maybe raised similar issues and, if yes, how the authors had replied. Dave therefore wrote to the referees to ask whether they would share their reports. The response was negative; they could not share their reports because “referee comments from peer review have to be kept confidential as it is an essential part of maintaining the integrity of the peer review process“. I was rather surprised by this response and was moved to write the following:

01/10/2015

Dear Shana, Chris

I am Raphael; Dave is a member of my group. Apologies for pitching in and for a rather long response!

Thank you both for your replies. [paragraph edited out about the issue of whether the PNAS guidelines on choice of reviewers were followed; see the PubPeer discussion for more]

I fully share your commitment to defend the integrity of the review process, but I would urge to you to reconsider your decision to keep your comments confidential, precisely because it does not serve that very commendable aim.

It is worth considering for a moment what is the role of confidentiality and anonymity in the peer review process, and also who is in charge of guaranteeing that confidentiality and anonymity. In a traditional peer review process, the reports are confidential and anonymous: the justification is the protection of the reviewers from potential reprisals if they were to write a very critical review. The editors are in charge of protecting the confidentiality and anonymity: this is part of the contract between the editor and the reviewers. If the editor was to publish the reviews or/and, worse, reveal their identity, he/she would breach that contract and this would significantly affect the trust between future reviewers and the journal. I have myself pondered on publishing on my blog the reviews of a (rejected) submission of one of my papers, and was eventually convinced (though I am still entirely sure this was the right decision) not to do so by the detailed comments of an editor who did point out that the reviewers expected their reviews to remain confidential and that I would therefore breach their trust by doing so [1]. It is however a completely different matter for reviewers who can decide to forego their right to anonymity both immediately at the stage of the review process (“signing reviews”, usually, precisely with the motivation of increasing transparency and integrity of the review process), or later, for various reasons (nearly 200 000 examples at Publons, a site that enables and encourages reviewers to share their reports [2-3]). A recent prominent example of a reviewer sharing her reviews (on PubPeer) is Vicki Vance, who had reviewed several of the papers of Olivier Voinnet and noted serious problems (they were nevertheless published) [4]. I have never heard anyone suggesting that a reviewer who would decide to share their reviews of a paper after publication, i.e. their own scholarly evaluation of published work, would be damaging the integrity of the peer review process. I also really fail to see by what mechanism it could do so.

Obviously, the PNAS “contributed submission” path is another can of worms. Many would argue that it is in itself damaging to the integrity of the peer review process with this very unusual situation where an author chose its reviewers. In this specific case, it is hard to see any justification at all for the confidentiality of the reviews: it does not serve to protect the reviewers from potential reprisals from the author since the author has chosen its referees in the first place. The only thing it does is prevent the public (and in particular other scientists) to benefit from the insights that would be provided by sharing the reviews. I would argue that here even more than in any other case, sharing the reviews would be the best way to protect the integrity of the peer review process and therefore I hope you will reconsider,

Best wishes

Raphael

[1] https://raphazlab.wordpress.com/2013/01/03/nature-materials-peer-view-of-stripy-revisited-july-to-september-2009-confidential-or-not/ (see first comment in particular)
[2] https://publons.com/
[3] The scientists who get credit for peer review, Nature, 2014
[4] http://www.lab-times.org/editorial/e_600.lasso

Unfortunately, I did not get a reply.

Do nanoparticles deliver? Merck’s SmartFlares and other controversies

Leonid Schneider’s article starts with a summary of the stripy controversy and then moves on to the SmartFlare. Of particular interest is the quote from Luke Armstrong, formerly at EMD Millipore, which demonstrates that the company ought to be well aware that the probes detect nucleases rather than mRNAs. This begs the question of why they are still selling and advertising this product. Unfortunately, they did not provide a statement to Leonid. [Picture above is from Leonid’s post]

For Better Science

A large body of scientific nanotechnology literature is dedicated to the biomedical aspect of nanoparticle delivery into cells and tissues. The functionalization of the nanoparticle surface is designed to insure their specificity at targeting only a certain type of cells, such as cancers cells. Other technological approaches aim at the cargo design, in order to ensure the targeted release of various biologically active agents: small pharmacological substances, peptides or entire enzymes, or nucleotides such as regulatory small RNAs or even genes. There is however a main limitation to this approach: though cells do readily take up nanoparticles through specific membrane-bound receptor interaction (endocytosis) or randomly (pinocytosis), these nanoparticles hardly ever truly reach the inside of the cell, namely its nucleocytoplasmic space. Solid nanoparticles are namely continuously surrounded by the very same membrane barrier they first interacted with when entering the cell. These outer-cell membrane compartments mature into endosomal and then…

View original post 2,353 more words

Are StickyFlares smarter than SmartFlares?

Update (19/08/2015): Dave Mason has posted a detailed critique of this paper at PubPeer

Update (19/10/2015): We have submitted a response to this paper as a Letter to the Editor of PNAS. It is currently available as a preprint.

Update (16/11/2015): Inder Verma, Editor of PNAS, has decided that our letter “does not contribute significantly to the discussion of the StickyFlare paper.”

A quick post before I take off to Boston tomorrow for the American Chemical Society national meeting. I informed Chad Mirkin of my Monday talk where I will discuss the SmartFlares (talk on Monday, abstract). In his reply, he pointed me to a contributed PNAS paper they published in July on StickyFlares (Links: article, Northwestern press release). The questions that this technology raises are the same as the ones raised by the SmartFlares, as discussed in a previous post. Eight years after the initial NanoFlare paper, they are still not answered in this new article.

Check the latest results of our SmartFlare studies on the open notebook and data repository.

Five cases of data re-use

Quick reminder reproduced from my previous post entitled Three months of stripy nanoparticles controversy):

the stripy nanoparticle hypothesis was first proposed in Nature Materials in 2004 by the group of Professor Stellacci (then at MIT and now at the EPFL). This hypothesis now forms the basis of 26 articles by the same group, mostly published in high impact journals including Nature Materials, Nature Nanotechnology, Nature Communications, Science, Journal of the American Chemical Society, Small, etc:  -1, 0, 1, 2, 3, 4, 5, 6, 7 , 8, 9, 10, 11, 12, 13,13a, 14, 15, 16, 17, 18, 19, 20, 21, 22 and 23. {apologies for the strange numbering}

The issues of data re-use/self-plagiarism has already been discussed here and in a number of posts at David Fernig’s blog.

I have looked at the issue again, produced figures, and communicated the findings to the EPFL ombudsman today. The five cases are illustrated below. [Update 29/10/2013: Following the report of an international independent panel, the direction of EPFL has decided to close the case.]

 

Case 1

Paper 1: Alicia M. Jackson, Jacob W. Myerson and Francesco Stellacci, Nature Materials, 2004, 3, 330-336

Paper 2: A. Centrone, E. Penzo, M. Sharma, J. W. Myerson, A. M. Jackson, N. Marzari, and F. Stellacci, Proceedings of the National Academy of Sciences, 2008, 105, 9886–9891

Fig 1a of paper 1 is re-used four years later in Fig 1 of paper 2.

Update (March 25, 2013): this case has now led to a correction in paper 2, the STM image in paper 2 has been substituted for another STM image.

The image has been manipulated substantially (contrast, cropping, possibly some filtering). When concerns were raised, an editor at Proceedings of the National Academy of Sciences initially could not see that there was an instance of data re-use as documented on Dave Fernig’s blog.

Case 2

This is a case of data re-use within the same article: Alicia M. Jackson, Ying Hu, Paulo Jacob Silva, and Francesco Stellacci, J Am Chem Soc, 2006, 128, 11135-11149

Figure 1a is a crop from the image shown in Figure 8, with a change of contrast. That same image is also re-used in another article (Case 3).

Case 3

Paper 1: Alicia M. Jackson, Ying Hu, Paulo Jacob Silva, and Francesco Stellacci, J Am Chem Soc, 2006, 128, 11135-11149

Paper 2: Ying Hu, Benjamin H. Wunsch, Sahil Sahni, and Francesco Stellacci, Journal of Scanning Probe Microscopy, 2009, 4, 1–11

Figure 1a of paper 1 (also re-used in fig 8 paper 1, see case 2) is re-used in fig 2 of paper 2. The image is cropped differently.

 

 Case 4

Paper 1: Ying Hu, Oktay Uzun, Cedric Dubois, and Francesco Stellacci, J. Phys. Chem. C 2008, 112, 6279-6284

Paper 2: Ying Hu, Benjamin H. Wunsch, Sahil Sahni, and Francesco Stellacci, Journal of Scanning Probe Microscopy, 2009, 4, 1–11

Figure 3, top left panel of paper 1 is re-used in Fig 3, Fig 4 and Fig 5 of paper 2.

In Fig 5, the figure legend says ‘The particles imaged are the same ones of Figure 3’ (SIC). This is a misleading statement: it is not two images of the same particles but the same image presented with different contrasts.

Cases 3 and 4 together indicate that all of the experimental STM figures in paper 2 (i.e. Fig 2-5) contain data re-use (from two different articles). 

Case 5

Note that this has recently (February 28, 2013) been the subject of a Corrigendum at Nature Materials; a reference has been inserted in the figure legend.

Paper 1: Oktay Uzun, Ying Hu, Ayush Verma, Suelin Chen, Andrea Centrone and Francesco Stellacci, Chem. Commun., 2008, 196–198

Paper 2: Ayush Verma, Oktay Uzun, Yuhua Hu, Ying Hu, Hee-Sun Han, Nicki Watson, Suelin Chen, Darrell J Irvine and Francesco Stellacci, Nature Materials, 2008, 7, 588-595

Fig 1 of paper 1 is re-used in Fig 1 of paper 2. 

This re-used STM image of a single nanoparticle is the only published STM evidence for the ‘stripiness’ of this type of water-soluble stripy nanoparticles (2:1 MUS:OT) used in paper 2 as well as in recent publications (e.g. Biointerphases, 2012, 7, 1-4).

 

Insoluble contradiction

Background (skip if you have been following this from the beginning): The stripy nanoparticle hypothesis was first proposed in Nature Materials in 2004 by the group of Professor Stellacci (then at MIT and now at the EPFL). This hypothesis now forms the basis of 23 articles by the same group, mostly published in high impact journals including Nature Materials, Nature Nanotechnology, Nature Communications, Science, Journal of the American Chemical Society, Small, etc: 1, 2, 3, 4, 5, 6, 7 , 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22 and 23. In November 2012, we published Stripy Nanoparticles Revisited which was followed by a response from Professor Stellacci.

Insoluble contradiction; In Stripy Nanoparticles Revisited (Fig S4, reproduced below), we pointed to a contradiction in the solubility  reported for  the same type of nanoparticles in two “stripy” articles: Jackson 2004 and Centrone 2008.

The response from Professor Stellacci was that the particles in both studies are different:

Stripy Nanoparticles Revisited claims that there are contradictions between our statements on particle solubility in Jackson 2004 and Centrone 2008 […] . Yet Jackson 2004 describes octanethiol, mercaptoproprionic acid-coated particles, Centrone 2008 describes octanethiol, mercaptoproprionate-coated particles, […]. Why would or should all of these particles behave the same way in terms of solubility […] given that the molecules that coat them are different? [emphasis mine]

Centrone 2008 focuses on the structure-solubility relationships of stripy nanoparticles in solvents of different polarities. The only measurement of structure in Centrone 2008 is in Fig 1. As shown below, Fig 1 in Centrone 2008  is data re-use (without attribution in the figure legend) from Jackson 2004.

There are only two possibilities:

1) The particles in Jackson 2004 are similar enough to those in Centrone 2008  to justify using the former STM images to discuss the latter structure and solubility. In this case, the data re-use is ‘ordinary’ self-plagiarism but the contradictions we noted in Stripy Nanoparticles Revisited remain to be explained.

2) The particles in Jackson 2004 are different from those in Centrone 2008 which may help explain the differences in solubility but the data re-use is a more misleading form of data re-use.

Update 1 (15/02): Dave Fernig has contacted journal Editors about various data reuse in the stripy articles. Here is the response from PNAS:

“Thank you for bringing this to our attention. We have taken a look at the figures, and while they are similar, they are not identical. Also, the authors have referenced the earlier Nature publication in their PNAS article.”

After pointing out to PNAS that the images are indeed identical and directing them to check out the images side by side, with contrast, colour balance and scale adjusted, on Raphael’s blog, they came back on a much more positive note:

“Thank you for providing a link to Raphael Levy’s blog posting. Based on this additional information and clear explanation of the issue, we will follow up with the authors to discuss the matter.”

Update 2 (25/03): The PNAS article has now been ‘corrected’: the re-used image has been substituted for another one. It would therefore appear that the second possibility is more accurate.

 

Adapted from Jackson 2004 and Centrone 2008; the figure legend of Centrone et al reads: “Fig. 1. STM image of gold nanoparticles coated with a 2:1 ratio of OT/MPA
showing ordered phase separation in their ligand shell. (Scale bar, 25 nm.)
(Insets) Close up of nanoparticles showing the encircling, ribbon-like domains
(Left) and a corresponding simplified schematic diagram in which the red
pillars represent MPA, and the yellow represent OT (Right). (Scale bar, 5 nm.)”

Figure S4: Comparison of the results reported by Centrone et al., 2008 (black curve, left Y axis) and by Jackson et al., 2004 (red curve, right Y axis) for solubility in ethanol; The solubility was measured as a saturation concentration expressed in molar in the 2008 study, while in the 2004 study, the solubility scale (right axis) was defined as follows: “4 = highly soluble, that is, no precipitation visually observed, 3 = mostly soluble, that is, little precipitation observed over time with consequent slight decolouration of the solution; 2 = slightly soluble, that is, most of sample precipitated but a small coloration of the solution remains, 1 = totally insoluble.” The data points are shown with symbols, black and red continuous lines (3-points splines) are shown to facilitate the visualization of the trends.