SmartFlare

SmartFlare controversy: independent confirmation of endosomal localization

Check this previous post for a quick summary of the SmartFlare controversy, or read all SmartFlare-related posts if you are really passionate.

At the centre of the SmartFlare controversy is the rather simple question, from an experimental point of view, of how many Spherical Nucleic Acids (to use Chad Mirkin’s terminology), if any, escape the endosomal pathway.

In contradiction with Chad Mirkin’s many peer reviewed articles and EMD Millipore marketing material, we concluded (Mason et al, 2016) that the Spherical Nucleic Acids do not escape endosomes and do not detect cytosolic mRNAs.

A few days ago, Sven Budik et al, an Austrian group published their evaluation of the SmartFlare in the context of equine embryo development. They write: “In all positive cells,
regardless of whether they occurred in equine conceptus, trophoblastic vesicle or fibroblast cell culture, the fluorescence signal showed a spotted pattern that is in accordance with the observations of Mason et al. (2016).

They also used electron microscopy to look at the intracellular localization of the particles. Here is the relevant part of their discussion and conclusion (emphasis mine):

The present study indicates that the intracellular process of nanogold particle uptake is endocytic and endosomal with a lysosomal sorting after longer incubation periods. This finding is in agreement with results from HeLa cells in vitro (Gilleron et al. 2013). Similarly, nanoparticles injected intravenously were taken up by endocytosis and later
clustered in lysosomes primarily in macrophages (Sadauskas et al. 2007). The incorporation time of lipid nanoparticlecontaining short interfering RNA gold particles in HeLa cells was similar (Gilleron et al. 2013) to that demonstrated in equine trophoblast vesicles in the present study. Accumulation of SmartFlare probes in residual bodies may be a consequence of increased stability of the immobilised oligonucleotides adjacent to the nanogold particles due to enhanced nuclease resistance (Rosi et al. 2006). In accordance with the results of Mason et al. (2016), we observed no or very few nanogold particles free in the cytoplasm, confirming a primarily endosomal and lysosomal localisation.

This observation raises the question how a specific SmartFlare probe is able to detect its target mRNA located in the cytoplasm. One possible explanation for the generation of lysosome-located specific fluorescence signals by SmartFlare probes could be the existence of specific RNA sequences imported for subsequent degradation into lysosomes (Fujiwara et al. 2013). Further studies using qRT-PCR investigating the isolated lysosomal fraction before and after incubation with specific SmartFlare probes are necessary to confirm this hypothesis. An 18S RNA nano-flare probe had a dose-dependent cytotoxic effect on porcine fetal fibroblasts (Fu et al. 2016). In contrast, no cytotoxic effects or changes in morphology after incorporation of antisense oligonucleotide nanogold particles in a mouse endothelial cell line were observed by Rosi et al. (2006). In addition, in the present study, there was no evidence that incubation with the SmartFlare probes had a toxic effect on the equine cells tested, even at higher concentrations. This is in accordance with the results of Pan et al. (2009) demonstrating that 15-nm gold particles have only low cytotoxic effects compared with the detrimental effects of small 1.4-nm gold particles.

In conclusion, SmartFlare probes pass into early equine conceptuses at stages used for embryo transfer, as well as trophoblast vesicles and cells cultured in vitro. In these early ZP equine conceptuses, the time frame (.5 and ,24 h) for SmartFlare uptake would be suitable for practical applications in commercial embryo transfer programs. Therefore, these probes are suggested to be applicable to pre-implantation genetic diagnosis before transfer of these conceptuses to the recipient.

In summary, the authors’ results are entirely consistent with our observations. They conclude, quite reasonably, that if SmartFlares detect mRNAs whilst being in endosomes, they cannot directly detect cytosolic mRNAs. This is in direct contradiction with Mirkin et al and EMD Millipore. Then, they propose that if the SmartFlares work, they maybe detect mRNAs which are in endosomes. This an interesting hypothesis that will require further study and is very different from anything published by Mirkin and EMD Millipore (the relevant reference is here). Since Budik et al do not provide any evidence that the SmartFlares actually detect mRNAs in the first place, maybe a simpler explanation is that the SmartFlares signal is unspecific and result from the probe degradation by nucleases in endosomes.

A welcome Nature Editorial

I reproduce below a comment I have left on this Nature editorial entitled “Go forth and replicate!“.

Nature Publishing Group encouragement of replications and discussions of their own published studies is a very welcome move. Seven years ago, I wrote a letter (accompanying a submission) to the Editor of Nature Materials. The last paragraph of that letter read: “The possibility of refuting existing data and theories is an important condition of progress of scientific knowledge. The high-impact publication of wrong results can have a real impact on research activities and funding priorities. There is no doubt that the series of papers revisited in this Report contribute to shape the current scientific landscape in this area of science and that their refutation will have a large impact.” [1]

The submission was “Stripy Nanoparticles Revisited” and it took three more years to publish it… in another journal; meanwhile Nature Materials continued to publish findings based on the original flawed paper [2]. The ensuing, finally public (after three years in the secret of peer review), discussions on blogs, news commentary and follow up articles were certainly very informative on the absolute necessity of changing the ways we do science to ensure a more rapid discussion of research results [3].

One of the lessons I draw from this adventure is that the traditional publishing system is, at best ill suited (e.g. Small: three years delay), or at worst (e.g. Nature Materials) completely reluctant at considering replications or challenges to their published findings. Therefore, I am now using PrePrints (e.g. to publish a letter PNAS won’t share with their readers [4]), PubPeer and journals such as ScienceOpen where publication happens immediately and peer review follows [5].

So whilst I warmly welcome this editorial, it will need a little more to convince me that it is not a complete waste of time to use the traditional channels to open discussions of published results.

[1] The rest of letter can be found at https://raphazlab.wordpress.com/2012/12/17/letter-to-naturematerials/
[2] The article was eventually published in Small (DOI:10.1002/smll.201001465

2 comments on PubPeer

); timeline: https://raphazlab.wordpress.com/2012/12/20/stripy-timeline/
[3] https://raphazlab.wordpress.com/stripy-outside/
[4] https://raphazlab.wordpress.com/2015/11/16/pnas-your-letter-does-not-contribute-significantly-to-the-discussion-of-this-paper/
[5] https://raphazlab.wordpress.com/2015/11/17/the-spherical-nucleic-acids-mrna-detection-paradox/

How many people are using the #SmartFlares? Freedom of Information request provide insights

Quick summary of previous episodes for those who have not been following the saga: Chad Mirkin’s group developed a few years ago a technology to detect mRNAs in live cells, the nano-flares. That technology is currently commercialised by Merck under the name smartflares. For a number of reasons (detailed here), I was unconvinced by the publications. We bought the smartflares, studied their uptake in cells as well as their fluorescent signal and concluded that they do not (and in fact cannot) report on mRNAs levels. We published our results as well as all of the raw data

This question – how many people are using the SmartFlares? – is interesting because surely, if a multinational company such as Merck develops, advertises and sells products, to scientists worldwide, these products have to work. As Chad Mirkin himself said today at the ACS National Meeting in Philadelphia “Ultimate measure of impact is how many people are using your technologies“.

So, we must be wrong. SmartFlares must work.

But our data say otherwise, so what is going on?

One hint is the very low number of publications using the smartflares and the fact that some of those are not independent investigations. This, however does not tell us how many groups in the world are using the smartflares.

Here is an hypothesis: maybe lots of groups worldwide are spending public money on probes that don’t work… and then don’t report the results since the probes don’t work. That hypothesis is not as far fetched as it may seem: it is called negative bias in science publishing and it is one of the causes of the reproducibility crisis.

To test this hypothesis, we would need to know how many research groups worldwide have bought the smartflares, an information that I suspected Merck was not going to volunteer. So, instead, I made Freedom of Information requests to (nearly) all UK research intensive universities (the Russell group) asking whether they had evidence of smartflare purchase orders.

Some Universities (6) declined because it would have been too much work to retrieve the information but most (14) obliged. The detailed results are available here. They show that a minimum of 76 different purchases were made between the launch of the product and June 2016. The money spent is £38k representing 0.0013% of these UK universities research income. As far as I can see, none has resulted in a publication so far.

All I can say is that these data do not falsify our hypothesis.

And if after reading this, you are still unconvinced of the need to publish negative data, check the upturnedmicroscope cartoon (warning: scene of violence).

 

 

 

Chad Mirkin on Nano Hype

Chad Mirkin did a Reddit AMA yesterday (h/t Neil Withers).

Capture

 

(highlight mine)

Of these 1800 commercial products, 1700+ are in fact a single product, the famous Spherical Nucleic Acids/SmartFlares.

More on this blog and our paper (The spherical nucleic acids mRNA detection paradox) here.

With the risk of being accused of having cynical views…

SmartFlare Maths

SmartFlare are nanoparticle sensors which are sold by Merck and are supposed to detect mRNA inside live cells. The technology has been developed by Chad Mirkin. In his papers, the nanoparticles are called Nano-Flares or Spherical Nucleic Acids. I am saying “supposed to” because the central question of how those sensors could possibly reach the target that they are supposed to detect has not been addressed by Mirkin nor by Merck.

After evaluating the SmartFlare, we published recently our conclusions at ScienceOpen. We ran this research as an open science project, sharing our experimental results, analyses and conclusions in quasi real time using an open science notebook. All of the imaging data can also be consulted via our online Open Microscopy Environment repository.

Gal Haimovich, who reviewed our paper, first on his blog and then at ScienceOpen, suggested we should do some SmartFlare Maths (point 4 of his list of comments). This had been at the back of my mind for some time. There are various ways to look at this problem, but all those I have tried lead to the same conclusion that the protocols, results and conclusion published do not add up. Here is what I believe the simplest way to think of the SmartFlare Maths problem. As usual, comments and corrections would be very much appreciated.

Estimation of the number of SmartFlares per cell

SF-figure adapted from Giljohann

Adapted from Giljohan et al, Figure 1b

Estimate 1. SmartFlares are added to cells at a final concentration of 0.1 nM (following Merck’s protocol). For 400,000 cells and 20 μL (following Merck’s protocol), this would result in 150,000 SmartFlares per cell, assuming that all nanoparticles are uptaken.

 

Estimate 2. Giljohann et al  (Mirkin’s group) published a quantitative study of uptake of SmartFlares in various cell lines in 2007. From their Figure 1b, we can see that in the lower concentration range tested, there is a linear correlation between SmartFlare concentration in the medium and number of particles per cell. For cells exposed to a medium concentration of 0.1 nM, this would result in an uptake of 75 000 SmartFlares per cell. In the following discussion, we will use this lower estimate. With ~50 oligo probes per SmartFlare, this would give 3,750,000 oligo probes per cell.

Oligo probes per cell versus mRNA per cell

The copy number of any specific mRNA per cell depends on sequence, cell types, signalling events etc, but typically it ranges from a few copies to a few thousands of copies. Our estimate above indicates an excess of oligo probes of at least three orders of magnitude over the most abundant mRNA.

If just 0.1% of these probes would bind their target, it would block 3,750 mRNA resulting in silencing. However, Merck and Mirkin both report that there is no silencing effect in the conditions of these experiments. It follows that more than 99.9% of the SmartFlares do not bind their target mRNA.

Fluorescence background

Seferos

Reproduced from Seferos et al, Figure 1.

Seferos et al (2007, Mirkin’s group) show that in the absence of release of the probe, fluorescence value of ~30% of the total value after release is observed (in ideal test-tube conditions, i.e. in the absence of nucleases). This is presumably due to a non-complete quenching of the fluorescence. For the SmartFlares to work, we would therefore have to detect a variation of less than 0.1% over a background of ~30%.

 

Lab Times: “Flare up over SmartFlares”

Stephen Buckingham interviewed me for Lab Times

On the face of it, Millipore’s SmartFlares are meant to be a tool cell biologists dream of – a way of measuring levels of specific RNA in real time in living cells. But does it really work? Raphaël Lévy and Gal Haimovich are in doubt.

Raphaël Lévy, Senior Lecturer in Biochemistry at the University of Liverpool, UK, was so unconvinced about SmartFlares that he decided to put the technique directly to the test (The Spherical Nucleic Acids mRNA Detection Paradox, Mason et al. ScienceOpen Research). As a result, Lévy has found himself at the centre of a row; not only over whether the technique actually does the job but as to whether it can actually work, at all – even in principle. Lab Times asked Lévy why he is in doubt that SmartFlares really work.

Lab Times:  What’s all the fuss about SmartFlares?

Read it all here (page 50-51).

I can’t resist also quoting this bit of pf the final paragraph…

In interview, Lévy is reasonable and measured in tone. But he is no stranger to controversy and can deliver fierce polemic with style.

If you have not yet, you should also check Leonid Schneider’s earlier and more complete investigation.

Nanoparticles & cell membranes: history of a (science) fiction?

One of the reason scientists, journalists and the general public are excited about nanoparticles is their supposed ability to cross biological barriers, including, the cell membrane. This could do wonders for drug delivery by bringing active molecules to the interior of the cell where they could interact with key components of the cell machinery to restore function or kill cancer cells. On the opposite side of the coin, if nanoparticles can do this, then there are enormous implications in terms of their potential toxicity and it is very urgent to investigate. But is it true? What is the evidence? How did this idea come into the scientific literature in the first place? I have been intrigued by this question for some time. It is the publication of an article about stripy nanoparticles magically crossing the cell membrane that led me to engage in what became the stripy nanoparticles controversy. It is this same vexing question that led me to question Merck/Mirkin claims about smartflare/nanoflare/stickyflare.

In the introduction of our article “The spherical nucleic acids mRNA detection paradox“, we describe the long history of the use of gold nanoparticles (“gold colloids”) in cell biology and conclude that

…, more than five decades of work has clearly established that nanoparticles enter cells by endocytotic mechanisms that result in their entrapment inside intracellular vesicles unless those nanoparticles are biological in nature and have acquired through evolution, advanced molecular tools which enable them to escape.

In the paragraph that followed, we were trying to make the point, in part using citation data of one of these 1950s pioneering articles, that this solid knowledge has been ignored in some of the thousands of recent articles on interactions of nanoparticles with membranes and cells that have appeared in the past 15 years. In his review of the first version of our article, Steve Royle criticises that latter paragraph (in his word, a “very minor” point):

I’m not a big fan of using number of Web of Science search results as an argument (Introduction). The number of papers on Gold Nanoparticles may be increasing since 2007, but then so are the number of papers on anything. It needs to be normalised to be meaningful. It’s also a shame that only 5 papers have cited Harford et al., but it’s an old paper, maybe people are citing reviews that cover this paper instead?

This is a fair point. While normalisation as well as more detailed and systematic searches might shed some light, it is rather difficult to quantify an absence of citation. Instead, I have tried to discover where the idea that nanoparticles can diffuse through membranes comes from. Here are my prime suspects (but I would be more than happy to update this post to better reflect the history of science and ideas so please leave comment, tweet, email), Andre Nel and colleagues, in Science, 3rd of February 2006, “Toxic Potential of Materials at the Nanolevel” :

“ Moreover, some nanoparticles readily travel throughout the body, deposit in target organs, penetrate cell membranes, lodge in mitochondria, and may trigger injurious responses.”

This claim is not supported by a reference, but later in the article Nel et al refer to an earlier paper entitled “Ultrafine particles cross cellular membranes by nonphagocytic mechanisms in lungs and in cultured cells” by Marianne Geiser and colleagues. These two papers, Nel et al, and, Geiser et al, have been cited respectively 5000 times and 850 times according to PubMed.

As early as 2007, Shayla Banerji and Mark Hayes had already challenged this idea of transport of nanoparticles across membranes in an elegant experimental and theoretical study which was a direct response to the two papers cited above “Examination of Nonendocytotic Bulk Transport of Nanoparticles Across Phospholipid Membranes“:

In accordance with these health concerns, Nel et al. have described some phenomena that can only potentiate fear of the negative health risks associated with nanotechnology.

[…]

Non-endocytotic transmembrane transport of large macromolecules is a significant exception to what is presently known about cell membrane permeability. Most early studies show that lipid bilayers are essentially impenetrable by molecules larger than water under physiological conditions: transport of most molecules across cell membranes is specifically cell-mediated by endocytosis.34 Endocytosis, unlike proposed passive, non-endocytotic transport, is an active cell-mediated process by which a substance gains entry into a cell. Specifically, a cell’s plasma membrane continuously invaginates to form vesicles around materials that originated outside the membrane: as the invagination continuously folds inward, the cell membrane constituents simultaneously reorganize in such a way that the material being transported into the cell is completely enclosed in a lipid bilayer, forming an endosome.35,36

[…]

The results suggest that a diffusive process of transport is not likely.

Figure 8 is particularly telling (!).

Capture

The article by Shayla Banerji and Mark Hayes has been cited 44 times.