post-publication peer review


Some readers might wonder why I am going on about this, so let me tell you: this is a considerably more important story than Stripy Nanoparticles Revisited. If, as I am arguing, some of this science is shaky, then it is not only the way we evaluate scientists and spend public money which are put into question, but the foundation of ongoing clinical trials.

Back to basics: in the section of Mirkin’s group PhD dissertation (previous post) that respond to our critique of their work on Spherical Nucleic Acid / SmartFlare / StickyFlare, they wrote the following:

Additionally, since the commercialization and sale of the nanoflare platform under the trade name Smartflare (Millipore), dozens of researchers around the world have participated in successful sequence-specific gene detection.[80]

Reference [80] correspond to six (half a dozen) articles, 80a to 80f (see below for details and links). Out of these six, only two are actual research papers, and, for both, the SmartFlares are a very minor addition to the work. Out of these two, only one is completely independent of Mirkin/EMD Millipore (the other one comes from Northwestern).

80a) is not primary research; it is an advertorial produced by EMD Millipore.

80b) is not primary research: it is a 300 words congress abstract (no figure). A follow up paper by the same group is discussed here.

80c) is a review and it is a collaboration between Northwestern (Mirkin’s University) and EMD Millipore. CoI statement from the paper: “D. Weldon is the R&D Manager at EMD Millipore responsible for the production of SmartFlares. Patents related to therapeutically targeting Nodal in tumor cells have been awarded to E.A. Seftor, R.E.B. Seftor, and M.J.C. Hendrix.

80d) is a research paper. It does not show in any way that SmartFlares work. It assumes it does. The SmartFlare is a minor part of the article.

80e) is not primary research: it is an advertorial in a magazine funded by company advertising (including EMD Millipore in that very issue). The author is a journalist working for the magazine, not a practicing scientist.

80f) is a research paper. It does not show in any way that SmartFlares work. It assumes it does. SmartFlares are a very minor part of the article. The authors are from Northwestern, i.e. Mirkin’s University.


Is targeting your target?

Warren Chan’s group published in June a perspective in Nature Reviews Materials entitled “Analysis of nanoparticle delivery to tumours” (Wilhelm et al). A key finding of their analysis of the literature is the absence of increase in the (very small) amount of nanoparticles delivered to tumours in the past 10 years. In a welcome departure from the usually overly diplomatic and confused style that is the trademark of most scientific writing, Wilhelm et al write the following:

 “These advantages [of nanoparticles] have been dampened by the lack of translation to patient care, despite the large investment (more than $1 billion in North America in the past 10 years) and success in imaging and treating tumours in mouse models. As a result, nanomedicine has acquired a reputation of being “hype” that cannot deliver and has not transformed patient care as it promised 15 years ago”


“We must admit that our current approach is broken, and that is why we have not observed significant clinical translation of cancer nanomedicines. Many academic studies focused on the potential of nanoparticles for in vivo applications and showed that nanoparticles may be delivered to tumours by the EPR effect. However, publishing ‘proof of concept’ studies will only lead to curing mice and will unlikely translate to cancer care, irrespective of the number of nanoparticle design permutations used for cancer targeting studies.”

Recognising the magnitude of the challenge, Wilhelm et al propose a thirty year strategy for nanomedicine.

Not surprisingly the publication sparked a debate; see for example Derek Lowe’s blog “Nanoparticles Mix It Up With Reality” and the comments therein, and the article by Michael Torrice for Chemical and Engineering News “Does nanomedicine have a delivery problem?” which features a number of quotes by various nanomedicine players, some of whom contesting Wilhelm et al’s findings, or their relevance to the development of nanomedicine. The debate has also continued in the scientific literature with a comment by McNeil “Evaluation of nanomedicines: stick to the basics” and a response by Chan.

Another comment by Lammers et al has been published 10 days ago “Cancer Nanomedicine: Is targeting our target?”. The implicit answer of the authors is no, targeting is not our target and therefore the absence of progress noted by Wilhelm et al matters little. Lammers et al’s argument is first that the percentage of the injected dose reaching the tumour is not a good indicator of the potential of a therapy, and second, that nanomedicine has in fact had some successes even without targeting. To illustrate this latter point, their first example is Doxil, a liposomal formulation of the anti-cancer drug Doxurubicin.

It is rather unconvincing that Lammers et al would use Doxil as an indication of the success of nanomedicine given that it was developed in the 80s and 90s, i.e. one or two decades before the “nanomedicine” word had been coined and Clinton had announced the $500M National Nanotechnology Initiative (January 2000). A bibliography search for the word “nanomedicine” suggests that it started to be used in the year 2000, with this MIT Technology Review being one of the very first examples:

Nanomedicine Nears the Clinic

Minuscule “smart bombs” that find cancer cells, kill them with the help of lasers and report the kills. Sound crazy? Guess again. That treatment scenario may be less than a decade away.

by David Voss
January 1, 2000

Since this infamous MIT technology review, we have seen so many similar promises and so little translation that Chan’s review and the debate that it provoked are indeed an incredibly positive and much needed development.

There is another amusing thing about Lammers et al’s review. The title suggesting that targeting is not our target is further echoed in the conclusion as follows:

“Patients do not benefit from targeting as such, and a reported tumour accumulation of 0.7%ID does not mean that nanomedicines do not work. We have to think beyond targeting, and beyond numbers, and focus on carrier-dependent drugs, combination therapies, protocols for patient selection and ways to enable rapid and more efficient clinical translation.”

Yet targeting as such seems very much to have been the target of these authors as the (non-exhaustive) list of articles below illustrate.

  1. Blume, G.; Cevc, G.; Crommelin, M.; Bakkerwoudenberg, I.; Kluft, C.;Storm, G.,Specific targeting with poly(ethylene glycol)-modified liposomes – coupling of homing devices to the ends of the polymeric chains combines effective target binding with long circulation times. Biochimica Et Biophysica Acta 1993, 1149 (1), 180-184.
  2. Vingerhoeds, M. H.; Steerenberg, P. A.; Hendriks, J.; Dekker, L. C.; vanHoesel, Q.;Crommelin, D. J. A.; Storm, G., Immunoliposome-mediated targeting of doxorubicin to human ovarian carcinoma in vitro and in vivo. British Journal of Cancer 1996, 74 (7), 1023-1029.
    3. Storm, G.; Crommelin, D. J. A., Colloidal systems for tumor targeting. Hybridoma 1997, 16 (1), 119-125.
    4. Mastrobattista, E.; Koning, G. A.; Storm, G., Immunoliposomes for the targeted delivery of antitumor drugs. Advanced Drug Delivery Reviews 1999, 40 (1-2), 103-127.
    5. Mastrobattista, E.; Kapel, R. H. G.; Eggenhuisen, M. H.; Roholl, P. J. M.; Crommelin, D. J. A.; Hennink, W. E.; Storm, G., Lipid-coated polyplexes for targeted gene delivery to ovarian carcinoma cells. Cancer Gene Therapy 2001, 8 (6), 405-413.
    6. Mastrobattista, E.; Crommelin, D. J. A.; Wilschut, J.; Storm, G., Targeted liposomes for delivery of protein-based drugs into the cytoplasm of tumor cells. Journal of Liposome Research 2002, 12 (1-2), 57-65.
    7. Metselaar, J. M.; Bruin, P.; de Boer, L. W. T.; de Vringer, T.; Snel, C.; Oussoren, C.; Wauben, M. H. M.; Crommelin, D. J. A.; Storm, G.; Hennink, W. E., A novel family of L-amino acid-based biodegradable polymer-lipid conjugates for the development of long-circulating liposomes with effective drug-targeting capacity. Bioconjugate Chemistry 2003, 14 (6), 1156-1164.
    8. Metselaar, J. M.; Wauben, M. H. M.; Wagenaar-Hilbers, J. P. A.; Boerman, O. C.; Storm, G., Complete remission of experimental arthritis by joint targeting of glucocorticoids with long-circulating liposomes. Arthritis and Rheumatism 2003, 48 (7), 2059-2066.
    9. Schiffelers, R. M.; Koning, G. A.; ten Hagen, T. L. M.; Fens, M.; Schraa, A. J.; Janssen, A.; Kok, R. J.; Molema, G.; Storm, G., Anti-tumor efficacy of tumor vasculature-targeted liposomal doxorubicin. Journal of Controlled Release 2003, 91 (1-2), 115-122.
    10. Schmidt, J.; Metselaar, J. M.; Wauben, M. H. M.; Toyka, K. V.; Storm, G.; Gold, R., Drug targeting by long-circulating liposomal glucocorticosteroids increases therapeutic efficacy in a model of multiple sclerosis. Brain 2003, 126, 1895-1904.
    11. van Steenis, J. H.; van Maarseveen, E. M.; Verbaan, F. J.; Verrijk, R.; Crommelin, D. J. A.; Storm, G.; Hennink, W. E., Preparation and characterization of folate-targeted pEG-coated pDMAEMA-based polyplexes. Journal of Controlled Release 2003, 87 (1-3), 167-176.
    12. Mulder, W. J. M.; Strijkers, G. J.; Griffioen, A. W.; van Bloois, L.; Molema, G.; Storm, G.; Koning, G. A.; Nicolay, K., A liposomal system for contrast-enhanced magnetic resonance imaging of molecular targets. Bioconjugate Chemistry 2004, 15 (4), 799-806.
    13. Schiffelers, R. M.; Ansari, A.; Xu, J.; Zhou, Q.; Tang, Q. Q.; Storm, G.; Molema, G.; Lu, P. Y.; Scaria, P. V.; Woodle, M. C., Cancer siRNA therapy by tumor selective delivery with ligand-targeted sterically stabilized nanoparticle. Nucleic Acids Research 2004, 32 (19).
    14. Verbaan, F. J.; Oussoren, C.; Snel, C. J.; Crommelin, D. J. A.; Hennink, W. E.Storm, G., Steric stabilization of poly(2-(dimethylamino)ethyt methacrytate)-based polyplexes mediates prolonged circulation and tumor targeting in mice. Journal of Gene Medicine 2004, 6 (1), 64-75.
    15. Visser, C. C.; Stevanovic, S.; Voorwinden, L. H.; van Bloois, L.; Gaillard, P. J.; Danhof, M.; Crommelin, D. J. A.; de Boer, A. G., Targeting liposomes with protein drugs to the blood-brain barrier in vitro. European Journal of Pharmaceutical Sciences 2005, 25 (2-3), 299-305.
    16. Zhang, C. F.; Jugold, M.; Woenne, E. C.; Lammers, T.; Morgenstern, B.; Mueller, M. M.; Zentgraf, H.; Bock, M.; Eisenhut, M.; Semmler, W.; Kiessling, F., Specific targeting of tumor angiogenesis by RGD-conjugated ultrasmall superparamagnetic iron oxide particles using a clinical 1.5-T magnetic resonance scanner. Cancer Research 2007, 67 (4), 1555-1562.
    17. Dolman, M. E. M.; Fretz, M. M.; Segers, G. W.; Lacombe, M.; Prakash, J.; Storm, G.; Hennink, W. E.; Kok, R. J., Renal targeting of kinase inhibitors. International Journal of Pharmaceutics 2008, 364 (2), 249-257.
    18. Lammers, T.; Hennink, W. E.; Storm, G., Tumour-targeted nanomedicines: principles and practice. British Journal of Cancer 2008, 99 (3), 392-397.
    19. Lammers, T.; Subr, V.; Peschke, P.; Kuhnlein, P.; Hennink, W. E.; Ulbrich, K.; Kiessling, F.; Heilmann, M.; Debus, J.; Huber, P. E.; Storm, G., Image-guided and passively tumour-targeted polymeric nanomedicines for radiochemotherapy. British Journal of Cancer 2008, 99 (6), 900-910.
    20. Rijcken, C. J. F.; Schiffelers, R. M.; van Nostrum, C. F.; Hennink, W. E., Long circulating biodegradable polymeric micelles: Towards targeted drug delivery. Journal of Controlled Release 2008, 132 (3), E33-E35.
    21. Crommelin, D. J. A., Nanotechnological approaches for targeted drug delivery: hype or hope? New Biotechnology 2009, 25, S34-S34.
    22. Mulder, W. J. M.; Castermans, K.; van Beijnum, J. R.; Egbrink, M.; Chin, P. T. K.; Fayad, Z. A.; Lowik, C.; Kaijzel, E. L.; Que, I.; Storm, G.; Strijkers, G. J.; Griffioen, A. W.; Nicolay, K., Molecular imaging of tumor angiogenesis using alpha v beta 3-integrin targeted multimodal quantum dots. Angiogenesis 2009, 12 (1), 17-24.
    23. Talelli, M.; Rijcken, C. J. F.; Lammers, T.; Seevinck, P. R.; Storm, G.; van Nostrum, C. F.; Hennink, W. E., Superparamagnetic Iron Oxide Nanoparticles Encapsulated in Biodegradable Thermosensitive Polymeric Micelles: Toward a Targeted Nanomedicine Suitable for Image-Guided Drug Delivery. Langmuir 2009, 25 (4), 2060-2067.
    24. Dolman, M. E. M.; Harmsen, S.; Storm, G.; Hennink, W. E.; Kok, R. J., Drug targeting to the kidney: Advances in the active targeting of therapeutics to proximal tubular cells. Advanced Drug Delivery Reviews 2010, 62 (14), 1344-1357.
    25. Lammers, T.; Subr, V.; Ulbrich, K.; Hennink, W. E.; Storm, G.; Kiessling, F., Polymeric nanomedicines for image-guided drug delivery and tumor-targeted combination therapy. Nano Today 2010, 5 (3), 197-212.
    26. Lammers, T.; Subr, V.; Ulbrich, K.; Peschke, P.; Huber, P. E.; Hennink, W. E.; Storm, G.; Kiessling, F., Long-Circulating and Passively Tumor-Targeted Polymer-Drug Conjugates Improve the Efficacy and Reduce the Toxicity of Radiochemotherapy. Advanced Engineering Materials 2010, 12 (9), B413-B421.
    27. Oerlemans, C.; Bult, W.; Bos, M.; Storm, G.; Nijsen, J. F. W.; Hennink, W. E., Polymeric Micelles in Anticancer Therapy: Targeting, Imaging and Triggered Release. Pharmaceutical Research 2010, 27 (12), 2569-2589.
    28. Talelli, M.; Iman, M.; Rijcken, C. J. F.; van Nostrum, C. F.; Hennink, W. E., Targeted core-crosslinked polymeric micelles with controlled release of covalently entrapped doxorubicin. Journal of Controlled Release 2010, 148 (1), E121-E122.
    29. van Rooy, I.; Cakir-Tascioglu, S.; Couraud, P. O.; Romero, I. A.; Weksler, B.; Storm, G.; Hennink, W. E.; Schiffelers, R. M.; Mastrobattista, E., Identification of Peptide Ligands for Targeting to the Blood-Brain Barrier. Pharmaceutical Research 2010, 27 (4), 673-682.
    30. Talelli, M.; Hennink, W. E., Thermosensitive polymeric micelles for targeted drug delivery. Nanomedicine 2011, 6 (7), 1245-1255.
    31. Talelli, M.; Rijcken, C. J. F.; Oliveira, S.; van der Meel, R.; Henegouwen, P.; Lammers, T.; van Nostrum, C. F.; Storm, G.; Hennink, W. E., Nanobody – Shell functionalized thermosensitive core-crosslinked polymeric micelles for active drug targeting. Journal of Controlled Release 2011, 151 (2), 183-192.
    32. van Rooy, I.; Mastrobattista, E.; Storm, G.; Hennink, W. E.; Schiffelers, R. M., Comparison of five different targeting ligands to enhance accumulation of liposomes into the brain. Journal of Controlled Release 2011, 150 (1), 30-36.
    33. Crielaard, B. J.; Lammers, T.; Schiffelers, R. M.; Storm, G., Drug targeting systems for inflammatory disease: One for all, all for one. Journal of Controlled Release 2012, 161 (2), 225-234.
    34. Crielaard, B. J.; Rijcken, C. J. F.; Quan, L. D.; van der Wal, S.; Altintas, I.; van der Pot, M.; Kruijtzer, J. A. W.; Liskamp, R. M. J.; Schiffelers, R. M.; van Nostrum, C. F.; Hennink, W. E.; Wang, D.; Lammers, T.; Storm, G., Glucocorticoid-Loaded Core-Cross-Linked Polymeric Micelles with Tailorable Release Kinetics for Targeted Therapy of Rheumatoid Arthritis. Angewandte Chemie-International Edition 2012, 51 (29), 7254-7258.
    35. Dolman, M. E. M.; Harmsen, S.; Pieters, E. H. E.; Sparidans, R. W.; Lacombe, M.; Szokol, B.; Orfi, L.; Keri, G.; Storm, G.; Hennink, W. E.; Kok, R. J., Targeting of a platinum-bound sunitinib analog to renal proximal tubular cells. International Journal of Nanomedicine 2012, 7, 417-433.
    36. Joshi, M. D.; Unger, W. J.; Storm, G.; van Kooyk, Y.; Mastrobattista, E., Targeting tumor antigens to dendritic cells using particulate carriers. Journal of Controlled Release 2012, 161 (1), 25-37.
    37. Kunjachan, S.; Jayapaul, J.; Mertens, M. E.; Storm, G.; Kiessling, F.; Lammers, T., Theranostic Systems and Strategies for Monitoring Nanomedicine-Mediated Drug Targeting. Current Pharmaceutical Biotechnology 2012, 13 (4), 609-622.
    38. Lammers, T.; Kiessling, F.; Hennink, W. E.; Storm, G., Drug targeting to tumors: Principles, pitfalls and (pre-) clinical progress. Journal of Controlled Release 2012, 161 (2), 175-187.
    39. van der Meel, R.; Oliveira, S.; Altintas, I.; Haselberg, R.; van der Veeken, J.; Roovers, R. C.; Henegouwen, P.; Storm, G.; Hennink, W. E.; Schiffelers, R. M.; Kok, R. J., Tumor-targeted Nanobullets: Anti-EGFR nanobody-liposomes loaded with anti-IGF-1R kinase inhibitor for cancer treatment. Journal of Controlled Release 2012, 159 (2), 281-289.
    40. Talelli, M.; Oliveira, S.; Rijcken, C. J. F.; Pieters, E. H. E.; Etrych, T.; Ulbrich, K.; van Nostrum, R. C. F.; Storm, G.; Hennink, W. E.Lammers, T., Intrinsically active nanobody-modified polymeric micelles for tumor-targeted combination therapy. Biomaterials 2013, 34 (4), 1255-1260.
    41. van der Meel, R.; Vehmeijer, L. J. C.; Kok, R. J.; Storm, G.; van Gaal, E. V. B., Ligand-targeted particulate nanomedicines undergoing clinical evaluation: Current status. Advanced Drug Delivery Reviews 2013, 65 (10), 1284-1298.
    42. Heukers, R.; Altintas, I.; Raghoenath, S.; De Zan, E.; Pepermans, R.; Roovers, R. C.; Haselberg, R.; Hennink, W. E.; Schiffelers, R. M.; Kok, R. J.; Henegouwen, P., Targeting hepatocyte growth factor receptor (Met) positive tumor cells using internalizing nanobody-decorated albumin nanoparticles. Biomaterials 2014, 35 (1), 601-610.
    43. Kunjachan, S.; Pola, R.; Gremse, F.; Theek, B.; Ehling, J.; Moeckel, D.; Hermanns-Sachweh, B.; Pechar, M.; Ulbrich, K.; Hennink, W. E.; Storm, G.; Lederle, W.; Kiessling, F.; Lammers, T., Passive versus Active Tumor Targeting Using RGD- and NGR-Modified Polymeric Nanomedicines. Nano Letters 2014, 14 (2), 972-981.
    44. Novo, L.; Mastrobattista, E.; van Nostrum, C. F.; Hennink, W. E., Targeted Decationized Polyplexes for Cell Specific Gene Delivery. Bioconjugate Chemistry 2014, 25 (4), 802-812.
    45. Theek, B.; Gremse, F.; Kunjachan, S.; Fokong, S.; Pola, R.; Pechar, M.; Deckers, R.; Storm, G.; Ehling, J.; Kiessling, F.; Lammers, T., Characterizing EPR-mediated passive drug targeting using contrast-enhanced functional ultrasound imaging. Journal of Controlled Release 2014, 182, 83-89.
    46. Liu, J.; Jiang, X. L.; Hennink, W. E.; Zhuo, R. X., A modular approach toward multifunctional supramolecular nanopolyplexes for targeting gene delivery. Journal of Controlled Release 2015, 213, E123-E124.
    47. Novo, L.; Takeda, K. M.; Petteta, T.; Dakwar, G. R.; van den Dikkenberg, J. B.; Remaut, K.; Braeckmans, K.; van Nostrum, C. F.; Mastrobattista, E.; Hennink, W. E., Targeted Decationized Polyplexes for siRNA Delivery. Molecular Pharmaceutics 2015, 12 (1), 150-161.
    48. Shi, Y.; Lammers, T.; van Nostrum, C.; Hennink, W. E., Long circulating and stable polymeric micelles for targeted delivery of paclitaxel. Journal of Controlled Release 2015, 213, E127-E128.
    49. Shi, Y.; van der Meel, R.; Theek, B.; Blenke, E. O.; Pieters, E. H. E.; Fens, M.; Ehling, J.; Schiffelers, R. M.; Storm, G.; van Nostrum, C. F.; Lammers, T.; Hennink, W. E., Complete Regression of Xenograft Tumors upon Targeted Delivery of Paclitaxel via Pi-Pi Stacking Stabilized Polymeric Micelles. Acs Nano 2015, 9 (4), 3740-3752.
    50. Ashton, S.; Song, Y. H.; Nolan, J.; Cadogan, E.; Murray, J.; Odedra, R.; Foster, J.; Hall, P. A.; Low, S.; Taylor, P.; Ellston, R.; Polanska, U. M.; Wilson, J.; Howes, C.; Smith, A.; Goodwin, R. J. A.; Swales, J. G.; Strittmatter, N.; Takats, Z.; Nilsson, A.; Andren, P.; Trueman, D.; Walker, M.; Reimer, C. L.; Troiano, G.; Parsons, D.; De Witt, D.; Ashford, M.; Hrkach, J.; Zale, S.; Jewsbury, P. J.; Barry, S. T., Aurora kinase inhibitor nanoparticles target tumors with favorable therapeutic index in vivo. Science Translational Medicine 2016, 8 (325).


A welcome Nature Editorial

I reproduce below a comment I have left on this Nature editorial entitled “Go forth and replicate!“.

Nature Publishing Group encouragement of replications and discussions of their own published studies is a very welcome move. Seven years ago, I wrote a letter (accompanying a submission) to the Editor of Nature Materials. The last paragraph of that letter read: “The possibility of refuting existing data and theories is an important condition of progress of scientific knowledge. The high-impact publication of wrong results can have a real impact on research activities and funding priorities. There is no doubt that the series of papers revisited in this Report contribute to shape the current scientific landscape in this area of science and that their refutation will have a large impact.” [1]

The submission was “Stripy Nanoparticles Revisited” and it took three more years to publish it… in another journal; meanwhile Nature Materials continued to publish findings based on the original flawed paper [2]. The ensuing, finally public (after three years in the secret of peer review), discussions on blogs, news commentary and follow up articles were certainly very informative on the absolute necessity of changing the ways we do science to ensure a more rapid discussion of research results [3].

One of the lessons I draw from this adventure is that the traditional publishing system is, at best ill suited (e.g. Small: three years delay), or at worst (e.g. Nature Materials) completely reluctant at considering replications or challenges to their published findings. Therefore, I am now using PrePrints (e.g. to publish a letter PNAS won’t share with their readers [4]), PubPeer and journals such as ScienceOpen where publication happens immediately and peer review follows [5].

So whilst I warmly welcome this editorial, it will need a little more to convince me that it is not a complete waste of time to use the traditional channels to open discussions of published results.

[1] The rest of letter can be found at
[2] The article was eventually published in Small (DOI:10.1002/smll.201001465

2 comments on PubPeer

); timeline:

More hype than hope? #Biomaterials16

Congratulations to the organisers of the World Biomaterials Congress for having a high profile debate on the following proposition:

Nanotechnology is more hype than hope

I wish I could have attended as it is a topic I have given some thought… Thankfully, one of the attendees, Professor Laura Poole-Warren has done some live tweeting from the floor. So here is a storify.

SmartFlare Maths

SmartFlare are nanoparticle sensors which are sold by Merck and are supposed to detect mRNA inside live cells. The technology has been developed by Chad Mirkin. In his papers, the nanoparticles are called Nano-Flares or Spherical Nucleic Acids. I am saying “supposed to” because the central question of how those sensors could possibly reach the target that they are supposed to detect has not been addressed by Mirkin nor by Merck.

After evaluating the SmartFlare, we published recently our conclusions at ScienceOpen. We ran this research as an open science project, sharing our experimental results, analyses and conclusions in quasi real time using an open science notebook. All of the imaging data can also be consulted via our online Open Microscopy Environment repository.

Gal Haimovich, who reviewed our paper, first on his blog and then at ScienceOpen, suggested we should do some SmartFlare Maths (point 4 of his list of comments). This had been at the back of my mind for some time. There are various ways to look at this problem, but all those I have tried lead to the same conclusion that the protocols, results and conclusion published do not add up. Here is what I believe the simplest way to think of the SmartFlare Maths problem. As usual, comments and corrections would be very much appreciated.

Estimation of the number of SmartFlares per cell

SF-figure adapted from Giljohann

Adapted from Giljohan et al, Figure 1b

Estimate 1. SmartFlares are added to cells at a final concentration of 0.1 nM (following Merck’s protocol). For 400,000 cells and 20 μL (following Merck’s protocol), this would result in 150,000 SmartFlares per cell, assuming that all nanoparticles are uptaken.


Estimate 2. Giljohann et al  (Mirkin’s group) published a quantitative study of uptake of SmartFlares in various cell lines in 2007. From their Figure 1b, we can see that in the lower concentration range tested, there is a linear correlation between SmartFlare concentration in the medium and number of particles per cell. For cells exposed to a medium concentration of 0.1 nM, this would result in an uptake of 75 000 SmartFlares per cell. In the following discussion, we will use this lower estimate. With ~50 oligo probes per SmartFlare, this would give 3,750,000 oligo probes per cell.

Oligo probes per cell versus mRNA per cell

The copy number of any specific mRNA per cell depends on sequence, cell types, signalling events etc, but typically it ranges from a few copies to a few thousands of copies. Our estimate above indicates an excess of oligo probes of at least three orders of magnitude over the most abundant mRNA.

If just 0.1% of these probes would bind their target, it would block 3,750 mRNA resulting in silencing. However, Merck and Mirkin both report that there is no silencing effect in the conditions of these experiments. It follows that more than 99.9% of the SmartFlares do not bind their target mRNA.

Fluorescence background


Reproduced from Seferos et al, Figure 1.

Seferos et al (2007, Mirkin’s group) show that in the absence of release of the probe, fluorescence value of ~30% of the total value after release is observed (in ideal test-tube conditions, i.e. in the absence of nucleases). This is presumably due to a non-complete quenching of the fluorescence. For the SmartFlares to work, we would therefore have to detect a variation of less than 0.1% over a background of ~30%.


Lab Times: “Flare up over SmartFlares”

Stephen Buckingham interviewed me for Lab Times

On the face of it, Millipore’s SmartFlares are meant to be a tool cell biologists dream of – a way of measuring levels of specific RNA in real time in living cells. But does it really work? Raphaël Lévy and Gal Haimovich are in doubt.

Raphaël Lévy, Senior Lecturer in Biochemistry at the University of Liverpool, UK, was so unconvinced about SmartFlares that he decided to put the technique directly to the test (The Spherical Nucleic Acids mRNA Detection Paradox, Mason et al. ScienceOpen Research). As a result, Lévy has found himself at the centre of a row; not only over whether the technique actually does the job but as to whether it can actually work, at all – even in principle. Lab Times asked Lévy why he is in doubt that SmartFlares really work.

Lab Times:  What’s all the fuss about SmartFlares?

Read it all here (page 50-51).

I can’t resist also quoting this bit of pf the final paragraph…

In interview, Lévy is reasonable and measured in tone. But he is no stranger to controversy and can deliver fierce polemic with style.

If you have not yet, you should also check Leonid Schneider’s earlier and more complete investigation.

Nanoparticles & cell membranes: history of a (science) fiction?

One of the reason scientists, journalists and the general public are excited about nanoparticles is their supposed ability to cross biological barriers, including, the cell membrane. This could do wonders for drug delivery by bringing active molecules to the interior of the cell where they could interact with key components of the cell machinery to restore function or kill cancer cells. On the opposite side of the coin, if nanoparticles can do this, then there are enormous implications in terms of their potential toxicity and it is very urgent to investigate. But is it true? What is the evidence? How did this idea come into the scientific literature in the first place? I have been intrigued by this question for some time. It is the publication of an article about stripy nanoparticles magically crossing the cell membrane that led me to engage in what became the stripy nanoparticles controversy. It is this same vexing question that led me to question Merck/Mirkin claims about smartflare/nanoflare/stickyflare.

In the introduction of our article “The spherical nucleic acids mRNA detection paradox“, we describe the long history of the use of gold nanoparticles (“gold colloids”) in cell biology and conclude that

…, more than five decades of work has clearly established that nanoparticles enter cells by endocytotic mechanisms that result in their entrapment inside intracellular vesicles unless those nanoparticles are biological in nature and have acquired through evolution, advanced molecular tools which enable them to escape.

In the paragraph that followed, we were trying to make the point, in part using citation data of one of these 1950s pioneering articles, that this solid knowledge has been ignored in some of the thousands of recent articles on interactions of nanoparticles with membranes and cells that have appeared in the past 15 years. In his review of the first version of our article, Steve Royle criticises that latter paragraph (in his word, a “very minor” point):

I’m not a big fan of using number of Web of Science search results as an argument (Introduction). The number of papers on Gold Nanoparticles may be increasing since 2007, but then so are the number of papers on anything. It needs to be normalised to be meaningful. It’s also a shame that only 5 papers have cited Harford et al., but it’s an old paper, maybe people are citing reviews that cover this paper instead?

This is a fair point. While normalisation as well as more detailed and systematic searches might shed some light, it is rather difficult to quantify an absence of citation. Instead, I have tried to discover where the idea that nanoparticles can diffuse through membranes comes from. Here are my prime suspects (but I would be more than happy to update this post to better reflect the history of science and ideas so please leave comment, tweet, email), Andre Nel and colleagues, in Science, 3rd of February 2006, “Toxic Potential of Materials at the Nanolevel” :

“ Moreover, some nanoparticles readily travel throughout the body, deposit in target organs, penetrate cell membranes, lodge in mitochondria, and may trigger injurious responses.”

This claim is not supported by a reference, but later in the article Nel et al refer to an earlier paper entitled “Ultrafine particles cross cellular membranes by nonphagocytic mechanisms in lungs and in cultured cells” by Marianne Geiser and colleagues. These two papers, Nel et al, and, Geiser et al, have been cited respectively 5000 times and 850 times according to PubMed.

As early as 2007, Shayla Banerji and Mark Hayes had already challenged this idea of transport of nanoparticles across membranes in an elegant experimental and theoretical study which was a direct response to the two papers cited above “Examination of Nonendocytotic Bulk Transport of Nanoparticles Across Phospholipid Membranes“:

In accordance with these health concerns, Nel et al. have described some phenomena that can only potentiate fear of the negative health risks associated with nanotechnology.


Non-endocytotic transmembrane transport of large macromolecules is a significant exception to what is presently known about cell membrane permeability. Most early studies show that lipid bilayers are essentially impenetrable by molecules larger than water under physiological conditions: transport of most molecules across cell membranes is specifically cell-mediated by endocytosis.34 Endocytosis, unlike proposed passive, non-endocytotic transport, is an active cell-mediated process by which a substance gains entry into a cell. Specifically, a cell’s plasma membrane continuously invaginates to form vesicles around materials that originated outside the membrane: as the invagination continuously folds inward, the cell membrane constituents simultaneously reorganize in such a way that the material being transported into the cell is completely enclosed in a lipid bilayer, forming an endosome.35,36


The results suggest that a diffusive process of transport is not likely.

Figure 8 is particularly telling (!).


The article by Shayla Banerji and Mark Hayes has been cited 44 times.