Spherical Nucleic Acid

Do planes fly and other difficult scientific questions

The Scientist magazine reported on the ACS meeting incident. Here is Chad Mirkin’s response to their questions:

Mirkin disagrees with Levy’s assessment of the endosome entrapment. “Levy’s narcissistic approach is akin to, ‘I bought an airplane, and I can’t make it fly. Therefore, planes don’t fly, despite the fact that I see them all above me,’” he tells The Scientist.

Mirkin stresses the number of studies in which the probes have been used successfully: “There is no controversy . . . There are over 40 papers reporting the successful use of such structures, involving over 100 different researchers, spanning three different continents,” he writes to The Scientist in an email. “I think the data and widespread use of such structures speak to their reliability and utility for measuring RNA content in live cells,” he adds.

After “dishonest Rafael [sic] Levy and his band of trolls“, “scientific terrorist” and “scientific zealot“, I suppose the “narcissistad hominem, could be considered more moderate?

1920px-John_William_Waterhouse_-_Echo_and_Narcissus_-_Google_Art_Project

Echo and Narcissus, John William Waterhouse, 1903, Walker Art Gallery, Liverpool. Narcissus, too busy contemplating his image, cannot see Echo let alone planes flying above him.

As The Scientist notes, I am hardly the only one who cannot make the SmartFlare plane fly. And the plane manufacturer has stopped selling its product and does not answer questions from journalists.

Guest post: my experience with the SmartFlares, by James Elliott

CaptureThis is a guest post by James Elliott, manager of the Flow Cytometry Facility at the MRC Institute (LMS) in Hammersmith.

***

I thought it may be useful to add to the discussion about SmartFlares, their marketing and the difficulties in disseminating negative results by passing on my own experience.

We tested the system back in 2013. Sorting primary murine T cells and thymocytes on the basis of RNA expression was perhaps of most immediate interest, but of course there were countless potential applications.

The Merck Millipore rep advised us that the caveat we should be aware of in using SmartFlares was that the particles are taken up by endocytosis and that not all cells possess the machinery to allow this. Indeed, he mentioned data he had seen that only around 20% of T cells take up probes. This was puzzling as it suggested either a specific subset of endocytosis-competent cells or alternatively that uptake by T cells was broad but weak, such that only 20% of cells fell into a positive, above background gate. This in itself seemed a potentially interesting question.

To address the usefulness of SmartFlares in primary T cells (and some lymphocyte lines we had in culture) it was agreed with the rep that the sensible first step was buy positive (an ‘Uptake’ probe where fluorescence is always ‘on’ even in the absence of specific RNA) and negative (scrambled, ‘fluorescence off’) controls.

Everyone rightly comments on the extremely high price of the reagents and though we were given a discount, it remained an expensive look-see experiment.

It was useful that on the day we tried out the probes we were lucky enough to have someone with us from Merck who we like and trust to oversee what we were doing – he could vouch for the fact that we did the experiments correctly. We looked for probe uptake both on a flow cytometer and an Imagestream imaging cytometer.

Whilst we had expected lymphocytes to take up the probes poorly, in fact the big problem we had was that whilst all, or nearly all cells took up the probes, the signal from cells given the scrambled probe – notionally ‘always off’ was just as high and in most cases a little higher than that with the positive control ‘Uptake’ probe. Both showed a marked, log shift in fluorescence.

So – big problems! Why was the scrambled probe, which should have been dim or ‘fluorescence off’ giving us such a high signal? Indeed, if anything our negative control was brighter than the positive.

The rep consulted with the technical team, who were quick to point out that more meaningful comparison would have been between a scrambled and housekeeping probe (the Uptake probe merely being useful to show a qualitative result), yet this seemed to me to fudge the issue: first, surely the uptake and scrambled probes should be roughly comparable in number of molecules of fluorochrome attached or the uptake control would be of limited value – it would give a yes/no answer as to whether the cells would take up probe, but would give little clue as to efficiency. Second, the strategy of validating the system had been agreed with the rep. It was not great to then come back and say that actually this was not a good test after all. Third, and most importantly, a system in which the negative control (‘fluorescence off’!) gives a log shift in fluorescence is likely to be almost completely useless! The background would be far too high for all but the most abundant markers.

In addition to which it hardly inspired confidence that the company seemed to have validated the system very poorly – why else would they be giving a vague suggestion that maybe 20% of T cells take up probe, when in our careful (and observed) hands, they did so rather efficiently. Interestingly, in this respect I later read on a cytometry forum that, according to one US user, the company had been very up front from the beginning that primary lymphocytes don’t take up the probes. This was doubly untrue – lymphocytes do take up the probes and in the UK anyway, we were not told primary lymphocytes didn’t take up probes – the rep thought 20% of T cells did so, but was unsure about the data. Again, I was left with the impression of a poorly validated system sold by reps who were largely in the dark.

The most likely explanation for our results in follow up discussions with the company was that scrambled probe had degraded intracellularly and that this can happen in a cell type-specific way. This would mean that there would be a cell type-specific optimum time window where there was a satisfactory balance between cleavage by target RNAs and non-specific cleavage. Of course we had followed the instruction we were given at the time, but now it appeared these probably weren’t correct for our (hardly esoteric!) cells.

The suggestion was therefore that as many controls as possible would be wise.

Clearly this had become completely untenable as a system – we would have to buy hugely expensive probes and – if they worked at all, which we still didn’t know – would have to do a lot of work to establish not only the usual factors such as concentration, but also timing. And how narrow might the optimal time window be where specificity was apparent? An hour? Less? And background from non-specific signal from degrading probes would be likely to be (at least in the cells we were most interested in) a major problem for any RNA that wasn’t highly expressed.

We decided to cut our losses. I applaud those who can follow up and publish negative data that will be useful to the scientific community, but it seemed likely that for us this would end up far too expensive financially and in time and effort – quite possibly simply to show that the system might just about work, but not in any way that would be practically useful.

 

 

 

 

 

 

Scientific terrorist

At the 2018 American Chemical Society National Meeting in Boston, I asked a question to Chad Mirkin after his talk on Spherical Nucleic Acids. This is what I said:

In science, we need to share the bad news as well as the good news. In your introduction you mentioned four clinical trials. One of them has reported. It showed no efficacy and Purdue Pharma which was supposed to develop the drug decided not to pursue further. You also said that 1600 forms of NanoFlares were commercially available. This is not true anymore as the distributor has pulled the product because it does not work. Finally, I have a question: what is the percentage of nanoparticles that escape the endosome.

I had written my question and I asked exactly this although not in one block as he started answering before I had made all my points. He became very angry. The exchange lasted maybe 5 minutes. Towards the end he said that no one is reading my blog (who cares), that no one agrees with me, he called me a “scientific zealot” and a “scientific terrorist”. The packed room was shell shocked. We then moved swiftly to the next talk.

Two group leaders, one from North America and the other one from Europe, came to me afterwards.

Group leader 1:

Science is ever evolving and evidenced based. The evidence is gathered by first starting to ask questions. I witnessed an interaction between two scientists. One asks his questions gracefully and one responding in a manner unbecoming of a Linus Pauling Medalist. It took courage to stand in front of a packed room of scientists and peers to ask those questions that deserved an answer in a non-aggressive manner. It took even more courage to not become reactive when the respondent is aggressive and belittling. I certainly commended Raphael Levy for how he handled the aggressive response from Chad Mirkin. Even in disagreements, you can respond in a more professional manner. Not only is name calling not appropriate, revealing the outcomes of reviewers opinions from a confidential peer-review process is unprofessional and unethical.*

Lesson learned: Hold your self to a high standard and integrity.

Group leader 2:

Many conferences suffer from interesting discussions after a talk in such way that there are questions and there are answers. So far so good. Only in rare cases, a critical mind starts a discussion, or ask questions which imply some disagreement with the presented facts. Here I was surprised how a renowned expert like Chad Mirkin got in rage by such questions of Raphael Levy and how unprofessional his reaction was. It was not science any longer, it was a personal aggression, and this raises the question why Chad Mirkin acted like this? I do not think that this strategy helps to get more acceptance by the audience. I tribute to Raphael Levy afterwards, because I think science needs critical minds, and one should not be calm because of the fear to get attacked by a speaker. Science is full of statements how well everything works, and optimism is the fuel to keep research running. There is nothing wrong with this, but definitely one also need critical questions to make progress, and what we don’t need is unprofessional behavior and discreditation.

* Group leader 1 refers here to the outcome of the reviews of this article which you can read on ChemrXiv and which was (predictably) rejected by Nature Biomedical Engineering. During the incident Chad Mirkin used these reviews to attack me.

Update: some reactions on Twitter:

“re. your exchange at if being a critical thinker is a I think this is something we should all aspire to be. Good for you.” @wilkinglab

“Do you know Rapha’s blog? Not true that no one is reading it! It is the true gem and a rare truth island!” @zk_nano

“Wow, that’s shockingly uncool.” @sean_t_barry

“What an unprofessional guy.”  @SLapointeChem

“Calling a fellow researcher a “scientific terrorist” for raising concerns and asking a question (even if you disagree with them) is shocking. Sorry to hear that there wasn’t any real discussion instead, would’ve been interesting.” @bearore

“Surprised this isn’t getting more pub. One must wonder at what point does one’s ego/reputation become more important than the science, which ABSOLUTELY must include the bad with the good.” @Ben_Jimi440

“Keep fighting the good fight tenaciously, Raphael. Like the detectives in those old film noir shows… 🤜🏼🤛🏽”  @drheaddamage

 

References for my talk at Gold 2018

Update: the slides are available (here: gold2018-Levy2) and there is a recording of a few minutes here.

Slide 1:

Slide 2: Nanotech is bs Tweet.

Slide 3: Calling Bullshit.

Slide 4:  Dinosaur. (from here)

Slide 5: Electron microscopy of Hela cells after the ingestion of colloidal gold; C.G. Harford, A. Hamlin, and E. Parker; 1957

Slide 6: The entry and distribution of herpes virus and colloidal gold in Hela cells after contact in suspension; M. A. Epstein, K. Hummeler, and A. Berkaloff; 1963

Slide 7:

Slide 8: The spherical nucleic acid paradox; D. Mason, G. Carolan, M. Held, J. Comenge, S. Cowman, and R. Lévy; 2015

Slide 9: Excerpt from email (shared with permission).

Slide 10:  Evaluation of SmartFlare probe applicability for verification of RNAs in early equine conceptuses, equine dermal fibroblast cells and trophoblastic vesicles;  S. Budik, W. Tschulenk, S. Kummer, I. Walter, and C. Aurich; 2017

Slide 11: SmartFlares fail to reflect their target transcripts levels; M. Czarnek and J. Bereta; 2017

Slide 12: Calcium-Binding Proteins S100A8 and S100A9: Investigation of Their Immune Regulatory Effect in Myeloid Cells; J. Yang, J. Anholts, U. Kolbe, J.A. Stegehuis-Kamp, F.H.J. Claas and M. Eikmans

Slide 13: SmartFlare catalog.

Slide 14:

The great answer to people saying that #preprints are not peer-reviewed

That perfect title is courtesy of (see tweet below)

On Monday (25/06), we will publish a preprint about the spherical nucleic acid technology. Our paper was prompted by the publication in Nature Biomedical Engineering of “Abnormal scar identification with spherical-nucleic-acid technology” by Yeo et al.

The great answer is… review them! I issued a call to review our preprint before it comes out and I have now sent the article to a number of colleagues across the world. I am very much looking forward to their comments good or bad. The comments will be posted on PubPeer. If you have some time on your hands this Friday or over the weekend to look at the paper, drop me an email and I will also send you a preview copy.

Yeo et al corresponding authors were provided with a copy of our preprint two weeks ago but unfortunately they have not responded. I hope they will post comments on PubPeer. We are planning to subsequently submit a version (hopefully improved thanks to the comments) to Nature Biomedical Engineering. It is however sometime rather difficult to debate the scientific literature through the official channels of traditional journals so this route via preprint will accelerate this important discussion.

 

Mind-boggling plagiarism of this blog

In January 2015, someone went to the effort of creating a fake raphazlab blog as the stripy nanoparticles controversy was descending from a scientific debate into the gutters of online discussions.

Fast forward three years. The Spherical Nucleic Acids controversy is slowly heating up. Chad Mirkin continues to win prizes after prizes, but he is unseemly asked to comment on the failing SmartFlare technology (the commercial name of the Spherical Nucleic Acids) by Dalmeet Singh writing for Chemistry World.

Dalmeet writes:

But Chad Mirkin, a chemist at Northwestern University in the US, who developed the precursor to SmartFlares, nanoflares, pointed Chemistry World to more than 30 papers, which, he says, have successfully used the technology.

Chemistry World contacted a number of groups that have used SmartFlares. Hirendranath Banerjee, a molecular biologist at Elizabeth City State University in North Carolina, describes SmartFlares as a ‘very novel and effective technique’, noting that it has been helpful in evaluating gene expression experiments in his lab.2

Now comes the mind-boggling part.

The introduction of Hirendranath’s paper (reference 2 above) is largely plagiarized… from this very blog. From the very first SmartFlare post on this blog, entitled How smart are the SmartFlares?

Below, is an excerpt from my post with, in red, the sentences that reference 2 copied.

RNA molecules play crucial roles in cells such as coding, decoding, regulation, and expression of genes, yet they are much more difficult to study. SmartFlares are nanoparticle-based probes for the detection and imaging of RNA in live cells. Could they become the GFP of the RNA world?

 Many certainly believe this to be the case. SmartFlare ranked second in TheScientist top ten 2013 innovations, with one of the judges, Kevin Lustig, commenting “These new RNA detection probes can be used to visualize RNA expression in live cells at the single-cell level.”  The following year, SmartFlare won an R&D100 award. The technology comes from Chad Mirkin’s lab at Northwestern University. Chad Mirkin is the winner of numerous prestigious prizes and a science adviser to the President of the United States. The scientific articles introducing the SmartFlare concept (under the name of Nano-Flare) were published in the Journal of the American Chemical Society in 2007, ACS Nano in 2009, etc. In 2013, the SmartFlare technology was licensed to EMD Millipore. Here is one of their promotional video:

For a molecular sensor to work, it needs a detection mechanism. The principle of the SmartFlare is explained from 0:45. A capture oligonucleotide (i.e. DNA) is bound to the gold nanoparticles. A reporter strand is bound to the capture strand. The reporter strand carries a fluorophore but that fluorophore does not emit light because it is too close to the gold (the fluorescence is “quenched”). In the presence of the target RNA, the reporter strand is replaced by the target RNA and therefore released, quenching stops, and fluorescence is detected.

This is plagiarism, with, in addition, a clear intent to deceive: whilst the article entire point appears to be the celebration of the SmartFlare technology, e.g concluding sentence (Thus Smart Flare novel gold nanoparticles could revolutionize the field of differential gene expression studies and drug discovery), the 2015 blog post was already doubting the validity of the technology.
I wrote to the Editor of the Journal who said that they would evaluate the claims and take some form of action. I contacted Hirendra Banerjee who declined to provide a statement; instead he issued a legal threat against the publication of this post.
Meanwhile, on Twitter
Capture
Capture
Capture
Conclusion of this exchange?
Capture