Do planes fly and other difficult scientific questions

The Scientist magazine reported on the ACS meeting incident. Here is Chad Mirkin’s response to their questions:

Mirkin disagrees with Levy’s assessment of the endosome entrapment. “Levy’s narcissistic approach is akin to, ‘I bought an airplane, and I can’t make it fly. Therefore, planes don’t fly, despite the fact that I see them all above me,’” he tells The Scientist.

Mirkin stresses the number of studies in which the probes have been used successfully: “There is no controversy . . . There are over 40 papers reporting the successful use of such structures, involving over 100 different researchers, spanning three different continents,” he writes to The Scientist in an email. “I think the data and widespread use of such structures speak to their reliability and utility for measuring RNA content in live cells,” he adds.

After “dishonest Rafael [sic] Levy and his band of trolls“, “scientific terrorist” and “scientific zealot“, I suppose the “narcissistad hominem, could be considered more moderate?


Echo and Narcissus, John William Waterhouse, 1903, Walker Art Gallery, Liverpool. Narcissus, too busy contemplating his image, cannot see Echo let alone planes flying above him.

As The Scientist notes, I am hardly the only one who cannot make the SmartFlare plane fly. And the plane manufacturer has stopped selling its product and does not answer questions from journalists.

Scientific terrorist

At the 2018 American Chemical Society National Meeting in Boston, I asked a question to Chad Mirkin after his talk on Spherical Nucleic Acids. This is what I said:

In science, we need to share the bad news as well as the good news. In your introduction you mentioned four clinical trials. One of them has reported. It showed no efficacy and Purdue Pharma which was supposed to develop the drug decided not to pursue further. You also said that 1600 forms of NanoFlares were commercially available. This is not true anymore as the distributor has pulled the product because it does not work. Finally, I have a question: what is the percentage of nanoparticles that escape the endosome.

I had written my question and I asked exactly this although not in one block as he started answering before I had made all my points. He became very angry. The exchange lasted maybe 5 minutes. Towards the end he said that no one is reading my blog (who cares), that no one agrees with me, he called me a “scientific zealot” and a “scientific terrorist”. The packed room was shell shocked. We then moved swiftly to the next talk.

Two group leaders, one from North America and the other one from Europe, came to me afterwards.

Group leader 1:

Science is ever evolving and evidenced based. The evidence is gathered by first starting to ask questions. I witnessed an interaction between two scientists. One asks his questions gracefully and one responding in a manner unbecoming of a Linus Pauling Medalist. It took courage to stand in front of a packed room of scientists and peers to ask those questions that deserved an answer in a non-aggressive manner. It took even more courage to not become reactive when the respondent is aggressive and belittling. I certainly commended Raphael Levy for how he handled the aggressive response from Chad Mirkin. Even in disagreements, you can respond in a more professional manner. Not only is name calling not appropriate, revealing the outcomes of reviewers opinions from a confidential peer-review process is unprofessional and unethical.*

Lesson learned: Hold your self to a high standard and integrity.

Group leader 2:

Many conferences suffer from interesting discussions after a talk in such way that there are questions and there are answers. So far so good. Only in rare cases, a critical mind starts a discussion, or ask questions which imply some disagreement with the presented facts. Here I was surprised how a renowned expert like Chad Mirkin got in rage by such questions of Raphael Levy and how unprofessional his reaction was. It was not science any longer, it was a personal aggression, and this raises the question why Chad Mirkin acted like this? I do not think that this strategy helps to get more acceptance by the audience. I tribute to Raphael Levy afterwards, because I think science needs critical minds, and one should not be calm because of the fear to get attacked by a speaker. Science is full of statements how well everything works, and optimism is the fuel to keep research running. There is nothing wrong with this, but definitely one also need critical questions to make progress, and what we don’t need is unprofessional behavior and discreditation.

* Group leader 1 refers here to the outcome of the reviews of this article which you can read on ChemrXiv and which was (predictably) rejected by Nature Biomedical Engineering. During the incident Chad Mirkin used these reviews to attack me.

Update: some reactions on Twitter:

“re. your exchange at if being a critical thinker is a I think this is something we should all aspire to be. Good for you.” @wilkinglab

“Do you know Rapha’s blog? Not true that no one is reading it! It is the true gem and a rare truth island!” @zk_nano

“Wow, that’s shockingly uncool.” @sean_t_barry

“What an unprofessional guy.”  @SLapointeChem

“Calling a fellow researcher a “scientific terrorist” for raising concerns and asking a question (even if you disagree with them) is shocking. Sorry to hear that there wasn’t any real discussion instead, would’ve been interesting.” @bearore

“Surprised this isn’t getting more pub. One must wonder at what point does one’s ego/reputation become more important than the science, which ABSOLUTELY must include the bad with the good.” @Ben_Jimi440

“Keep fighting the good fight tenaciously, Raphael. Like the detectives in those old film noir shows… 🤜🏼🤛🏽”  @drheaddamage


How do I know if an article is good? an #ACSBoston tale

This week I attended the fall meeting of the American Chemical Society in Boston. A little meeting of about 14,000 attendees. I was speaking in a symposium with an impossibly long title but which turned out to be good fun and interesting; big thanks to the organisers: Kimberly Hamad-Schifferli, Clemens Burda and Wolfgang Parak.

IMG_0117I spent most of my time in that symposium but also went to a few other sessions which tackled questions surrounding the ways we do and communicate science. I learnt a bit more about the activities of the Center for Open Science and the platform they offer to researchers to organise, plan, record and share their work (and I was even offered a T-shirt). Probably the best lecture I heard – and certainly the most entertaining – was in a science communication session “The poisoner’s guide to communicating chemistry” by Deborah Blum (now I really need to read her book).

I joined a session with the promising title of “Scientific Integrity: Can We Rely on the Published Scientific Literature?“. Judith Currano (Head of the Chemistry Library, University of Pennsylvania) discussed how to help students evaluate the quality of scientific articles; I reproduce the abstract below (italics and bold mine):

This paper, by a chemistry librarian and a professor who edits an online journal, frames the challenges facing scientists at all levels as a result of the highly variable quality of the scientific literature resulting from the introduction of a deluge of new open-access online journals, many from previously unknown publishers with highly variable standards of peer review. The problems are so pervasive that even papers submitted to well-established, legitimate journals may include citations to questionable or even frankly plagiarized sources. The authors will suggest ways in which science librarians can work with students and researchers to increase their awareness of these new threats to the integrity of the scientific literature and to increase their ability to evaluate the reliability of journals and individual articles. Traditional rules of thumb for assessing the reliability of scientific publications (peer review, publication in a journal with an established Thomson-Reuters Impact Factor, credible publisher) are more challenging to apply given the highly variable quality of many of the new open access journals, the appearance of new publishers, and the introduction of new impact metrics, some of which are interesting and useful, but others of which are based on citation patterns found in poorly described data sets or nonselective databases of articles. The authors suggest that instruction of research students in Responsible Conduct of Research be extended to include ways to evaluate the reliability of scientific information.

Now the problem of (rapidly) evaluating the reliability of an article, especially for new researchers in a particular field is a serious and acute one, so I fully approve the author’s suggestions.

However the entire paper is based largely on a false premise: the idea that it is the “introduction of a deluge of new open-access online journals” which creates this reliability problem. This is hardly the case. The difficulty in identifying poor articles is not the deluge of open access journals nor is it predatory publishing. The growth in the volume of publications is not particularly related to open access and predatory publishing can be easily identified (with a little bit of common sense and a few pointers). The abstract (and to a lesser extent the talk) also conflates the evaluation of the reliability of a journal (an impossible task if you ask me) and the reliability of an article (an extremely onerous task if you ask me, but more on this later). Do I need to comment on the “rule of thumbs“?

I do teach third year undergraduate students on a similar topic. I ask them this same question: “how can you evaluate the validity of a scientific article?”. I write their answers on the white board; in whatever order, I get: the prestige of the University/Authors/Journal, the impact factor, the quality (?) of the references… I then cross it all. I show the Arsenate DNA paper published in Science, the STAP papers published in Nature. I try to convince them that no measure of prestige can help them evaluate the quality and reliability of a paper, that the only solution they have is to read the paper carefully and critically analyse the data. If necessary, discuss it with others. If necessary ask questions to the authors.

Of course, reading carefully takes time, but there is (currently) no alternative. There is absolutely no reason to think that a paper is reliable because it is in an high impact factor journal. The Scottish philosopher David Hume (1771-1776) wrote that “A wise man proportions his belief to the evidence”…and should “always reject the greater miracle.” Many articles in high impact journals resemble such miracles and eventually turn out to be irreproducible.

The second part of my own scientific presentation focused on our ongoing SmartFlare project. On the last slide, it featured the David Hume quote as well as an updated 21st version (see below).


With the PubPeer browser extension, you can immediately see, on the journal page (or anywhere else the article is cited) if there is an existing discussion at PubPeer

There is however something simple that we can do immediately to make it easier for every body to evaluate the reliability of individual articles: sharing our critiques (positive or negative) of articles we read. If we all commit to use PubPeer and start sharing at least one review per month, this will go a very long way towards generating open discussions around articles. It will obviously not alleviate the need to read the articles and the reviews critically, but it will crowd source the evaluation and this can be very powerful (it is the model of SJS, ScienceOpen, F1000).