post-publication peer review

Open peer review of (not so) controversial articles

Publishing articles that are critical of previously published work is notoriously difficult but the secrecy of peer review makes it hard to explain the kind of biases and tricks that one faces in this enterprise. Opening peer review, i.e. sharing reports and responses, would certainly help. Here is an interesting exemple related to an article (nicely discussed by Philip Moriarty in a previous post) which is not even critical of prior literature but touches on the stripy nanoparticles controversy. That was too much for Reviewer #1 (hyperlinks added by me; they point to relevant blog posts here or at PubPeer):

Reviewer #1 (Remarks to the Author):
This paper describes the scanning tunnelling microscopy imaging (STM) of a silver cluster (Ag374). To the best of my knowledge there is no report of such things to date. As such I think this paper should be published but in a specialised journal or a broad journal with reporting functions as Scientific Reports.

The significance of this paper as such is minimal. The STM does not add anything to what X-ray crystallography has shown so far also on the same cluster. In fact it requires strong support from calculation.

The STM itself has been widely published on nanoparticles by the group of Stellacci. The authors do reference a controversy there but do not comment on it an neither add to it.

The approach used is almost identical to the one described by such group in Ong et al ACS Nano (non cited), and the results achieved are similar to the ones described in the same paper and in Moglianetti et al. (not cited). Their minimal difference is that they achieved these results in liquid nitrogen and helium temperature, but low temperature results were described in Biscarini et al. (not cited).

Given the scant discussion in the paper (lacks any point) and the two major objections report, I suggest rejection.

The other, more supportive reports, and the response from the authors, can be downloaded from Nature Communications.

Probes, Patterns, and (nano)Particles

philipmoriarty

Philip Moriarty

This is a guest post by Philip Moriarty, Professor of Physics at the University of Nottingham (and blogger).

“We shape our tools, and thereafter our tools shape us.”

Marshall McLuhan (1911-1980)

My previous posts for Raphael’s blog have focussed on critiquing poor methodology and over-enthusiastic data interpretation when it comes to imaging the surface structure of functionalised nanoparticles. This time round, however, I’m in the much happier position of being able to highlight an example of good practice in resolving (sub-)molecular structure where the authors have carefully and systematically used scanning probe microscopy (SPM), alongside image recognition techniques, to determine the molecular termination of Ag nanoparticles.

For those unfamiliar with SPM, the concept underpinning the operation of the technique is relatively straight-forward. (The experimental implementation rather less so…) Unlike a conventional microscope, there are no lenses, no mirrors, indeed, no optics of any sort [1]. Instead, an atomically or molecularly sharp probe is scanned back and forth across a sample surface (which is preferably atomically flat), interacting with the atoms and molecules below. The probe-sample interaction can arise from the formation of a chemical bond between the atom terminating the probe and its counterpart on the sample surface, or an electrostatic or magnetic force, or dispersion (van der Waals) forces, or, as in scanning tunnelling microscopy (STM), the quantum mechanical tunnelling of electrons. Or, as is generally the case, a combination of a variety of those interactions. (And that’s certainly not an exhaustive list.)

Here’s an example of an STM in action, filmed in our lab at Nottingham for Brady Haran’s Sixty Symbols channel a few years back…

Scanning probe microscopy is my first love in research. The technique’s ability to image and manipulate matter at the single atom/molecule level (and now with individual chemical bond precision) is seen by many as representing the ‘genesis’ of nanoscience and nanotechnology back in the early eighties. But with all of that power to probe the nanoscopic, molecular, and quantum regimes come tremendous pitfalls. It is very easy to acquire artefact-ridden images that look convincing to a scientist with little or no SPM experience but that instead arise from a number of common failings in setting up the instrument, from noise sources, or from a hasty or poorly informed choice of imaging parameters. What’s worse is that even relatively seasoned SPM practitioners (including yours truly) can often be fooled. With SPM, it can look like a duck, waddle like a duck, and quack like a duck. But it can too often be a goose…

That’s why I was delighted when Raphael forwarded me a link to “Real-space imaging with pattern recognition of a ligand-protected Ag374 nanocluster at sub-molecular resolution”, a paper published a few months ago by Qin Zhou and colleagues at Xiamen University (China), the Chinese Academy of Science, Dalian (China), the University of Jyväskylä (Finland), and the Southern University of Science and Technology, Guandong (China). The authors have convincingly imaged the structure of the layer of thiol molecules (specifically, tert-butyl benzene thiol) terminating 5 nm diameter silver nanoparticles.

What distinguishes this work from the stripy nanoparticle oeuvre that has been discussed and dissected at length here at Raphael’s blog (and elsewhere) is the degree of care taken by the authors and, importantly, their focus on image reproducibility. Instead of using offline zooms to “post hoc” select individual particles for analysis (a significant issue with the ‘stripy’ nanoparticle work), Zhou et al. have zoomed in on individual particles in real time and have made certain that the features they see are stable and reproducible from image to image. The images below are taken from the supplementary information for their paper and shows the same nanoparticle imaged four times over, with negligible changes in the sub-particle structure from image to image.

This is SPM 101

This is SPM 101. Actually, it’s Experimental Science 101. If features are not repeatable — or, worse, disappear when a number of consecutive images/spectra are averaged – then we should not make inflated claims (or, indeed, any claims at all) on the basis of a single measurement. Moreover, the data are free of the type of feedback artefacts that plagued the ‘classic’ stripy nanoparticle images and Zhou et al. have worked hard to ensure that the influence of the tip was kept to a minimum.

Given the complexity of the tip-sample interactions, however, I don’t quite share the authors’ confidence in the Tersoff-Hamann approach they use for STM image simulation [2]. I’m also not entirely convinced by their comparison with images of isolated molecular adsorption on single crystal (i.e. planar) gold surfaces because of exactly the convolution effects they point towards elsewhere in their paper. But these are relatively minor points. The imaging and associated analysis are carried out to a very high standard, and their (sub)molecular resolution images are compelling.

As Zhou et al. point out in their paper, STM (or atomic force microscopy) of nanoparticles, as compared to imaging a single crystal metal, semiconductor, or insulator surface, is not at all easy due to the challenging non-planar topography. A number of years back we worked with Marie-Paule Pileni’s group on dynamic force microscopy imaging (and force-distance analysis) of dodecanethiol-passivated Au nanoparticles. We found somewhat similar image instabilities as those observed by Zhou et al…

A-C above are STM data

A-C above are STM data, while D-F are constant height atomic force microscope images [3], of thiol-passivated nanoparticles (synthesised by Nicolas Goubet of Pileni’s group) and acquired at 78 K. (Zhou et al. similarly acquired data at 77K but they also went down to liquid helium temperatures). Note that while we could acquire sub-nanoparticle resolution in D-F (which is a sequence of images where the tip height is systematically lowered), the images lacked the impressive reproducibility achieved by Zhou et al. In fact, we found that even though we were ostensibly in scanning tunnelling microscopy mode for images such as those shown in A-C (and thus, supposedly, not in direct contact with the nanoparticle), the tip was actually penetrating into the terminating molecular layer, as revealed by force-distance spectroscopy in atomic force microscopy mode.

The other exciting aspect of Zhou et al.’s paper is that they use pattern recognition to ‘cross-correlate’ experimental and simulated data. There’s increasingly an exciting overlap between computer science and scanning probe microscopy in the area of image classification/recognition and Zhou and co-workers have helped nudge nanoscience a little more in this direction. Here at Nottingham we’re particularly keen on the machine learning/AI-scanning probe interface, as discussed in a recent Computerphile video…

Given the number of posts over the years at Raphael’s blog regarding a lack of rigour in scanning probe work, I am pleased, and very grateful, to have been invited to write this post to redress the balance just a little. SPM, when applied correctly, is an exceptionally powerful technique. It’s a cornerstone of nanoscience, and the only tool we have that allows both real space imaging and controlled modification right down to the single chemical bond limit. But every tool has its limitations. And the tool shouldn’t be held responsible if it’s misapplied…

[1] Unless we’re talking about scanning near field optical microscopy (SNOM). That’s a whole new universe of experimental pain…

[2] This is the “zeroth” order approach to simulating STM images from a calculated density of states. It’s a good starting point (and for complicated systems like a thiol-terminated Ag374 particle probably also the end point due to computational resource limitations) but it is certainly a major approximation.

[3] Technically, dynamic force microscopy using a qPlus sensor. See this Sixty Symbols video for more information about this technique.

 

Scientific terrorist

At the 2018 American Chemical Society National Meeting in Boston, I asked a question to Chad Mirkin after his talk on Spherical Nucleic Acids. This is what I said:

In science, we need to share the bad news as well as the good news. In your introduction you mentioned four clinical trials. One of them has reported. It showed no efficacy and Purdue Pharma which was supposed to develop the drug decided not to pursue further. You also said that 1600 forms of NanoFlares were commercially available. This is not true anymore as the distributor has pulled the product because it does not work. Finally, I have a question: what is the percentage of nanoparticles that escape the endosome.

I had written my question and I asked exactly this although not in one block as he started answering before I had made all my points. He became very angry. The exchange lasted maybe 5 minutes. Towards the end he said that no one is reading my blog (who cares), that no one agrees with me, he called me a “scientific zealot” and a “scientific terrorist”. The packed room was shell shocked. We then moved swiftly to the next talk.

Two group leaders, one from North America and the other one from Europe, came to me afterwards.

Group leader 1:

Science is ever evolving and evidenced based. The evidence is gathered by first starting to ask questions. I witnessed an interaction between two scientists. One asks his questions gracefully and one responding in a manner unbecoming of a Linus Pauling Medalist. It took courage to stand in front of a packed room of scientists and peers to ask those questions that deserved an answer in a non-aggressive manner. It took even more courage to not become reactive when the respondent is aggressive and belittling. I certainly commended Raphael Levy for how he handled the aggressive response from Chad Mirkin. Even in disagreements, you can respond in a more professional manner. Not only is name calling not appropriate, revealing the outcomes of reviewers opinions from a confidential peer-review process is unprofessional and unethical.*

Lesson learned: Hold your self to a high standard and integrity.

Group leader 2:

Many conferences suffer from interesting discussions after a talk in such way that there are questions and there are answers. So far so good. Only in rare cases, a critical mind starts a discussion, or ask questions which imply some disagreement with the presented facts. Here I was surprised how a renowned expert like Chad Mirkin got in rage by such questions of Raphael Levy and how unprofessional his reaction was. It was not science any longer, it was a personal aggression, and this raises the question why Chad Mirkin acted like this? I do not think that this strategy helps to get more acceptance by the audience. I tribute to Raphael Levy afterwards, because I think science needs critical minds, and one should not be calm because of the fear to get attacked by a speaker. Science is full of statements how well everything works, and optimism is the fuel to keep research running. There is nothing wrong with this, but definitely one also need critical questions to make progress, and what we don’t need is unprofessional behavior and discreditation.

* Group leader 1 refers here to the outcome of the reviews of this article which you can read on ChemrXiv and which was (predictably) rejected by Nature Biomedical Engineering. During the incident Chad Mirkin used these reviews to attack me.

Update: some reactions on Twitter:

“re. your exchange at if being a critical thinker is a I think this is something we should all aspire to be. Good for you.” @wilkinglab

“Do you know Rapha’s blog? Not true that no one is reading it! It is the true gem and a rare truth island!” @zk_nano

“Wow, that’s shockingly uncool.” @sean_t_barry

“What an unprofessional guy.”  @SLapointeChem

“Calling a fellow researcher a “scientific terrorist” for raising concerns and asking a question (even if you disagree with them) is shocking. Sorry to hear that there wasn’t any real discussion instead, would’ve been interesting.” @bearore

“Surprised this isn’t getting more pub. One must wonder at what point does one’s ego/reputation become more important than the science, which ABSOLUTELY must include the bad with the good.” @Ben_Jimi440

“Keep fighting the good fight tenaciously, Raphael. Like the detectives in those old film noir shows… 🤜🏼🤛🏽”  @drheaddamage

 

The conference dinner chatter way of (not) correcting the scientific record

One of the common responses of senior colleagues to my attempts to correct the scientific record goes somewhat like this:

You are giving X [leading figure in the field] too much credit anyway. We all know that there are problems with their papers. We discussed it at the latest conference with Y and Z. We just ignore this stuff and move along. Though of course X is my friend etc.

This approach is unfair, elitist and contributes to the degradation of the scientific record.

First, it is very fundamentally unfair to the many scientists who are not present at these dinner table chatters and who may believe that the accumulation of grants, prizes, and high profile papers somewhat correlate with good science. That group of scientists will include pretty much all young scientists as well as most scientists from less advantadged countries who cannot get so easily to these conferences where the truth about scientific achievements is discussed between drinks at the end of a play-acting day (for inquisitive questions at the end of talks are of course also not the right way to act).

Second, it devalues fundamentally the role of the scientific record. We are basically accepting that it does not matter whether what gets published is right or wrong.  Here, I’ll insert an anecdote. I reviewed, a couple of years ago, a high profile review authored by a senior colleague in the field of nanoparticles. One of the figures highlighted a paper which I knew to be fundamentally wrong. In my review, I pointed that fact. The senior colleague did not dispute that the paper was flawed but he opted for keeping the figure, not discussing the fact the paper was wrong. His (post-modern) argument was that the  “concept” was important.

Dinner chatter is fine. But please also share your criticisms, e.g. via PubPeer.

(and by the way, if you could comment on our preprint on Spherical Nucleic Acids, that would be much appreciated)

 

The great answer to people saying that #preprints are not peer-reviewed

That perfect title is courtesy of (see tweet below)

On Monday (25/06), we will publish a preprint about the spherical nucleic acid technology. Our paper was prompted by the publication in Nature Biomedical Engineering of “Abnormal scar identification with spherical-nucleic-acid technology” by Yeo et al.

The great answer is… review them! I issued a call to review our preprint before it comes out and I have now sent the article to a number of colleagues across the world. I am very much looking forward to their comments good or bad. The comments will be posted on PubPeer. If you have some time on your hands this Friday or over the weekend to look at the paper, drop me an email and I will also send you a preview copy.

Yeo et al corresponding authors were provided with a copy of our preprint two weeks ago but unfortunately they have not responded. I hope they will post comments on PubPeer. We are planning to subsequently submit a version (hopefully improved thanks to the comments) to Nature Biomedical Engineering. It is however sometime rather difficult to debate the scientific literature through the official channels of traditional journals so this route via preprint will accelerate this important discussion.

 

Mind-boggling plagiarism of this blog

In January 2015, someone went to the effort of creating a fake raphazlab blog as the stripy nanoparticles controversy was descending from a scientific debate into the gutters of online discussions.

Fast forward three years. The Spherical Nucleic Acids controversy is slowly heating up. Chad Mirkin continues to win prizes after prizes, but he is unseemly asked to comment on the failing SmartFlare technology (the commercial name of the Spherical Nucleic Acids) by Dalmeet Singh writing for Chemistry World.

Dalmeet writes:

But Chad Mirkin, a chemist at Northwestern University in the US, who developed the precursor to SmartFlares, nanoflares, pointed Chemistry World to more than 30 papers, which, he says, have successfully used the technology.

Chemistry World contacted a number of groups that have used SmartFlares. Hirendranath Banerjee, a molecular biologist at Elizabeth City State University in North Carolina, describes SmartFlares as a ‘very novel and effective technique’, noting that it has been helpful in evaluating gene expression experiments in his lab.2

Now comes the mind-boggling part.

The introduction of Hirendranath’s paper (reference 2 above) is largely plagiarized… from this very blog. From the very first SmartFlare post on this blog, entitled How smart are the SmartFlares?

Below, is an excerpt from my post with, in red, the sentences that reference 2 copied.

RNA molecules play crucial roles in cells such as coding, decoding, regulation, and expression of genes, yet they are much more difficult to study. SmartFlares are nanoparticle-based probes for the detection and imaging of RNA in live cells. Could they become the GFP of the RNA world?

 Many certainly believe this to be the case. SmartFlare ranked second in TheScientist top ten 2013 innovations, with one of the judges, Kevin Lustig, commenting “These new RNA detection probes can be used to visualize RNA expression in live cells at the single-cell level.”  The following year, SmartFlare won an R&D100 award. The technology comes from Chad Mirkin’s lab at Northwestern University. Chad Mirkin is the winner of numerous prestigious prizes and a science adviser to the President of the United States. The scientific articles introducing the SmartFlare concept (under the name of Nano-Flare) were published in the Journal of the American Chemical Society in 2007, ACS Nano in 2009, etc. In 2013, the SmartFlare technology was licensed to EMD Millipore. Here is one of their promotional video:

For a molecular sensor to work, it needs a detection mechanism. The principle of the SmartFlare is explained from 0:45. A capture oligonucleotide (i.e. DNA) is bound to the gold nanoparticles. A reporter strand is bound to the capture strand. The reporter strand carries a fluorophore but that fluorophore does not emit light because it is too close to the gold (the fluorescence is “quenched”). In the presence of the target RNA, the reporter strand is replaced by the target RNA and therefore released, quenching stops, and fluorescence is detected.

This is plagiarism, with, in addition, a clear intent to deceive: whilst the article entire point appears to be the celebration of the SmartFlare technology, e.g concluding sentence (Thus Smart Flare novel gold nanoparticles could revolutionize the field of differential gene expression studies and drug discovery), the 2015 blog post was already doubting the validity of the technology.
I wrote to the Editor of the Journal who said that they would evaluate the claims and take some form of action. I contacted Hirendra Banerjee who declined to provide a statement; instead he issued a legal threat against the publication of this post.
Meanwhile, on Twitter
Capture
Capture
Capture
Conclusion of this exchange?
Capture

Three little (nano) controversies and their morals

This post is a translation of an article originally published in French in Médecine/Sciences. The Editorial of the same issue (also in French) by Pierre Corvol is entitled Scientific integrity: the need for a systemic approach (open access).  You can download a pdf of my article, or, read at the publisher’s website (subscription). For citation, please refer to the original article as follows:

Trois petites (nano) controverses et leurs morales; Raphaël Lévy; Med Sci (Paris), 33 8-9 (2017) 797-800; Publié en ligne : 18 septembre 2017; DOI: https://doi.org/10.1051/medsci/20173308027

« Selon que vous serez puissant ou misérable, les jugements de cour vous rendront blanc ou noir » [1] [Depending on your social height, The law will see your crime as black—or else as white.] Thus concludes the Fable, by Jean de La Fontaine, The Animals sick of the plague : the donkey, guilty of the theft of a few blades of grass, is condemned to death, whilst the Lion and other powerful animals guilty of much more serious crimes are treated to praise and flattery. It is tempting and comforting to think that scientific judgments are of an altogether different nature. Seen in this light, science would reside outside of power struggles and the few mishaps (mistakes, frauds, conflicts of interest) would be rapidly corrected since the reality of the material world would quickly come back to us through experimental results if we were to try to ignore it for too long. The truth is however very different. A large fraction of published scientific results cannot be reproduced. It is not a few mishaps but structural problems which affect the foundations of the scientific enterprise [2, 3]. Peer evaluations seems to encourage the publication of extraordinary stories in high impact factor journals rather than careful and rigorous experimental studies. Contradictory or “negative” data are rarely published: scientific journals are not really interested, and us, scientists, are not particularly motivated by publicly stating our doubts on the work of colleagues who could be in charge of evaluating our next article or grant application. It is particularly urgent to repair our knowledge production system because science is at the center of numerous challenges critical for the future of human beings and the planet. The (real) problems of reproducibility have already been harnessed by lobbies to attack the credibility of scientists [4]. After the election for president of the largest scientific and military power of the world of a man who denies climate change, is very positive about the use of the atomic bomb, and, more broadly wages an open war against science and truth [5], we have a paramount need for science to be open, robust, capable of defending its independence, integrity and universal values. This seems a distant prospect.

The near absence of critical discussion in the scientific literature in many areas of science could make us forget that controversies are an essential aspect of the quest for knowledge, allowing to identify weak points of experiments and theories, thus enabling to consolidate or invalidate them [6]. They are consubstantial to the scientific practice [7]. The analysis of controversies is also a tool to “symmetrically map” the actors to better understand the roles of individuals and social processes [8]. In this piece, I describe three recent controversies in my area of research: gold nanoparticles applied to biology and medicine. This is no “symmetric map”: I am not a neutral observer but a scientist active, to various degrees, in each of those. I am trying nevertheless to draw some lessons and suggestions to improve the ways we work as scientists.

Stripy Nanoparticles

In 2004, Francesco Stellacci’s group at the Massachusetts Institute of Technology (MIT) published in the prestigious journal Nature Materials an article describing gold nanoparticles covered by a mixture of two molecules that self-assemble to form stripes that are observed by scanning probe microscopy [9]. This article and the numerous other ones that will follow in the same journal and in others just as prestigious such as the Journal of the American Chemical Society [10], Science [11] and Proceedings of the National Academy of Sciences (PNAS) [12], suggest that, thanks to their stripes, these nanoparticles have unique properties in terms of wetting, self-organization, interaction with proteins, penetration in cells, with lots of potential applications for biomolecular sensing, or even drug delivery. These articles certainly contributes to the progress of their authors’ careers, but the stripes are an experimental artefact well known by users of scanning probe microscopy. How to explain then that more than 20 “stripy” articles were published between 2004 and 2012? It is obvious that specialists (and even enlightened amateurs) had identified the problem as early as 2004. Yet, the articles and reviews of that period show no sign of it. One now knows that Predrag Djuranovic has been the first to engage into a scientific investigation aiming at testing, and eventually, contesting, the evidence for the the existence of the stripes. In 2005, this rigorous and brave scientist was a student in Francesco Stellacci’s lab. His experimental results and numerical simulation showing how the stripes originate from a poorly adjusted feedback control system were unambiguous but MIT ensured that these results would remain secret [13]. In 2007, I submitted a technical comment responding to the Science article. This first attempt, limited in its scope to the Science article itself, was unsuccessful: Science did ask Francesco Stellacci to respond but then decided not to publish the exchange of views [14]. In 2008, a new article from the MIT group, again in Nature Materials, report that, thanks to their stripes, these nanoparticles can cross the cell membrane and directly access the cytosol [15]. This is accompanied by a commentary entitled “Particles slip cell security” [16]. After discussions with several of my students, we decide to propose a more exhaustive answer. A few months later, the article “Stripy Nanoparticles Revisited” is ready. It includes a new analysis of the stripy images concluding that the stripes are a scanning artefact as well as a critical discussion of the physicochemical and biological properties which, together with experimental results, contradict the claim of direct access to the interior of cells. The article is first submitted to Nature Materials (rejected), then NanoLetters (rejected), and, finally, Small… where it is published after an editorial process that lasted three years [17-19]. The publication of our article, in November 2012, does not end the controversy. Instead, it expands in the scientific literature (a little) and it also takes new forms (in particular on my blog and others [20-23]). Problems with the reuse of images in different publications emerge and eventually lead to two corrections ([12] and [15]). After a number of requests, Philip Moriarty and Julian Stirling (School of Physics and Astronomy, university of Nottingham, UK) are given access to the original data of the 2004 article. They demonstrate, among other things, that the stripes are present in the entire image, i.e. even between the gold nanoparticles [24], a conclusion still rejected by Francesco Stellacci [25].

Homeopathic nanoparticles

The laboratory of Molly Stevens at Imperial College is one of the most prestigious in the field of biomaterials. In 2012, two articles from the group relate the particularly interesting properties of nanoparticles for diagnostic applications. The first one, published in Nature Materials, reports a phenomenon which is entirely extraordinary in which the signal detected increases when the concentration of molecules to detect decreases (“inverse sensitivity”) [26]. Even more incredible, this phenomenon extends to the point where there is less than a molecule of enzyme, on average, in the volume under study. The second article published in Nature Nanotechnology goes further : no need for instruments, the detection of concentration of analytes in the same range is achieved thanks to a colour change visible with the naked eye [27]. Detailed critiques of these articles are available on the website PubPeer [28, 29] as well as in a preprint authored by Boris Barbour; the objections are both simple and profound but the authors have chosen not to respond. One can note that the Avogadro number includes lots of zeros (630 000 000 000 000 000 000 000) and that the detection of a macroscopic change of property due to the presence of a single molecule is therefore an achievement that requires extremely solid proofs. One of the posts on PubPeer indicates that someone contacted the Editor of Nature Nanotechnology in January 2013, but, four years later, no doubts are expressed on the journal website, in the traditional scientific literature nor in the newspapers that had covered this story (e.g. Le Monde and the Daily Mail) when it was initially published [30, 31].

Spherical Nucleic Acids

The laboratory of Chad Mirkin at Northwestern University (USA) is one of the most prestigious in the field of nanosciences applied to biology and medicine. One major theme of their research are the Spherical Nucleic Acids (SNAs), a term introduced by Mirkin to describe gold nanoparticles functionalised with DNA (or RNA) strands. These SNAs are supposed to have properties very different from linear DNA [32]. In particular, they can access the cytosol of live cells, where they could detect and regulate, the presence and quantity of mRNAs. One could ask why this solution did not appear during evolution : to access the cell machinery, viruses and bacteria would have only needed to package themselves within their genetic materials. The first articles (in Science [33], the Journal of the Americal Chemical Society [34], NanoLetters [35], ACS Nano [36]) proposing this surprising theory do no mention the mechanism of the SNAs into cells whatsover. The following one, e.g. [37], propose that the particles enter by endocytosis, but do not explain the mechanism by which the SNAs would escape endosomes. After several dozens of articles on this topic, the proportion of particles reaching the cytosol is still to be measured and reported (in spite of the fact that gold nanoparticles have been used since the 1950s to study intracellular trafficking; such a study would not be difficult). One article from the Mirkin group suggests that SNAs are degraded in the endosomes and that a “small unquantifiable portion escapes […]” [38]. Nevertheless, the particles are now commercially available under the name SmartFlares (Merck Millipore) to detect RNA inside cells. We have studied the entry of nanoparticles in cells and their ability to detect RNA. Given our difficult experience with the publication of Stripy Nanoparticle Revisited, we decided to adopt a different strategy. The project has been open and we have shared our results in quasi real time on our blog. In contradiction with the descriptions made by Mirkin and by Merck Millipore, we have observed that the SmartFlares were degraded in endosomes and were not able to detect mRNA.  Mirroring the tale of Predrag Djuranovic and the stripy nanoparticles, we were not the first to have doubts about the technology: Luke Armstrong, who had been in charge of developing the SmartFlares at Merck Millipore in California (before leaving the company) had reached the same conclusion [39]. To ensure speedy publication and transparency, we published our article on the (not so prestigious) ScienceOpen platform where peer review occurs after publication [40]. We invited comments by Mirkin to no avail. Another article by the same group in PNAS describe a new version of the SmartFlares [41]. Our analysis of the raw data (obtained after multiple insistent requests) show that the signal comes from endosomes. Our letter submitted to PNAS has been rejected by the editorial board because it “[did] not contribute significantly to the discussion of this paper” [42].

 Morals

Access to raw data is essential and guaranteed by clear rules adopted by Universities, scientific journals and funding agencies. It is therefore generally possible to access data with some efforts. It is obviously preferable to publish data at the same time as the articles. This is already the norm for some categories of results and it should become generalised. Researchers should also adopt the Manifesto for reproducible research [43]. The tools are in place to improve the practice of science.

Evaluations of science and scientists must imperatively be based on a critical analysis of their work and the robustness or their results, not on the prestige of the institutions or journals. This requires a change of mind and a clear commitment from researchers who are in positions of power, i.e. everyone who features on promotion or recruitment committees. To say that an article is good because it has been published in a prestigious journal is a moral and logical error which needs to be challenged.

Institutions and scientific journals are not motivated by the quest for scientific truth. The decisisons taken by MIT (keeping Predrag Djuranovic’s findings secret), by Nature Materials (not publishing the exchange with Francesco Stellacci [14]), and by PNAS (not publishing [42]) have directly impacted progress of knowledge. These institutions have commendable principles but, in practice, they aim first at defending their reputation and finances [44]. The latter objective only partially aligns with scientific progress which requires rapid and open discussion of results and conclusions. The Worldwide Web, invented for the sharing of science, enables this discussion. Researchers therefore should embrace the following tools: 1) Pubpeer to comment on articles; 2) Preprints to publish rapidly, minimise the influence of editors, and, dissociate publication, i.e. sharing of information, from evaluation, i.e. peer review; 3) Social networks, e.g. Twitter and blogs, which constitute an ongoing scientific conference to discuss experiments, results, methods, analyses, and new publications.

Acknowledgements: I thank Marianne Noel (IFRIS) for her critical reading of this piece, and, Marianne Lévy for comments on the grammar and style [French version] very necessary after 14 years in an English-speaking country…

Conflicts of interest: The author declares that he has no conflict of interest related to this article.

References:

  1. La Fontaine J (de). Les animaux malades de la peste. Fables, 1678. [full text]
  2. Ioannidis JPA, Boyack K, Klavans R, et al. How to make more published research true. PLoS Med 2014 ; 11 : e1001747.
  3. Baker M, Dolgin E. Cancer reproducibility project releases first results. Nature 2017; 541 : 269-270.
  4. Laframboise D. How many scientific papers just aren’t true? The Spectator 2016.
  5. Krauss LM. Donald Trump’s war on science. New Yorker 2016.
  6. Dascal M. The study of controversies and the theory and history of science. Sci Context 1998 ; 11 : 147.
  7. Latour B. Pasteur et Pouchet : hétérogenèse de l’histoire des sciences (sous la direction de Michel Serres). Éléments d’histoire des sciences 1989 : 423-45.
  8. Pestre D. L’analyse de controverses dans l’étude des sciences depuis trente ans. Mil neuf cent. Rev d’histoire Intellect 2007 ; 25 : 29-43.
  9. Jackson AM, Myerson JW, Stellacci F. Spontaneous assembly of subnanometre-ordered domains in the ligand shell of monolayer-protected nanoparticles. Nat Mater 2004 ; 3 : 330-6.
  10. Jackson AM, Ying Hu Y, Silva PJ, Stellacci F. From homoligand- to mixed-ligand- monolayer-protected metal nanoparticles: a scanning tunneling microscopy investigation. J Am Chem Soc 2006 ; 128 : 11135-47.
  11. DeVries GA, Brunnbauer M, Hu Y, et al. Divalent metal nanoparticles. Science 2007 ; 315.
  12. Centrone A, Penzo E, Sharma M, et al. The role of nanostructure in the wetting behavior of mixed monolayer-protected metal nanoparticles. Proc Natl Acad Sci USA 2008 ; 105 : 9886-91.
  13. Djuranovic P. Seven years of imaging artifacts: what gives? Rapha-Z-Lab 2012. https://raphazlab.wordpress.com/2012/12/11/seven-years-of-imaging-artifacts/
  14. Levy R. Divalent metal nanoparticles. PubPeer. https://pubpeer.com/publications/4DA88768C5B279E24E469CC0080A47
  15. Verma A, Uzun O, Hu Y, et al. Surface-structure-regulated cell-membrane penetration by monolayer-protected nanoparticles. Nat Mater 2008 ; 7 : 588-95.
  16. Xia T, Rome L, Nel A. Nanobiology: particles slip cell security. Nat Mater 2008 ; 7 : 519-20.
  17. Cesbron Y, Shaw CP, Birchall JP, et al. Stripy nanoparticles revisited. Small 2012 ; 8 : 3714-9.
  18. Yu M, Stellacci F. Response to “Stripy nanoparticles revisited”. Small 2012 ; 8 : 3720-6.
  19. Levy R. Stripy timeline. 2012. Rapha-z-lab https://raphazlab.wordpress.com/2012/12/20/stripytimeline/
  20. Dove A. Do these stripes make my nanoparticles look weird? 2012. http://alandove.com/static/2012/12/do-these-stripes-make-my-nanoparticles-look-weird/
  21. Fernig DG. Ferniglab Blog. https://ferniglab.wordpress.com/?s=stripy
  22. Neuroskeptic. Science is interpretation. Discov Mag Blogs 2014. http://blogs.discovermagazine.com/neuroskeptic/2014/01/04/reanalysis-science/
  23. Natelson D. A nano-controversy. Nanoscale views 2012. http://nanoscale.blogspot.co.uk/2012/12/a-nano-controversy.html
  24. Stirling J, Lekkas I, Sweetman A, et al. Critical assessment of the evidence for striped
    nanoparticles. PLoS One 2014 ; 9 : e108482.
  25. Ong QK, Stellacci F, Jeschke G, et al. Response to “Critical Assessment of the Evidence for Striped Nanoparticles.” PLoS One 2015 ; 10 : e0135594.
  26. Rodríguez-Lorenzo L, la Rica R de, Álvarez-Puebla RA, et al. Plasmonic nanosensors with inverse sensitivity by means of enzyme-guided crystal growth. Nat Mater 2012 ; 11 : 604-7.
  27. la Rica R de, Stevens MM. Plasmonic ELISA for the ultrasensitive detection of disease biomarkers with the naked eye. Nat Nanotechnol 2012 ; 7 : 821-4.
  28. PubPeer “Plasmonic nanosensors with inverse sensitivity by means of enzyme-guided crystal growth”. https://pubpeer.com/publications/3E8208F0654769A44C22D4E78DA2B8
  29. PubPeer “Plasmonic ELISA for the ultrasensitive detection of disease
    biomarkers with the naked eye”. https://pubpeer.com/publications/54AECF24E96162E3A563AED08BE0B3
  30. Des nanoparticules d’or pour dépister VIH ou cancer à l’œil nu. Le Monde
    2012.
  31. Bates C. Colour-coded blood test that turns blue if you have HIV is 10 times
    more sensitive than current methods. The Daily Mail 2012.
  32. Cutler JI, Auyeung E, Mirkin CA. Spherical nucleic acids. J Am Chem Soc 2012 ;
    134 : 1376-91.
  33. Rosi NL, Giljohann DA, Thaxton CS, et al. Oligonucleotide-modified gold
    nanoparticles for intracellular gene regulation. Science 2006 ; 312 :
    1027-30.
  34. Seferos DS, Giljohann DA, Hill HD, et al. Nano-flares: probes for transfection
    and mRNA detection in living cells. J Am Chem Soc 2007 ; 129 : 15477-9.
  35. Zheng D, Seferos DS, Giljohann DA, et al. Aptamer nano-flares for molecular
    detection in living cells. Nano Lett 2009 ; 9 : 3258-61.
  36. Prigodich AE, Seferos DS, Massich MD, et al. Nano-flares for mRNA
    Regulation and Detection. ACS Nano 2009 ; 3 : 2147-52.
  37. Choi CHJ, Hao L, Narayan SP, et al. Mechanism for the endocytosis of
    spherical nucleic acid nanoparticle conjugates. Proc Natl Acad Sci USA
    2013 ; 110 : 7625-30.
  38. Wu XA, Choi CHJ, Zhang C, et al. Intracellular fate of spherical nucleic acid
    nanoparticle conjugates. J Am Chem Soc 2014 ; 136 : 7726-33.
  39. Schneider L. Do nanoparticles deliver? Merck’s Smart Flares and other
    controversies, 2015. https://forbetterscience.com/2015/11/20/do-nanoparticles-deliver-mercks-smart-flares-and-other-controversies/
  40. Mason D, Carolan G, Held M, et al. The spherical nucleic acids mRNA
    detection paradox. ScienceOpen Res 2016 ; DOI: 10.14293/S2199-
    1006.1.SOR-CHEM.AZ1MJU.v1.
  41. Briley WE, Bondy MH, Randeria PS, et al. Quantification and real-time
    tracking of RNA in live cells using Sticky-flares. Proc Natl Acad Sci USA
    2015 ; 112 : 9591-5.
  42. Mason D, Levy R. Sticky-flares: real-time tracking of mRNAs…
    or of endosomes? bioRxiv 2015. http://biorxiv.org/content/early/2015/10/19/029447
  43. Munafò MR, Nosek BA, Bishop DVM, et al. A manifesto for reproducible
    science. Nat Hum Behav 2017 ; 1 : 21.
  44. Glanz J, Armendariz A. Years of ethics charges, but star cancer researcher
    gets a pass. New York Times 2017