Challenging published results is an onerous but necessary task. Today, our article entitled Stripy Nanoparticles Revisited has been published in Small, three years after its initial submission to this journal (3/12/09) and about three and a half years after the first submission (to Nature Materials, 21/07/09).
As its title indicates, the article challenges the evidence for the existence and properties of “stripy” nanoparticles. The stripy nanoparticle hypothesis was first proposed in Nature Materials in 2004 by the group of Professor Stellacci (then at MIT and now at the EPFL). This hypothesis now forms the basis of 23 articles by the same group, mostly published in high impact journals including Nature Materials, Nature Nanotechnology, Nature Communications, Science, Journal of the American Chemical Society, Small, etc: 1, 2, 3, 4, 5, 6, 7 , 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22 and 23. Our article today is followed by a response from Professor Stellacci.
The stripy hypothesis can be described in simple terms as follows:
- take a ball (gold nanoparticle of ~4 nm diameter)
- cover it with two types of hairs, some short yellow ones (a small molecule with a thiol to bind to the gold) and some longer red ones (a small -but slightly longer- molecule)
- the hair will spontaneously organize to form a stripy ball with red and yellow hairs forming parallel lines (see cartoon below)
Image from Stellacci/Irvine press release reproduced from http://www.nanotech-now.com/news.cgi?story_id=29597
This is a remarkable hypothesis. It did not come from a theoretical argument but from an observation. On the nanoscale, it is extremely difficult to visualize how molecules organize. In the 2004 article, the authors decided to use scanning tunneling microscopy to “watch” molecules on nanoparticles. Scanning tunneling microscopy does allow atomic resolution imaging on very flat surfaces (and its inventors obtained the nobel prize for this reason); it works by scanning the surface with a very sharp tip. It is however a technique which requires particular care in the interpretation as artefacts can easily occur. It works well on very flat surfaces but is not normally used to look with sub-nanometre resolution at very bumpy surfaces such as films of adsorbed particles. In the 2004 article, the authors obtained images and proposed a model to fit these images; in you follow the link (first figure of the 2004 article), you can see some images in the panel a) and b), and the cartoon in panel c).
In Stripy Nanoparticles Revisited, we demonstrate that the stripy hypothesis is based on an artefact. First, we show that the images are not compatible with the stripy hypothesis for basic geometrical reasons (if stripes are regularly spaced in 3D they cannot be in a 2D projection). Second, we use an image analysis technique called fast Fourier transform to study the periodicity in these images and we prove that the stripes are in fact an imaging artefact related to the scanning direction.
This explains the extremely improbable fact that the 10 “stripy” nanoparticles seen in panel a are all aligned in the same direction; why would they? and can you find what’s wrong in the movie below?
In Stripy Nanoparticles Revisited, we also consider some of the follow-up articles and in particular some of the very special structure-related physico-chemical and biological properties claimed and we fail to corroborate those.
In summary, Stripy Nanoparticles Revisited shows that 23 peer reviewed articles published over the course of 8 years in prestigious journals are based on a simple microscopy artefact. This perhaps suggests a failure of the peer review system, in particular given that ~ 10 of these articles have been published while our attempt to open this discussion into the scientific community was slowly going from submission (3/12/2009) to publication (23/11/2012).
We welcome comments on the scientific argument as well as the broader issue of the process of establishing scientific knowledge.
Update 1: Responding to response?
Update 2: From above, the view of my Head of Department, Dave Fernig
Update 3: Seeing is believing? Not always… guest post by Philip Moriarty
Update 4: Scientific claims should be supported by experimental evidence, part 1, transmission electron microscopy
Update 5: Scientifc claims should be supported by experimental evidence, part 2, water-soluble stripy nanoparticles
Update 6: Alan Dove comments on the web and peer review in his post entitled Do These Stripes Make My Nanoparticles Look Weird?
Update 7: Scientific claims should be supported by experimental evidence, part 3, non-specific interactions with proteins
Update 8: Simon Hadlington covers the controversy, Chemistry World
Update 9: Gaping holes in the gap; no data to support the existence of the stripes that are supposed to catch toxic ions in the latest Nature Materials installment of the stripy series…
Update 10: Predrag Djuranovic, former graduate student of Francesco Stellacci, explains how he came to the conclusion that the stripes were an artefact seven years ago.
Update 11: I am not updating here anymore; instead, see the round up post
Rapha-z-lab 
Ive been saying for a long time that people seem to find it so easy to publish “amazing” results, which drop into their laps so easily! Wh then do I find it so hard to do the same? Maybe because I count to ten and have a deep think about the potential artifacts!
Well done Raphael!! You give me hope!
LikeLike
Scientific journals are more like tabloids nowadays. “Letters” with “shocking results” based on one or two experiments rather than comprehensive studies are preferred even for traditional journals like JACS. This is an unhealthy trend for science, which only promotes impetuosity and rashness. I believe Editors should be blamed to a large part for this. While everyone is seeking citation rates, Editors, as caregivers to the journal, should be the most important resort to protect scientific journals. Especially, I want to criticize the Nature Groups, which expedites the spread of the fast food taste to the scientific community in all areas.
LikeLike
I was forwarded an email written by you from a colleague containing this link. I find the controversy all quite interesting, and have thus spent the morning scanning the links from the Stelacci group you provided.
If I understand correctly, are you asserting that 23 ‘high impact’ papers, since they are all based on, as you say, an incorrect model, are all falsified? That is quite an accusation based on your evidence of only a single image from the stripy particles paper! You even declare a failure of the peer review system! Admittedly, when I get a paper rejected I ring my hands at the peer review system as well.
More to the point, I have just looked at the first 10 or so papers in the list you provided, but it seems as though they are describing more of the effects and properties of the stripy particles, rather than the existence of stripy particles themselves. A more general question is, shouldn’t the onus of proof be on the critic to provide an alternative explanation of these many properties? Otherwise you should simply come out and say what you mean. That is Professor Stelacci (and dozens of co-authors) knowingly falsified 23 papers over the past 8 years.
All that being said, I certainly agree with you that challenging published results is an onerous task, and I commend you for sticking your neck out, it certainly takes a great amount of bravery. I only wish you would provide some more compelling evidence besides the single photo from 2004 for the serious allegations you have made!
LikeLike
Good work. The first time I saw those images, I was very very doubtful about them. I think it would be very easy to produce such images, by having some periodic noise in a scanning image of “normal” nanoparticles.
LikeLike
Charlotte; thank you for your comment and for engaging in the controversy. I’ll provide a more comprehensive reply later, but I’d just like to clarify immediately that nowhere do I suggest that data have been falsified in the 23 papers mentioned.
LikeLike
Yes sorry, re-reading my comment the next day and it seems a bit inflammatory, that’s the danger of late night blog posts I suppose. I guess what I was trying to get it was, if the stripy particles aren’t real, but the properties of particles are, then what could be an alternative explanation? Maybe this could be framed as a larger debate of the responsibility (or not) of the scientific community when critiquing articles.
LikeLike
Very interesting work, and congratulations for your patience (3 years!) to publish a paper. It looks like Small forced you to publish your paper on the same day as Dr. Stellacci’s response, which is against common habit in the scientific community – he must be politically well connected to obtain this treatment! When I look at the images from his papers (like the one you provided above), I see clearly that he always meant that the nanoparticles have stripes throughout their body, and not only in local patches as he’s now stating in his response(!). Also the hairy ball theorem used to prove the existence of point defects should only hold if you have one domain (hence one continuous vector field of ligands); if you have multiple stripy domains on a surface, you will have line defects at the boundaries, not point defects. Thus his own response seems very controversial, and more like a desperate defense. For example, in Figure 3 and 4 of his response I REALLY struggle to see the stripes they mark with dashed lines – if they were orthogonal to the ones shown they would make as much sense to me!:) I also wonder how many of the 23 papers would be directly affected (and thus downright wrong) if the stripy domains are only local and not global, or if they don’t exist at all. For example, the Science paper on the chain formation due to two point defects would seem to be wrong on this basis. If you are convinced about your results, you should go ahead and present this controversy to Nature and Science and ask to have those papers withdrawn!
LikeLike
Tiberiu Onuta
I have a question for Dr. Levy. Who pays for this research? What scientific organization gives grants in order to prove that the research of an another group is wrong? What research grant have you used for this “onerous” task? I would like to ask that European (??) scientific organization if it is aware of how its money was spent.
LikeLike
In the good old days it was considered good and honorable practice to independently test the results of others. It is a pity that this kind of debate can be seen as a waste of time and money nowadays.
LikeLike
@Hector; thank you for your comment. It is normal practice (and it makes sense in my view) to publish the critique and the response the same day. See for example the technical comments in Science, or similar exchanges of views in other publications.
@Tiberiu; as indicated in the article acknowledgments, I was then a BBSRC David Philips Fellow (http://www.bbsrc.ac.uk/funding/fellowships/david-phillips.aspx), i.e. funded by the British tax payer. By “onerous task”, I meant difficult, long, etc, rather than costly, but it is nevertheless a significant investment. BBSRC is aware of my involvement in this controversy, and, I am thankful for the fact that I have never been discouraged to pursue this line of research by the funding agency nor my institution. Both recognized the importance of scientific debate and scientific controversy in the making of science.
LikeLike
Haha, is it just Stellacci’s work or this is common practice for you to spend your time trying to refute published work? I barely have enough time to cope with my own research! Was it because his work with cells was similar to yours?
@Hector, I think part of the danger in this debate is comments like yours, that it’s hard to ‘see’ the stripes. Visually identification of any trend should be not considered as our eyes are not to be trusted. Rather using statistical approaches with software to count large numbers of images should be better, as they did in their publication.
@Lévy, I think your argument showing the improbability that the stripes would align is quite persuasive. I wonder, how would you refute the many other pictures they provide where this is not the case, such as those they statistically analyzed from their 2006 STM follow up paper? It seems like that picture is only one small part of their argument for stripes.
LikeLike
@Tom -1; looking critically at the papers we read is certainly part of our jobs as scientists. The possibility of refutation is an essential condition of the progress of science. The reasons we took the unusual step of going from “critical look” to “refutation” include the impact of those papers on the understanding of phenomena which are of primary interest in my lab, including molecular self-organization on nanoparticles and interactions of nanoparticles with cells.
@Tom -2; they did not use automated unbiased image analysis of a large number of images in the first publication nor in the follow up 2006 JACS paper. In the latter, there are a large number of images (analysed by measuring spacing by hand, very little detail provided), and they recognize that the feedback artefact is present but argue that it is only present on the flat surface and not on the particles.
LikeLike
As I am likely to have forgotten all about this problem by the next time you publish a response, I wonder if you could give us a brief idea of your thoughts on the response. I would guess based on these two papers that you were offered a chance to see their rebuttal before publishing and kept on. To me this means you must have a response to their response!
If I could sum up the arguments you and they make:
1) From their original work, the stripes are all lined up and you find that a reasonable explanation for that would be noise, in addition to the logical argument that a surface randomly coated with striped balls would not result in an alignment. Also, the stripes should decrease in width at the edges, since you are representing a 3D ball with a 2D projection.
Their response is that the tip does follow a 3D shape while tracing the particle surface, not moving in a 2D line central to your argument in your Figure 1, and that the current may tunnel at an angle through the particle core, both of which result in a flattening of the image that is consistent with their interpretation. Furthermore, they posit that your 2D-3D argument should apply not just to striped particles but also to homoligand SAMs on curved particles. They present new experimental images that show isolated head groups at the center and edges of a single particle, yet without distortion of the spacing. They also posit that your argument (with a spice of ad hominem attack) is invalid since they never claimed to see stripes at the edges of particles, where the images are fuzzy. Finally they show in Figure 3 stripes in different directions within a single image (as opposed to their initial paper).
2) The FT transform argument you present and they contest is maybe more tricky to briefly discuss, and complicated for easy response here so you can tackle it if you want but probably would need images and such, for which this is not such a good format.
3) You argue that there is an artefact in scanning due to a feedback loop with STM imaging. They say that this should be dependent on tip speed, and at different experimental tip speeds, they show images that the spacing is the same. In contrast to Hector, the earlier commenter, the stripes in their images are pretty clear for me to identify in the lower panels of Figure 3 of their response. Figure 4, however, in my opinion is a stretch to visually identify stripes.
4) He argues that you ignore all simulation data presented by their group showing the formation of stripes under certain conditions.
5) Cell work – This one is more in my wheelhouse so I’ll expand due to selfish personal interest!
You argue that fluorescence measurement is unreliable due to certain limitations, namely that they require the attachment of a dye molecule, that may be quenched by the gold, fluorescing brightly only upon release from the NP. They argue that this would be the same regardless of NP type, considering their ligand density and size is the same, so this still does not explain why they see a striking difference in the MUS/OT striped case. You then argue that DC cells are no good for internalization studies. They reply that they’ve recently released a paper showing the same results in HeLa cells and even others (albeit with fluorescence).
You argue that with photothermal heterodyne imaging where you image directly the signal from the particles, you saw no difference at 3 hours between the particle types. They cite their 2011 ACS Nano publication using heterodyne imaging in which they in fact find a significant difference between the MUS/OT and MUS particles, with pictures. They also show some cryo-TEM pictures showing NPs distributed in the cytosol, which they argue you ignored. Finally they cite papers in which they’ve observed the same results in one hour incubation (and actually I’ve found in their Biointerphases 2012 paper fluorescent images at less than 10 mins) that should be well under the time frame of endocytotic uptake. For my own refereeing I might also point out that you did not try the chemical endocytotic blockers or low temperature 4 C cell incubation, where endocytotic pathways are significantly retarded from an energetic stand point. They don’t argue that endocytosis is absent, and in fact even in their 2004 paper, the diffuse signal is overlaid on endocytotic spots at 37 C, but not at 4 C. Finally, they always image live cells, while you image fixed cells, and as a cell imager myself, that is quite a large discrepancy.
I’d love to hear your point-by-point responses when you feel up to it!
Either way, you both should hand it to yourselves. Constructive disagreement is what our great study of science is all about!! Good job! I hope your negative publishing experience with this paper doesn’t refrain you from questioning results in the future.
LikeLike
on point 1; see “responding to the response”; for the other points, you will have to wait a bit more… https://raphazlab.wordpress.com/2012/11/25/responding-to-the-response/
LikeLike
Has anyone ever rotated the sample and showed images as a function of the rotation angle? (I’m totally ignorant on STM but I’m wondering if such an experiment is feasible?)
LikeLike
@S: excellent question.
JACS 2006, p11135-11149; figures 8 and 9.
Figure 8 attempts to convince that stripes can be seen which are at 45 degrees to the scanning direction but it uses a zoomed figure – panel d – where the image has been rotated (probably accidentally while using a 3D viewing).
Figure 9 does two things: 1) it shows two scans where the scanning direction has been rotated by 30 degrees and highlight one particle where the directions of the stripes is supposed to have become “difficult to discern” (panels a and b); 2) it attempts to convince, using a cartoon, that, in fact, the stripes should only be seen when they are perpendicular to the scanning direction (panels c and d).
Far from convincing…
LikeLike
Thank you for pointing out those references.
LikeLike
It used to be common for scientists to challenge published results in subsequent papers. Some examples from the good old days(?!): ‘bond stretch isomerism’ (http://pubs.acs.org/doi/abs/10.1021/ar00022a004), manganese(II)-phosphine ‘reversible binding of dioxygen’ that turned out to be phosphine oxide complex formation, ‘negative rate constants’ (yes, I didn’t make that up; http://pubs.acs.org/doi/abs/10.1021/jo00022a001), a magnetic effect on enantioselectivity of reactions (http://onlinelibrary.wiley.com/doi/10.1002/anie.199404541/pdf), and so on. Some of these were honest mistakes, one at least was outright fraud. I find some of the responses to Raphael’s paper and blog along the lines of ‘why are you wasting your time / the public’s money challenging other people’s findings?’ disturbing. This is supposed to be how science works! Well done, Raphaël et al for pursuing the correct scientific course. It’s not all about getting your name in a high-impact journal for whatever you can get away with.
LikeLike
I fully agree with Simon. I have initially not seen the need to be so persistent with this, as I thought science would correct itself eventually, but that may only be the case for flawed claims of potentially really important discoveries, such as cold fusion, for example. Relatively inconsequential claims can stick around for a long time if they are not challenged directly. Think of the iron content of spinach! I can see this may also be the case here and I am happy that Raphael has finally succeeded in getting his views published in a respected peer reviewed journal. The stripes are now officially “debated ones”, an invitation for the nanoparticle community to do more work. I hope journal editors will learn from this episode and become more open to scientific debate in the future. The sooner this happens the more damage can be prevented, to all involved.
LikeLike
Everyone leaves encouraging words for Levy and co, like he’s fighting a losing battle against the machine, and against all odds! Does anyone really think that the people he is publishing against really believe that scientific debate should be eliminated? Levy has a binary mind where any piece of evidence that is presented against him, he simply finds an explanation (weak or not) to the counter. This could be done to literally any argument that one could come up with, including any paper published by any of you. Really, you only challenge the work of ONE scientist, who happens to publish in an area that you seem to have trouble in. The experiments you used in your attack simply do not stand up to the rigorous level that you demand from the one you are attacking! The cell experiments are laughable they are so incomplete of controls. The article is dripping of sarcasm and personal opinion. If I really felt that this debate was actually about stripes and not about one guy trying to make a name for himself without his own ideas but rather by attacking another, we could have a civil discussion. Instead, you chose to blog online your personal opinions about how ‘far from convincing’ peer-reviewed articles are. Well I’m peer-reviewing your baseless blog: this stinks! You have to pick one, are you a scientist or a blogger? Can’t be both.
You know where attacks like this don’t happen? In peer-reviewed articles.
LikeLike
If you had read more carefully the blogs that encourage or rather
congratulate Raphael you would have noticed that what is welcomed is the debate. Nobody has preempted an outcome of this debate. An exception is Moriarty’s guest blog, which is openly in support of Raphael’s interpretation of the Stellacci data and simply adds to the debate some quite interesting new points on the resolution of SPM and the care that has to be taken when interpreting software zooms. I agree, it would be better to appear in the peer reviewed literature, but maybe Moriarty has more important things to do, and after all, unlike you, he puts some weight behind his blog by signing it with his name that is well known in the community. Knowing about the huge resistance Raphael experienced, before a peer reviewed journal was finally willing to publish his paper, I am afraid there is indeed an element of institutional opposition against this form of debate. This is not new at all and has been pointed out again and again, no space here to amplify on it but believe me. Your ramblings about Raphael’s “binary mind” and the simplicity to find arguments against any evidence are a bit too postmodern for me. Scientists usually assume the existence of a external world of which we can obtain objective knowledge using the scientific method. Now, since you remain anonymous, you may not be a scientist, in which case you can believe what ever you like, but you should then not really be interested in this debate. Regarding the choice of whose work Raphael could have challenged instead, of course it is work in his own field. Nobody has the expertise to simply read the scientific literature and pick a subject for debate. The postulate of stripy particles, in my own view, is difficult to reconcile with many other existing data including now also several X-ray structures. This is enough reason to carefully look at the experimental base on which this postulate relies, and this is what raphael has done in his paper published last week in Small. Regarding your offer of a civilised debate, readers will note that we had exactly that before you came in. With your aggressive anonymous appearance you are not doing a favour to the course or person you wish to represent here, and your accusation that Raphael uses this to make a name for himself does not merit a reply. If you honestly believe this you have very little understanding of what forms scientific reputation. Finally, of course it is legitimate to have a debate outside the peer reviewed literature. People blog about anything, why not about science? There are dedicated internet sites that do nothing but exchange scientific views. If the stripes are real, have no fear, the bloggers will not take them away from you. Consensus in the scientific community will eventually be reached through the peer reviewed literature, despite its recognised shortcomings. Now, there we can agree again.
LikeLike
It could be that “Someone has to say it.” (geolocation in Switzerland according to IP address) has a personal stake in the controversy, or, it may be that he/she is just an independent observer. It does not matter.*
Let’s take the points one by one:
1. “Fighting a battle against the machine” – well there is certainly some element of truth into this; paper initially rejected by Nature Materials based on an unfair evaluation process (at the moment, this is an uncorroborated claim, I will add some information in the coming weeks to demonstrate this point); publication taking three years to come through in Small while additional “stripy” publications were coming through regularly (and their authors had had access to our article, arguments, and experimental results).
2. “Blogging versus peer-reviewed articles” – the question of whether blogs can make useful contributions to scientific debates is an interesting one. The consensus seems to be evolving towards a yes. A case in point is the recent “arsenic-based life” controversy. You definitely don’t have to pick between being a scientist and a blogger.
3. “You know where attacks like this don’t happen? In peer-reviewed articles.” That is generally correct: negative articles are rare and this is a pity because, as noted above, criticism is an essential component of progress of scientific thought. In this particular case, however, after three years and 9 referees reports (!), a level of scrutiny quite extraordinary although probably justified by the size of the body of evidence challenged, the peer reviewed “Stripy Nanoparticles Revisited” article was published in Small.
* just in case, authors of the articles challenged are reading this blog: you are welcome to comment, your comments will be published unaltered. You are also welcome, if you need more space to address some of the scientific points, to write a guest blog which I will publish without changes.
LikeLike
Interesting debate.
While the stripes may very well be real I agree with you that the evidence you mention is far from convincing. That said I find the refutation of your spacing argument valid in that while the scan is indeed a projection it is not a simple projection.
But presented STM images of the stripes are no where near enough to prove they are real. The only way such images would count as evidence in my book would be if they were able to detect many stripes on different particles and aligned in different directions in a single image and using an automated image processing software. Anything detected manually on as poor quality images as those supposedly showing stripes of different alignments is completely unconvincing.
LikeLike
This is a case of another Hwang Woo-suk, the notorious stem cell researcher in Korea.
LikeLike
Please refrain from making such broad brush and very serious accusations here. Stick to the facts. Thanks.
LikeLike
If I’m reading this right, you made stripy particles (well followed the synthesis). this had to be done to test the cell incorporation. Did you image them? what did you see?
LikeLike
adobe lightroom 6 upgrade price autocad license version price in india Adobe Photoshop Elements 12 search store inventory barnes and noble microsoft excel 2010 for windows 8.1
LikeLike