Pep Pàmies, Editor at Nature Materials, has commented on this blog prolifically as Pep. He later apologized and insisted in his note (“On my comments on Lévy’s blog note“) that the note and all of his comments had been made in personal capacity. Philip Moriarty provided an excellent response.
There is one aspect of the note that Philip does not address. Pep wrote:
Nature Materials gladly considered for peer review Levy’s Correspondence detailing scientific concerns on Stellacci’s work published in 2004 in the journal. The outcome of the peer-review process, which I know for a fact that it was carried out appropriately, was clear-cut: sufficient technical arguments were raised against publication of the Correspondence. Lévy stated that the process was unfair because reviewers did not have the appropriate expertise. This is false.
Pàmies comments on the peer review process of stripy revisited and uses arguments of authority based on confidential information – available to him as an Editor at Nature Materials -.
Pàmies – in personal capacity -writes that the reviewers reports (July to September 2009) included “sufficient technical arguments […] against publication of the Correspondence” but, unfortunately, Nature Materials does not allow those reports to be published and therefore the existence and validity of those arguments, if they exist, cannot be discussed.
I did ask for authorization to share this information back in December but Vincent Dusastre – writing as Chief Editor of Nature Materials – replied the following:
As for your request to publish our decision and referees’ reports I am afraid that we are tied by a confidentiality agreement with our reviewers so I cannot give you the green light on this matter. Of course I cannot prevent you from posting this confidential information but it would be without our consent.
Rapha-z-lab 
Raphaël,
I have been following this story recently with some interest, not least because in a previous lifetime I was the Editor-in-Chief of Small up to the end of 2007, the journal in which you published your Correspondence article regarding the work from Stellacci and co-workers, and where their response was also published. I am no longer working in the scientific publishing area, so I feel that I can be totally objective about the whole matter, but I would first like to congratulate you and my successor at Small for embarking on a course of action that should be mandatory whenever there are any concerns about the veracity of a piece of scientific research. Correspondence articles are intended for this very purpose, and I have always believed that they form an integral part of the make-up of a quality scientific journal – there must always be a route available to question findings published as fact, and to use the outcomes of these point/counterpoint discussions as a basis for a conclusion that satisfies both the readers and, ideally, the investigators involved.
Having read many of the details of this case, I would summarise by saying that in my opinion (I’m not a trained microscopist, but was a materials chemist and an Editor) there is clearly some doubt about how these results are being interpreted and this needs to be ironed out. I feel that the arguing between the two “rival factions” has become somewhat undignified at times through the blogging media, and this is unfortunate. At all times, the language and discussion must remain professional and impersonal, and I feel that standards at times have dropped. On the subject of professionalism, I was staggered by the actions of the editor from Nature Materials. While editors are certainly entitled to opinions of their own, particularly where subject matters impinge on current or previous scientific expertise, I feel it is utterly unprofessional to appear to support or defend any individual piece of research in this manner. As Editors, we control what appears in a journal by not only coordinating the best possible peer-review processes, but also by acting as the guardian of the interests of the journal and its readership. Editors should never, in my view, raise their head above the parapet on a blog such as this or any other, and defend a piece of research, particularly in such a half-baked pseudo-anonymous manner. Since I am no longer an Editor, I don’t have to play by these rules! My name is easy enough to find if anyone really wants to, but it’s not relevant to this discussion really…
The only way in which I agree with his comments is that I would concur that all evidence needs to be considered in order to reach a sensible conclusion to this issue, which is as accurate as possible. Where mutliple analytical techniques are being employed to study a material, a team of experts that really understands the various techniques is essential and no corners should be cut. I am aware that raw data of the original experiments has been made available, and this has been instrumental in casting doubt on the published findings – this is good scientific practice, and should be applauded. But now it is essential that a route is found to get to the bottom of the this, and reach tangible conclusions rather than bat arguments across the internet. If the follow-up data in the Corespondence article from Yu and Stellacci is not good enough to satisfy you, then you must find a way to obtain the “right” answers.
There is really only one sensible way to settle this issue: The two “sides” must work together to find solutions to whether the stripes on these nanomaterials are real or artefacts. In an ideal world, I would hope that you would look to carry out a joint study with Prof. Stellacci’s group, and do “what’s right for science”. I’m not naive – I realise that it is unlikely to happen, but it is the only logical way to deal with this in a professional and mature manner. I do not speak for my former journal any more, but it would seem eminently sensible to engage again with my successor at Small to put together a plan to publish whatever joint findings might result. I don’t think I’ve ever met you – apologies if I did – and I know Prof. Stellacci reasonably well. I would like to think that both of you have the best interests of scientific endeavour and ethics to heart. I hope you find a way to work together to fully show that this is the case.
Two final points… I wanted to pick up on a comment that you made regarding the length of time taken for the Correspondence article to be published following its submission to Small. Clearly, I was not there in person, so I have no idea of the minutiae of what went on over that time, but I am very confident that my former colleague(s) will not have felt able to publish your article until such a time that they were satisfied that it was reasonable to do so. This will have included the period in which Prof. Stellacci was made aware of your article and given time to respond. This is absolutely standard practice and takes time. You are right to point out that other related articles have been published over that period, and that these also, in your view, may contain questionable data. Well, you have now started a process through your Correspondence, which will have the necessary outcome of proving or disproving your objections. If you are right, things can be done to correct the situation. That’s all part of publishing science. My final point attempts to answer the question you tantalise us with in the title above… this is only my view, but I feel that Dr. Dusastre is correct and I would have said exactly the same thing. As an editor, I personally view confidentiality of peer-reviewer’s comments as sacrosanct. Such agreements to keep comments confidential do much to retain the status quo between the author, the publisher and the rest of the community. I personally feel it would be somewhat arrogant to ignore such confidentialities and disclose comments that were given in good faith. Others will have differing opinions, and may feel that all comments (and indeed reviewer identities) should be made public, and I am aware that various peer-reviewing bodies have processes that move from one end of this scale to the other. I respect these views but only aim to give you my opinion, and as such I would ask that you respect the trust that the reviewer placed when filing his or her report. Whether that reviewer was properly qualified to deliver this assessment, which you infer was not the case, is another matter and one that should be dealt with directly with the publisher.
I welcome the opportunity to state my thoughts on these matters. I am fairly confident that this will be my one and only contribution, so I express my wish that some good will come out of this. I am glad that your Correspondence was published and that it has served to move the issue forward. It will be best for everyone if you now work together to either prove or disprove the findings concerned and put the record straight one way or the other. I wish you all the best of luck in doing so…
LikeLike
A quick thank you (prior to a longer response) to GH for this valuable contribution and advice, in particular on the confidentiality of the reviewers comments; a point on which I am unsure as to the best course of action.
LikeLike
I would like to echo Raphael’s thanks for your considered and extremely helpful contribution to the debate. In addition, I want to address two points you raise, as briefly as possible (because I have already contributed more than my fair share of comments here).
The undignified language to which you refer was indeed regrettable and I am certainly guilty of overstepping the mark in my initial response to Pep. I spent some time over the Christmas holiday mulling over just what provoked such a strong reaction. To my mind, there are three especially frustrating aspects of the ‘striped nanoparticle’ controversy which raise(lower?) it beyond a simple/minor case of misinterpretation of measurements.
(i) The lack of professionalism shown by Pep Pamies, which you yourself say you found staggering.
(ii). The extremely low level of quality control in peer review spanning a number of papers. These are not rare, esoteric artifacts only identifiable by a few experts in the world; STM./AFM manufacturers discuss these artifacts in the manuals they supply with their instruments. (I have already mentioned elsewhere how worrisome it was to be informed by an editor of Nature Materials that reproducibility of features from one scan to another is not a criterion for publication.)
(iii). Most importantly, a graduate student (Predrag (Pedja) Djuranovic) did the appropriate control experiments and failed to reproduce the stripes found by others in the group. He raised his concerns as long ago as 2005 and was effectively ignored (or worse).
When students/postdocs careers are being directly affected then we can no longer afford to consider this as a ‘storm in a teacup’ regarding a relatively niche area of nanoscience. The implications are much broader.
The other point in your comment to which I’d like to respond is the following:
The two “sides” must work together to find solutions to whether the stripes on these nanomaterials are real or artefacts.
In one of my first e-mails to Francesco Stellacci, I asked if he could send some of his nanoparticles to me so that our group in Nottingham could image these using low temperature (5K) UHV STM and dynamic force microscopy. Francesco agreed to do this and this is very welcome.
But, and although this might sound churlish, even if we (or another group) find evidence for stripes on the nanoparticles, this will not vindicate the STM data and analysis published in the papers of the Stellaci group to date .
Best wishes,
Philip
LikeLike
Please excuse the typos.
That should be, of course,
“When students’/postdocs’ careers…”
I also left out a question mark:
“He raised his concerns as long ago as 2005 and was effectively ignored (or worse?).”
Apologies.
LikeLike
GH,
Thanks again for your comments. I fully agree with Philip response below that highlights three reasons why this case cannot be described just as a difference of interpretation over a set of results. I would add a fourth one: the absence of data to support key claims – to me, this fact, and the absence of concerns from Editors, remains an astonishing mystery. We do need to understand and rectify this issue. Why Pep – and other editors – do not consider it highly problematic that key claims in the 2004 paper, but also in others, including in the latest 2012 paper, are not backed up by any published experimental evidence at all? Scientific journals have a responsibility in preserving the integrity of the scientific record and there is here a clear and undeniable failure (totally independently of the interpretation of STM images). I am not sure what is the right forum to discuss and resolve this problem – would COPE be the right organization to look at this (http://publicationethics.org)?
On the second point, i.e. the way to resolve the purely scientific issue, I do not fully agree with you. I have met Francesco Stellacci a number of times over the past few years and I have had a number of civil exchanges with him but I have come to the firm conclusion that those exchanges are not a suitable route to sort out this issue as agreement will not be reached. In any event, resolution, as far as the scientific controversy is concerned, is the emergence of a consensus opinion among specialists in the field: my/Francesco’s personal opinions are of little importance. Exposing arguments and experimental evidence through peer reviewed articles as well as blogs and conferences will contribute to this resolution. Other techniques (crystal structures, EPR, NMR, etc) will also provide new insights that may render this controversy obsolete as noted by Mathias Brust in an early comment on this site.
Finally, on the Small editorial process. I certainly thank your successor, Jose Oliveira, for accepting to consider the article and for getting the process through. It would be interesting to hear directly from him why this has taken so long. I’d like to stress that during those three years, the ms was in “our camp” (responding to reviewers comments, proofs, etc) for less than two months in total. It was in Francesco’s “camp” for a little longer (see timeline post), but the overwhelming fraction of the time, it was with Small while Jose was trying to obtain reviewers reports. Why has it been so hard to get those reports, I do not know.
LikeLike
1. Pep seems sincere and pleasant.
2. The pseudonomty does not make sense. He says his name would be obvious, but when challenged, he did not disclose it. And when exposed, he left. [And yes, I realize I’m pseudo too…but I have been out of science for years…work in finance…just not my fight to have my name here…but I care about science…and enjoy a little blog drama. ;)]
3. The private versus public seems very confused. He was privy to the peer review process (in 2009), but says his remarks are private only? He seems to have learned of the matter and become interested in it based on work as an editor (or a friend of FS?)
4. What has his work been as an editor that touched on this? Was he responsible for any of the FS papers?
4.5. I can feel for Pep since he seems to honestly be trying to weigh arguments and make sure you don’t cherrypick your crits…but the lack of clue on the imaging is worrying. How many other “snowballs” are going to get past him because of people citing different analytical techniques? Perhaps it is the NPG tendancy to publish short, sexy papers that leads to this (not having solid data and evidence to back up discussion/conclusions)
5. You probably should not share the reviewer remarks if you want to publish at NPG, but based on the Small paper, this blog and even the cover letter…I have a strong impression that NPG spiked this. It is a very normal experience with Comments or critical papers. That the Small paper came out and is (in GH’s opinion) worthy of at least airing, seems to show that NPG screwed up.
6. I agree with Phil’s churlish point…that if the stripes are found, that 2004 image will still be junk. But based on the extent of time, the guy has had to definitely show the stripes and that he has not…I really doubt the stuff is striped. In any case, from a pure materials standpoint, it would be good to learn if the striping is happening or not (for one thing, it encourages examination of the particles for properties, etc…and conversely if they are not striped, it maybe indicates a broader issue of sketchy work on the properties of the balls). Based on what I’ve seen, I suspect the latter.
LikeLike