Open science to settle stripy controversy?

Julian Stirling

This is a guest post by Julian Stirling, a final year PhD student at the University of Nottingham.

Those who have been following the stripy nanoparticle controversy will know that Raphaël, along with other researchers, has been writing a paper both analysing the archived STM data provided by Stellacci, and the work presented in some more recent papers. I am happy to announce that our paper was submitted just before Christmas, and we have uploaded a pre-print to the arXiv. Over the coming weeks, we may write some posts to give more background for some of the arguments presented in the paper (I, for one, plan to write a post putting the Fourier analysis in the paper into context for those who are not familiar with it). In this post, I, instead, would like to give a bit of personal background as to how I got involved with the work, and how we have published the work.

I first heard of the stripy work from Phil (Philip Moriarty, who has previously written guest posts at the blog), my supervisor. My reaction was to laugh at what appeared, to me, to be obvious SPM artefacts. It was a good 6 months later (I think), when I heard that Phil had given a new PhD student in the group (Ioaniss – a co-author on the paper) the task of reproducing the stripy images with other nanoparticles. I suggested we could also combine these results with images from an SPM simulator which I wrote as an undergrad. I also offered to help read the raw data into MATLAB as I run an open-source SPM image analysis toolbox for MATLAB. This lead to me helping with analysis, and, before I knew it, I had become the main author on the paper.

The more I delved into the stripy papers and the archived data the more disillusioned I became with the peer review system. This is not an exaggeration. It was not just the stripy artefacts in the original paper, these were bad, but you simply imagine that the reviewers were experts on perhaps nanoparticle synthesis not SPM. The problem was the later papers using “statistical analysis” to separate real stripes from feedback noise. The “statistical analysis” showed such poor methodology that it reminded me of some of the worst data analysis I have seen when marking undergraduate lab work. An example, is averaging using ‘eye-balled’ measurements from SPM images, then mindlessly (I do not use this word lightly) applying standard error calculations to generate uncertainties for these widths which correspond to 0.026 pixels in the original image. Without unbelievable CSI-style image enhancement this cannot be reliable!

The project has slowly consumed more and more of my time (while I was simultaneously trying to write my PhD thesis). A lot of this time was spent doing the analysis which should have been done when the original data was taken. Other times I got stuck trying to understand and reproduce analysis which should never have been done in the first place. Managing to reproduce the published figures from the raw data was not an easy task, often involving extreme offline zooms, combined with interpolation and contrast adjustment. Once this became clear we made the decision to make all of our analysis open. All image analysis was scripted in MATLAB (no graphical programs used), and all data and scripts have been uploaded to figshare. This way anyone who is interested in this controversy can double check every single step of our analysis if they wish. I feel the only way to settle a controversy is to be as open as is possible in terms of both raw data and its analysis.

Finally, when we finished the paper. We had two options. To wait for the paper to be published before making this announcement, or to upload a pre-print to the arXiv. Our decision to release the pre-print also came from our desire to be as open as possible. A number of news sites have picked up on this controversy, which leads us to understand it is of some interest to the nanoscience community. It seems a shame to delay our analysis more than is necessary, especially considering there may be PhD students across the world wasting their valuable time trying to generate striped particles for their work.

I hope you will read our paper. And  I wish you a merry Christmas and a stripe-free New Year.

“Open science to settle stripy controversy?”