In this post and the next two, I want to address a problem which goes beyond the question of interpretation of data, namely the problem of important claims which are supported by barely any published evidence, or even in one case, no published evidence at all.
I will give three examples, this post contains the first example.
Example 1: Transmission Electron Microscopy
In the response to “Stripy Nanoparticles Revisited”, the authors state that:
The STM evidence produced was backed up by:
* Trasmission electron microscopy (TEM) […] [2 ]
In 2004, we presented a series of STM images whose interpretation was corroborated by […] TEM data. [ 2 ]
Reference  is Jackson et al, Nature Materials, 2004 and indeed, a similar claim is made:
Additional confirmation of the presence of ordered phase-separated domains was provided by transmission electron microscopy (TEM) images. In fact, in these images (see Supplementary Information, Fig. S2) we have found that there is an observable ring around the nanoparticles’ metallic cores consisting of discrete dots spaced ∼0.5–0.6 nm.
The entirety of the published TEM evidence that “backs up” or “corroborates” the STM is reproduced below:
1) this is showing 3 particles in total; the “ring of dots” is shown on one particle in total.
2) there is no evidence that the “dots” seen in fig S2A corresponds to the stripes [the grid itself shows similar fluctuation in grey levels; the origin of the contrast is attributed to “adsorbed” ions (which ones? how does their electronic density compare to gold? controls with different ions?);
3) Fig S2b does not show anything;
4) Fig S2c does not show anything either, except arrows which are supposed to indicate the stripes.
5) No controls (i.e. images of non-stripy nanoparticles taken in similar conditions) are shown
To say that TEM corroborates the existence of stripes therefore corresponds to an extremely optimistic interpretation of a very small body of published evidence.
Update (21/12/2012): David A. Muller, professor of Applied and Engineering Physics at Cornell University, and the co-director of the Kavli Institute at Cornell for Nanoscale Science, said… (commenting at Douglas Natelson “Nanoscale views” blog)
Funny thing is the TEM image from their appendix, Fig S2a, that is cited as independent confirmation is also an instrumental artifact. The ring of black dots is the out of focus point spread function (basically a Fresnel fringe). This is a very common problem for casual users of a TEM who are looking for core-shell nanoparticle structures. By changing focus, either a dark or bright ring can be created. Going in to focus will make it go away. The focus of the image can be determined from a quick FFT of the amorphous background, and sure enough, the passband is about a factor of two off from the optimal defocus.
I guess one of the dangers of interdisciplinary work if you try to do everything yourself instead of forming teams of experts is that there are an awful lot of artifacts to learn about – sometimes the hard way.