Rapha-z-lab 2.0

I published my first post on this blog ~ 7 years ago. Since then, it has had ~ 170 000 views.stats web

I have used it both as a webpage for my group and as a personal group webpage, and although no team member has ever complained, it is now time to sort out this ambiguity and separate the group website from my personal blog. So, please welcome my group website where you will meet group members and learn more about our various research projects.

My personal blog remains here.

There is a third element to Rapha-z-lab 2.0: an open science notebook. Here is team member Dave Mason introducing it. And here is Gemma giving an introduction to the first project that is going to happen in the open.

What does PubPeer need?

Apparently, PubPeer needs legitimacy, accountability and better moderation of comments… and an Editorial Board is going to fulfill all these needs. The debate takes place here.

One proponent of an Editorial Board is Niklaus J. Grünwald, a plant biologist who serves as a senior editor of several journals. At PubPeer, he writes:

In my mind having an editorial board provides legitimacy and accountability to the process of running PubPeer. […]

The legitimacy of PubPeer comes from the communities of scientists that embrace it, not from an Editorial Board. There is maybe still a concern among some scientists about the legitimacy of open post publication peer review per se. I would suggest a manifesto for post publication peer review that state our support for open discussion of science, welcome criticism of our published work, and commit to engage in discussions whenever appropriate. Such a manifesto, if signed by enough (of the right) people might contribute to the much required culture change.

Also in support of an Editorial board, albeit for different reasons is Physical Chemist Francois-Xavier Coudert. He, and a few others including Philip Moriarty, emphasize the need for moderation of comments. François-Xavier starts his comment by arguing that since there is moderation taking place at PubPeer, there is in fact already an (anonymous) editorial board. Moderation of comments is a challenge that PubPeer faces: there is room for progress on how discussions are organised and moderated. It is a real challenge to combine quality discussion with respect for full anonymity (as we have experienced again recently in the stripy controversy), but an Editorial Board will not solve that problem. The current discussion explores a number of interesting models that involve modern communication technologies empowered community efforts, see the comment of Peer 9 who refers us to Slashdot and the comment of Sylvain Bernès who highlights http://stackexchange.com/ (also mentioned by François-Xavier) and http://mathoverflow.net/. Maybe, rather than an editorial board, we just need another Topic at PubPeer to explore these ideas?

 

Novelty, reproducibility, and data sharing in (nano)materials science

Half-random ranty post that might develop into something more structured at some point… Feedback very much welcome.

Andrew Maynard has blogged about the extent to which novelty should (or, in fact, should not) be the main consideration for the evaluation of nanomaterials risks (initially published as an editorial in Nature Nanotechnology). It’s entitled “Is novelty in nanomaterials overrated when it comes to risks” and is well worth reading in full. A central point is that:

Novelty as a result is a subjective, transient, and consequently a rather unreliable indicator of potential risk. It tends to obscure the reality that conventional behaviour can sometimes lead to harm, and that mundane risks are still risks. And it favours the interesting (and possibly the headline-grabbing) over the important. But if novelty is an unreliable guide to potential risk, how can approaches be developed that help identify, understand and manage plausible risks associated with emerging materials and the products that use them?

Apparently unrelated (but wait for the next paragraphs), there are various initiatives to encourage or even mandate sharing of data related to the characterization of (nano)materials. It is thought that this will boost innovation and facilitate the coming together of computational and experimental work. Maybe the most impressive and concerted effort comes from the White House Office for Science and Technology as exemplified by this post It’s Time to Open Materials Science Data. Publishers have smelled something and are moving to the area of providing services for data sharing and curation; NPG launched Scientific Data in partnership with FigShare; Elsevier has just launched an initiative specifically targeted to open data in Materials Science.

Now for the (arguably subtle and tenuous) link. Novelty is overrated not just when it comes to risk. It is overrated in materials science full stop. This seems not intuitive; surely scientific endeavour in materials science is about discovering new materials. The problem here (and arguably the opportunity too) is that there is an immense combinatorial space of potential new materials. We work on peptide-capped gold nanoparticles. By varying the peptide sequences and making various mixed monolayers, we can potentially generate hundreds of novel materials every day (and we do make a fair number). The combinatorial space of potential nanomaterials vastly exceed the number of potential molecules. Most of these materials are not interesting, but they are novel: nobody made them before.

I see a lot of research articles which can be summarised as

  1. This is a novel nanomaterial (and it truly is: nobody has made before this gold-nanorod-with-carbon-dots-at-the-tips-graphene-oxide-on-the-side-and-some-antibody-labelled-conductive-polymer-wrapped-around [1])
  2. It could be used for [delete as appropriate] energy/biological imaging/curing cancer (and it will never be).

When it comes to safety, Andrew argues convincingly that the focus should be on plausible scenarios rather than on novelty. When it comes to what should be curiosity-driven science, there seems to be a lot of new materials generated for the sole purpose of highly improbable applications rather than in the pursuit of general principles that would help us explore the materials landscape. This has the very unfortunate consequence that the materials characterisation is often poor and limited to whatever is thought to enable the envisioned application. An extremely large proportion of these new materials are made by a single group for the purpose of a single paper. The experiments are not reproduced independently. Capturing all of this data into platforms that are open and suitable for data mining is a noble and worthwhile purpose which I support, but it must be accompanied by a change of focus and higher standards of characterisation otherwise I fear that it will not help understanding much.

Thanks to who chronicled the reaction of materials scientists to an OFST presentation at the MRS conference in Boston in December 2014.

[1] Novel Nano-Lychees for Theranostics of Cancer; Charles Spencer and Edna Purviance; Nature Matters-to-all (2015) 7  101-114

The death of Nanonymous

Before you worry: nobody died.
But it is really with great regret that I relay here Nanonymous’ decision “to retire the moniker” [see below]. Nanonymous was a clever handle and his/her comments have constituted important and enjoyable contributions to the stripy nanoparticles controversy. We first encountered Nanonymous in October 2013 at ChemBark commenting on the Response to ACS Nano Editorial on Reporting Misconduct where he/she defended post-publication peer review as a legitimate form of scientific discussion and urged authors to engage, notingIf this keeps up, “nano”science will get the reputation it deserves. What a shame.
Nanonymous became interested in the controversy; he/she was at the beginning undecided and looking at the scientific issues with an open mind, asking, still in the ChemBark thread

Raphael,
Do you have a response to (what I understand to be) the latest Stellacci article?
http://pubs.acs.org/doi/pdf/10.1021/la403546c

I mean, as someone who knows very little about this field I have a hard time understanding how one could demonstrate “stripiness” as a a scanning artifact…unless a bunch (more) people were wrong:
http://pubs.acs.org/doi/pdf/10.1021/la403546c

Since then Nanonymous made numerous contributions here and at PubPeer, with great scientific insights, many questions to me/Julian/Philip, forcing us to clarify our thinking further, as well as outstanding dissection of “unreg”[…] arguments and argumentative strategies.
Below is the last comment of Nanonymous, left this morning on the previous post.
nanonymous |

Raphael,
The latest post on pubpeer:

https://pubpeer.com/publications/58199FBEA31FB5755C750144088886#fb23615

falsely attributes authorship to my handle. It is certainly the same troll who has been spamming recently. My response isn’t showing up on pubpeer yet, I post it below. My, this is getting quite silly. I’m afraid the “encyclopedia dramatica” aspects will detract from the serious nature of the debate. I will probably sign up as registered peer to avoid this.
——————————————————————————————

The above post (Unreg February 1st, 2015 6:04pm UTC ) was not written by the real nanonymous (me). The poster (who is quite clearly associated with all the gish galloping) uses a deliberately sacchrin tone, perhaps for the purposes provoking a “spot the impostor” cliche:

http://tvtropes.org/pmwiki/pmwiki.php/Main/SpotTheImposter

The persistence of anonymous handles is a measure of how untrustworthy a forum can be, the fact that the above Unreg has approriated a handle that is not his/her own (this was already done once on the fakerapha site, there is no doubt that it is the unreg above is the same identity that is responsible for that site).

This is an attack against the utility of PubPeer since we must presume that the vast majority of comments that do not mention Russian brides, Nigerean banks or viagara are written in good faith and that no one would appropriate an established handle in good faith. This clearly demonstrates that we need to figure out ways around these attacks, we all knew this was inevitable. While the PubPeer admins certainly don’t have time, one could envision some sort of digital signature type scheme in PubPeer 2.0. (there are probably a few good reasons we don’t want mere registered usernames).

Suppose I don’t have much choice but to retire the moniker, any further posts labelled “nanonymous” are not written by me, the identity that assumed the (relatively) novel name during the course of learning about the stripy saga. I may consider joining as a regular Peer with my academic address, but will need to consider carefully for a bit.

I always thought that no one could possibly vest so much time and effort into trying to obfuscate facts about (what are to that vast majority of people) obscure subjects like nanoparticles but fakerapha/unreg/bionanochair/ProfSTM/gustav/weichen/etc. has proven me wrong. Learning that one is wrong anout something is perhaps the part that is most fun about internet discussions.

Identity theft: a new low in the stripy nanoparticles controversy

If you are familiar with the controversy and wonder why a PloS One comment thread disappeared today, skip directly to recent events.


Jackson et al, Nature Materials 2004

Jackson et al, Nature Materials, published 18 April 2004

In 2004, Alicia Jackson (now deputy director of the Biological Technologies office at DARPA), Jacob Myerson and Francesco Stellacci (now Constellium Professor at EPFL) mis-interpreted a common scanning probe microscopy artefact for a real structure. They submitted their results to Nature Materials, which published the work. This error was to become the starting point of a controversy which, more than 10 years later, is still unraveling.

Predrag Djuranovic

Predrag Djuranovic was a PhD student in Stellacci’s group at MIT

In 2005, concerns were raised by Predrag Djuranovic, then a student in Francesco Stellacci’s group. According to Lauren Wolf writing for Chemical & Engineering News last year:

he discussed his concerns with his department head at MIT […]. A few months later, an official investigation of Stellacci launched at MIT and Djuranovic left Stellacci’s research group. MIT closed its review three years later, in 2008, finding Stellacci not guilty of academic misconduct. The university did, however, suggest that Stellacci should do additional work to substantiate his 2004 findings.”

MIT's Building 10 and Great Dome; adapted from this  pic

MIT’s Building 10 and Great Dome; adapted from this pic

MIT was not convinced by the published data but instead of sharing those concerns with the scientific community, they kept them secret and Stellacci was asked to “substantiate” what he had published. This -trying to substantiate a structure which had been postulated to explain an experimental artefact- is what he has now been doing for more than 10 years… with over 30 articles published.

stripy nanoparticles revisited

Stripy Nanoparticles Revisited, published 23 Nov 2012

The controversy entered the public domain in 2012 with the publication of our article entitled Stripy Nanoparticles Revisited in Small: I was not aware of Predrag’s work but had come to the same conclusion. Francesco Stellacci published his response alongside our paper. He, and co-authors, also published several other articles around the same period. At the end of last year, Julian Stirling, and co-authors, published in PloS One an extensive Critical Assessment of the Evidence for Striped Nanoparticles which demonstrates that there is no evidence for the existence of the stripy nanoparticles. A response from Francesco Stellacci is still expected at PloS One.

Scientists interested in the details of the scientific arguments can refer to the articles linked to above (and references therein) as well as the blog posts and PubPeer discussions.

I wish this was all about science and differences of interpretation. I have touched previously on the re-use of figures in different articles. This eventually led to two corrections, one at Nature Materials and one at PNAS.  An EPFL investigation was triggered, opened, and, eventually, following the report of an international independent panel, closed. A third article, pretty much entirely based on data re-use, has not been corrected.

I decided to start blogging about the controversy after the publication of our article in 2012. I used the blog to present our paper, discuss the response (without having to wait three years), publish an invited post from Predrad Djuranovic, other experts (Philip Moriarty, Mathias Brust, and Quanmin Guo), and many other things . Other blogs and the post-publication peer review platform PubPeer also became important fora.

Unfortunately, just as the traditional literature has its flaws and can be gamed by scientists who do not respect the rules, so do these new modes of scientific communication. I have argued earlier that the stripy controversy was a window into the scientific process revealing the serious failures of self correction mechanisms. It is also a window into the frightening flaws of online post-publication peer review – I say this as a strong and determined supporter of post-publication peer review.

Those who believe in stripes have never engaged openly in the online debate (they have even vigorously complained that it was taking place, likening it to cyberbullying). Instead, we’ve had, in chronological order:

  • a troll posting over a period of two days a large number of comments on this blog… he eventually was identified as an Editor of the journal.
  • A fakerapha blog (that still exists) and a fakerapha twitter (the account has been removed) who impersonated me on the Chembark blog.
  • A text book case of Gish Gallop by “unreg” (a debating technique named after, and often used by, creationists) at PubPeer, It is to their immense credit that Julian Stirling, Philip Moriarty and several anonymous peers have engaged, repeatedly and honestly with “unreg”… until it became absolutely beyond doubt that this was no ordinary – if a little heated – discussion. Maybe best to let Nanonymous, an anonymous but insightful and regular commentator of the controversy describe that thread:

    Wow, the same thing again and again, in spite of being addressed again and again.

    and later

    […] it is interesting to reflect on the motivations for comments like these. They seem to come from an identity that either: 1)Does not understand basic scientific reasoning.2)Is vested in obfuscating some fairly straightforward arguments against clear and obvious errors in scientific work.Perhaps both. It could be just some random troll, but clear evidence that there are people acting in very bad faith out there.   Nanonymous

  • Possibly that thread also includes sockpuppetry with “unreg” and “ProfSTM/BioNanoChair” being the same person.

These things happened on blogs or at PubPeer, i.e. platforms which accept anonymous writing. It is hard to think of how they could be completely avoided while retaining anonymous comments.  The discussions of the costs/benefits of anonymous post-publication peer review is ongoing (I agree with Peer 1 in that thread).

In the last few weeks, the stripy nanoparticles discussion has moved to PloS One which enables comments directly on the article. Plos One has a clear policy requiring contributors to “unambiguously identify themselves with their first and last names, their geographic location, […] Any registered user who is found to have provided false name or location information will have their account suspended and any postings deleted.” In spite of this:

  • A “Gustav Dhror” posted over 10 comments (Gish Gallop again) about “Problems with Fig 4″. There is simply no doubt that Gustav is not a real person but instead a sockpuppet. There is no person with that name who has ever published any article on anything, let alone STM, nanoparticles or surface science. Even first year PhD students get their names listed on some website in academic institutions but  “Gustav Dhror” draws zero hits on Google, or more precisely two: the PloS One thread and a baby name website. After I posted a comment (and contacted PloS One) raising the issue of false identity, that thread stopped… but a few days later…
  • A “Dr Wei Chen” started another thread entitled “Multiple flaws with this paper“. The user profile linked to a real and existing Wei Chen, Professor at Suzhou Institute of Nano-Tech and Nano-Bionics in China. Given the context and previous events, I eventually took the liberty of calling Wei Chen to check whether he was “Wei Chen”, commenter at PloS One. And he was not. He had no idea. He had never posted anything at PloS One. Someone had stolen his identity to post these comments. Someone who wanted to discredit our paper (with flawed pseudo-scientific arguments), was not prepared to do it under their own name, but was prepare to steal someone else’s identity to do it.

PloS One has been contacted and has now removed the “Dr Wei Chen” thread. They have been in contact with “Gustav Dhror” and should post a statement regarding that thread tomorrow. In the paragraphs above, I have used links to WebArchives where you can see how the pages looked like this morning.