How many people are using the #SmartFlares? Freedom of Information request provide insights

Quick summary of previous episodes for those who have not been following the saga: Chad Mirkin’s group developed a few years ago a technology to detect mRNAs in live cells, the nano-flares. That technology is currently commercialised by Merck under the name smartflares. For a number of reasons (detailed here), I was unconvinced by the publications. We bought the smartflares, studied their uptake in cells as well as their fluorescent signal and concluded that they do not (and in fact cannot) report on mRNAs levels. We published our results as well as all of the raw data

This question – how many people are using the SmartFlares? – is interesting because surely, if a multinational company such as Merck develops, advertises and sells products, to scientists worldwide, these products have to work. As Chad Mirkin himself said today at the ACS National Meeting in Philadelphia “Ultimate measure of impact is how many people are using your technologies“.

So, we must be wrong. SmartFlares must work.

But our data say otherwise, so what is going on?

One hint is the very low number of publications using the smartflares and the fact that some of those are not independent investigations. This, however does not tell us how many groups in the world are using the smartflares.

Here is an hypothesis: maybe lots of groups worldwide are spending public money on probes that don’t work… and then don’t report the results since the probes don’t work. That hypothesis is not as far fetched as it may seem: it is called negative bias in science publishing and it is one of the causes of the reproducibility crisis.

To test this hypothesis, we would need to know how many research groups worldwide have bought the smartflares, an information that I suspected Merck was not going to volunteer. So, instead, I made Freedom of Information requests to (nearly) all UK research intensive universities (the Russell group) asking whether they had evidence of smartflare purchase orders.

Some Universities (6) declined because it would have been too much work to retrieve the information but most (14) obliged. The detailed results are available here. They show that a minimum of 76 different purchases were made between the launch of the product and June 2016. The money spent is £38k representing 0.0013% of these UK universities research income. As far as I can see, none has resulted in a publication so far.

All I can say is that these data do not falsify our hypothesis.

And if after reading this, you are still unconvinced of the need to publish negative data, check the upturnedmicroscope cartoon (warning: scene of violence).

 

 

 

Advertisements

2 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s