This is a guest post by James Elliott, manager of the Flow Cytometry Facility at the MRC Institute (LMS) in Hammersmith.
I thought it may be useful to add to the discussion about SmartFlares, their marketing and the difficulties in disseminating negative results by passing on my own experience.
We tested the system back in 2013. Sorting primary murine T cells and thymocytes on the basis of RNA expression was perhaps of most immediate interest, but of course there were countless potential applications.
The Merck Millipore rep advised us that the caveat we should be aware of in using SmartFlares was that the particles are taken up by endocytosis and that not all cells possess the machinery to allow this. Indeed, he mentioned data he had seen that only around 20% of T cells take up probes. This was puzzling as it suggested either a specific subset of endocytosis-competent cells or alternatively that uptake by T cells was broad but weak, such that only 20% of cells fell into a positive, above background gate. This in itself seemed a potentially interesting question.
To address the usefulness of SmartFlares in primary T cells (and some lymphocyte lines we had in culture) it was agreed with the rep that the sensible first step was buy positive (an ‘Uptake’ probe where fluorescence is always ‘on’ even in the absence of specific RNA) and negative (scrambled, ‘fluorescence off’) controls.
Everyone rightly comments on the extremely high price of the reagents and though we were given a discount, it remained an expensive look-see experiment.
It was useful that on the day we tried out the probes we were lucky enough to have someone with us from Merck who we like and trust to oversee what we were doing – he could vouch for the fact that we did the experiments correctly. We looked for probe uptake both on a flow cytometer and an Imagestream imaging cytometer.
Whilst we had expected lymphocytes to take up the probes poorly, in fact the big problem we had was that whilst all, or nearly all cells took up the probes, the signal from cells given the scrambled probe – notionally ‘always off’ was just as high and in most cases a little higher than that with the positive control ‘Uptake’ probe. Both showed a marked, log shift in fluorescence.
So – big problems! Why was the scrambled probe, which should have been dim or ‘fluorescence off’ giving us such a high signal? Indeed, if anything our negative control was brighter than the positive.
The rep consulted with the technical team, who were quick to point out that more meaningful comparison would have been between a scrambled and housekeeping probe (the Uptake probe merely being useful to show a qualitative result), yet this seemed to me to fudge the issue: first, surely the uptake and scrambled probes should be roughly comparable in number of molecules of fluorochrome attached or the uptake control would be of limited value – it would give a yes/no answer as to whether the cells would take up probe, but would give little clue as to efficiency. Second, the strategy of validating the system had been agreed with the rep. It was not great to then come back and say that actually this was not a good test after all. Third, and most importantly, a system in which the negative control (‘fluorescence off’!) gives a log shift in fluorescence is likely to be almost completely useless! The background would be far too high for all but the most abundant markers.
In addition to which it hardly inspired confidence that the company seemed to have validated the system very poorly – why else would they be giving a vague suggestion that maybe 20% of T cells take up probe, when in our careful (and observed) hands, they did so rather efficiently. Interestingly, in this respect I later read on a cytometry forum that, according to one US user, the company had been very up front from the beginning that primary lymphocytes don’t take up the probes. This was doubly untrue – lymphocytes do take up the probes and in the UK anyway, we were not told primary lymphocytes didn’t take up probes – the rep thought 20% of T cells did so, but was unsure about the data. Again, I was left with the impression of a poorly validated system sold by reps who were largely in the dark.
The most likely explanation for our results in follow up discussions with the company was that scrambled probe had degraded intracellularly and that this can happen in a cell type-specific way. This would mean that there would be a cell type-specific optimum time window where there was a satisfactory balance between cleavage by target RNAs and non-specific cleavage. Of course we had followed the instruction we were given at the time, but now it appeared these probably weren’t correct for our (hardly esoteric!) cells.
The suggestion was therefore that as many controls as possible would be wise.
Clearly this had become completely untenable as a system – we would have to buy hugely expensive probes and – if they worked at all, which we still didn’t know – would have to do a lot of work to establish not only the usual factors such as concentration, but also timing. And how narrow might the optimal time window be where specificity was apparent? An hour? Less? And background from non-specific signal from degrading probes would be likely to be (at least in the cells we were most interested in) a major problem for any RNA that wasn’t highly expressed.
We decided to cut our losses. I applaud those who can follow up and publish negative data that will be useful to the scientific community, but it seemed likely that for us this would end up far too expensive financially and in time and effort – quite possibly simply to show that the system might just about work, but not in any way that would be practically useful.