Last week, we saw the House of Commons' Science and Technology Select Committee question 'expert' witnesses in a review of the evidence for homeopathy. In the aftermath of what was a fairly dismal performance by homeopathy advocates, the BHA have issued the following, dire press release:
Robert Mathie, the BHA's Research Development Adviser, told the committee that there is evidence that homeopathy has an effect greater than placebo. You can read the BHA's written submission to the committee here.
Robert comments: "A key point that the select committee has perhaps not fully taken on board is that the total number of randomised controlled trials (RCTs) in homeopathy is minuscule, especially when compared with research activity in most areas of conventional medicine. The fact that there are just 87 placebo-controlled (or "efficacy") trials of homeopathic medicines in the total scientific literature, but that 37 of them have reported positive findings, is something that must be properly recognised. That this evidence exists seems to have escaped the notice of some commentators."
Now, there are several points at which this statement goes horribly wrong. I'll take them in turn:
1: "A key point that the select committee has perhaps not fully taken on board is that the total number of randomised controlled trials (RCTs) in homeopathy is minuscule..."
So the BCA BHA (Freudian slip there! - Ed) are basically saying that the amount of good-quality evidence for homeopathy is 'miniscule'. It's a fact that I personally am keen to see the committee take on board, although I struggle to see how it helps the BHA's members.
But it gets even better. The BHA are complaining that this point - that there haven't been many RCTs of homeopathy - may not have been 'taken on board' by the select committee. Well, Robert Mathie was in the hearing. He did not make this point as far as I saw (although to be fair he did speak of the need for more research, particularly into cost effectiveness), and when he mentioned RCT's it was to say things like:
"... in fact many [homeopathic medicines with 'molecular content'] are not not just sugar pills, many have been investigated in randomized controlled trials..."
At least Mathie was more restrained than his homeopathic colleagues. I'm sure the British Homeopathic Association will be rushing to condemn homeopaths like Robert Wilson, who also appeared, for overstating the evidence and undermining their message with irresponsible comments like:
"It definitely does work [outside the placebo effect]. [...] We have many trials that show a strong efficaciousness for homeopathy."
2: "The fact that there are just 87 placebo-controlled (or "efficacy") trials of homeopathic medicines in the total scientific literature..."
Apropos of nothing, here is a line from the methodology of their written evidence to the committee: "One hundred and twenty out of the total of 142 RCTs (85%) were placebo controlled."
Which is it? 120 or 87? And given that this is a multi-million pound industry prescribing medicine for the best part of two centuries, why have these people not made more of an attempt to test the products they're making profit from? Would you allow a pharmaceutical firm to market pills without rigorous trials?
3: "there are just 87 placebo-controlled (or "efficacy") trials of homeopathic medicines in the total scientific literature, but that 37 of them have reported positive findings"
Let's just ignore the fact that 52 out of 120 in their written evidence has magically turned into 37 out of 87, since to be fair it isn't that significant statistically (we're at 40-odd percent either way). And let's ignore the incredibly vague use of "positive findings."
Their written evidence makes the point (paragraph 3.1.3) that due to poor methodology, the papers included in their review can't be used to demonstrate clinical significance. In spite of this liberal approach to including poor trials, only 37 out of 87 trials returned a positive result. For the maths whizzes out there, that mean that 50 of the trials didn't return a positive result - they were either inconclusive or definitely negative.
So in a glorious brain fart of a press release the BHA have suggested (admitted?) that the evidence for homeopathy is 'miniscule', suggested that other homeopaths giving evidence to the select committee were being misleading, made a statement which their own written evidence contradicts, and pointed out that the majority of RCTs on homeopathy do not produce positive results.
For once, I find myself in agreement with the BHA. I hope that the select committee do take these points on board.
http://layscience.net/trackback/825
“Their written evidence makes the point that due to poor methodology, the papers included in their review can't be used to demonstrate clinical significance.”
Which is an irrelevant point anyway since RCTs of homeopathy cannot be used to demonstrate the clinical efficacy of homeopathy no matter how (apparently) good the methodology. To put it another way, the rational interpretation of a positive result from an apparently good large homeopathy RCT is that it is evidence that the methodology was flawed.
I let this point drop in the recent BS.net thread because you and Dead Badger convinced me it wasn't all that important and that it's probably okay to let the homeopaths carry on doing their futile pseudoscientific RCTs - often even going so far as to join them in their pseudoscience by taking their 'evidence' seriously. But I've changed my mind: I don't think it's ethical to conceal the truth from them - or fail to make it explicit - however desperately they crave scientific recognition and however much it'll upset them.
Would it not have been easier for them to just shoot themselves in the foot?
A few months ago I received an email from this committee asking what areas they should seek evidence on. On 5th August I emailed suggesting homeopathy
"What [should be investigated]?
May I suggest you consider complementary therapies and whether the National Health Service should fund these? Please note that this is a separate matter to licensing of homeopathic products by the MHRA, as it covers treatments such as acupuncture, osteopathy and chiropractic, none of which (I suspect) are "products" covered by your previous investigation.
Why?
It's estimated that the NHS spends a few hundred million pounds each year on complementary medicine. We should be confident that this money is being spent well.
Who [should give evidence]?
Dr Ben Goldacre
Professor David Bellamy OBE (who is a patron of the British Homeopathic Association)
Ellen Raphael, UK Director of Sense About Science, who can talk about evidence-based medicine
Dr Kieran Breen, Director of Research, Parkinson's Disease Society
Mandy Payne, Medical Journalist"
So (while I am sure that others must have given out similar suggestions) I feel quite pleased at what happened.
Kicking in the heads of atheists one at a time...
http://nostradamus-america.atspace.com/
PZ, I thought the Morris Police Department was going to save you from the wrath of God...
Looking at RCTs is a bit of a red herring anyway. With enough time you are going to get a p=0.05 result eventually (especially with flaky methodology).
It would be much better if any homeopaths could present a plausible explanation as to how their pills work (beyond placebo, talking therapy etc).
The point is that homeopaths don't really want to conduct appropriate research using validated scientific methodology. They know they will be found out as scammers, and prefer to keep things on a "smoke and mirrors" level.
Well, Robert Mathie - the guy quoted in the statement and the man who compiled their written evidence submission - has come out elsewhere talking some vague guff about the need to evaluate whether placebo-controlled trials are really the best way to go for evaluating the 'holistic' impact of homeopathy in the real world.
There's a lot of double-think right the way through their approach to evidence.
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
Many? So some of them are? Which ones?
Another interesting comment from Mathie, in view of the frequent objections from homoeopathic apologists that RCTs are not a valid test of homoeopathy because the treatment needs to be individualised (see here, for example), in his answer to Q145:
"There are 3,000 homeopathic medicines in the pharmacopoeia. We need to understand the efficacy of each, ideally, but let us do it gradually with those specific medicines where they are frequently used and have been researched in efficacy research and can become gradually evidence-based contributions to homeopathy as a system of care."
Testing "specific medicines" seems to be precisely what the homoeopaths object to.
Funny that while they repeatedly cite the Linde et al. 1997 meta-analysis, they never cite Linde and Melchart's 1998 review of trials of individualised homoeopathy.