Well over a year after Simon Singh's 'libelous' article on Chiropractic was published; with Singh preparing to launch an appeal against Eady's ruling in the preliminary hearing of the result trial; and with the chiropractic profession under siege from a PR nightmare; the British Chiropractic Association have finally decided to release the evidence that they claim backs up their promotion of Chiropractic treatments. In doing so, they have misrepresented the evidence for the effectiveness of chiropractic in treating bed-wetting.
"In the spirit of wider scientific debate, and having taken appropriate professional advice, the BCA has decided that free speech would be best facilitated by releasing details of research that exists to support the claims which Dr. Singh stated were bogus"
So states the BCA press release. The research consists of 29 references. I have access to most of them, so I'll cover them here. Other bloggers will be covering various papers themselves in more detail, and I'll provide links to them all in this article - if you're a blogger and you've written about this, please let me know so I can link to you.
To avoid any accusations of shifting goalposts, here is the evidence that I want the BCA to present - a comprehensive selection of randomised, placebo-controlled, double-blind clinical trials producing significant results in favour of chiropractic producing a clinical effect beyond that of a placebo.
I've discussed the conflict between alternative medicine and scientific research before, and when alt-med types cite a list research papers, they tend to fall into the same sort of pattern: papers will be cherry picked; even so a number of papers won't in fact be relevant; some papers will not actually support the position the quack thinks they do; and others will be of poor quality that the quack is unable to distinguish due to a lack of understanding about e.g. clinical trials. I highlighted a good case study regarding acupuncture recently.
The same pattern is evident in the BCA's list. Of the 29 references, 1 is just the GCC's code of practice; 6 is an irrelevent paper about medical ethics; 8, 9, 10 and 17 are about osteopathy; 26 is a description of evidence-based medicine; 27, 28 and 29 are about NSAIDs. That's 10 down straight away, but what's interesting about these is that 6 of them are just attacks on conventional medicine. In other words, this is not a particularly comprehensive or focused review of the literature. It is far from the 'plethora' of evidence promised.
A further three papers, (12, 13 and 14) cover the safety of chiropractic, which has come under considerable criticism. Curiously, this brief selection ignores the numerous studies showing an increased risk from chiropractic. 14 isn't a study at all, 12 is considerably less bullish than the BCA suggest it is pointing to a significant number of side-effects "with a possible neurologic involvement", and 13 provided stronger support ("We found no evidence of excess risk of VBA stroke associated chiropractic care compared to primary care."), but should be taken in the context of the wider range of studies finding the opposite.
That brings us down to just sixteen papers remaining to deal with the efficacy of chiropractic. Even before we look at them, it's worth noting that it's a miserably small number. Of those 16 papers, 9 cover infantile colic, 1 looks at asthma, 2 study ear infections, 3 look at bed-wetting and 1 at a variety of conditions.
Professor David Colquhoun has reviewed the 9 infantile colic papers on his own blog, and the results are, well, poor. 2 had no control group, with the authors simply following 300 babies and finding that most eventually got better as you would expect even without treatment (without a control group you can't measure the effect of chiropractic itself). Ditto 3 and 25. 4 compares chiropractic with the use of dimethicone. Apparently, this is an ingredient in some over-the-counter remedies for colic, which are themselves unproven, so hardly the greatest thing to compare your treatment with - the results simply show that chiropractic is as good as another unproven remedy. Meanwhile, 5 brilliantly states that:
"The observed improvements are unlikely as a result of the specific effects of chiropractic spinal manipulation alone."
...suggesting that the effect obseved is largely placebo.18 and 20 are both reports on a couple of individual case studies, and therefore simply anecdote rather than the sort of evidence you'd get from a trial involving hundreds of patients. 19 simply compares one chiropractic treatment against another, and as Professor Colquhuon puts it, "shows that both are equally effective, or equally plausibly, both are equally ineffective." 24 isn't actually a study at all. In summary then, as Colquhuon says:
"The nine papers they cite for colic are truly pathetic. Not a single one of them amounts to anything that would be recognised as evidence in the real world."
So that leaves us with just seven papers remaining for the BCA to show that they can present some sort of coherent, clinical evidence: 1 on asthma, 2 on ear infections, 3 on bed-wetting and 1 on 'various'. 7, the asthma paper, is simply a letter to the editor and contains no actual evidence, so we're down to six.
Of the ear infection papers, 15 is an uncontrolled study that simply shows that children with ear infections tend to eventually get better but can't say if that's down to chiropractic. Unfortunately I can't get hold of 23 at the moment - if anyone else can I'd be grateful, but from the BCA description it looks rather like the same sort of thing again. Edit: Some helpful commenters found the abstract here, which confirms my suspicions.
So on to bed-wetting then, and at last the BCA are able to cite a proper study! 22 is a study of one person (an anecdote), and 21 is another uncontrolled "they eventually got better" study, but 16 is a meta-study by the Cochrane Collaboration no less, the gold-standard in medical research. The BCA quote this study as saying: "There was weak evidence to support the use of [chiropractic]." Could this be the evidence we're looking for?
No. Here's the quote in full, unedited by the BCA - I've underlined the bits the BCA quoted:
"There was weak evidence to support the use of hypnosis, psychotherapy, acupuncture and chiropractic but it was provided in each case by single small trials, some of dubious methodological rigour."
The BCA appear to have been rather selective in their quote-mining of the Cochrane review, cutting off the inconvenient part of the sentence. The review goes on to clearly state that, "each of these findings came from small single trials, and need to be verified in further trials," suggesting that the results are inconclusive, yet the BCA falsely suggest the trial supports their position. I simply cannot see how you could extract the quote the BCA took from that without being aware that you were seriously misrepresenting the research, by cutting off the qualifying clause of the sentence. If this was a genuine mistake, I expect the BCA to correct it.
In short then, the 'plethora' of evidence provided by the BCA is pathetically inadequate. How much of this is due to ignorance, delusion or dishonesty is hard to say; my experiences dealing with other alternative medicine practitioners in the past suggests that they often genuinely believe that evidence supports their case even when it clearly doesn't, and that there's a element of misunderstanding about that. I've no doubt that in their presentation of this 'evidence' the BCA are the same. However, when they quote a Cochrane review in such a way that they remove the part of a sentence that qualifies the claim they present, for me that is strongly suggests dishonesty. I just don't see how they could make such a convenient error by accident.
With numerous bloggers gearing up to provide their own, detailed analyses of the research presented, it will be interesting to see how the BCA respond.
------------------------------------------------------
Further Reading from Others:
Jack of Kent - General commentary and legal background.
Prof. Colquhuon - Detailed look at the nine colic papers.
Ministry of Truth - General review focusing on three of the colic papers.
Andy - Comment on the BCA statement.
Evidence Matters - Review of the paediatric asthma papers.
Gimpy - Review of the ear infection papers.
Phil Plait - An overview of the BCA statement and aftermath.
HolfordWatch - What would constitute good evidence?
Apgaylard - A more detailed look at the bed-wetting papers.
JDC - General comment on the BCA statement.
Think Logic - General comment on the BCA statement.
Cubik's Rube - A review of blog posts on the BCA statement.
Zeno (Think Humanism) - A look at the BCA's quote-mining of the GCC code.
Dr. Peter Lipson - Provides an overview and looks at the asthma studies.
Dr. Petra Boynton - Provides an overview of resources readers can use to evaluate the evidence for themselves.
Cochrane Reference:
(Full references can be seen listed on the BCA statement)
16. Cathryn MA Glazener, Jonathan HC Evans, & Daniel KL Cheuk (2009). Complementary and miscellaneous interventions for nocturnal enuresis in children Cochrane Database of Systematic Reviews
-------------------------------------------------
Follow me on Twitter! @mjrobbins
http://layscience.net/trackback/598
So - plethora of evidence boils down to quotemining?
What the holy zarquon is wrong with these people? How can they fail to see that they would be caught, and that they would end up looking dishonest, ekstremly silly or (most likely) both in the prosess.
Great work Martin,
How could they quote-mine so blatantly and expect not to have repercussions. Simply stunning.
Milt
"to my knowledge dimethicone - a key ingredient in silly putty - isn't used to treat colic"
Your knowledge is slightly lacking in this regard. 'Activated dimethicone', also known as Simeticone/simethicone, is a mixture of dimethicone and silicon dioxide, and is the 'active' ibgrediant of Infacol - and OTC 'remedy' for colic. As far as I know, it is no more beneficial than placebo.
Dimethicone (Polydimethylsiloxane) is used as an anti-foaming agent in industry, and its use as a carminative is on the basis that it encourages foams to coalesce and therefore facilitates burping and passing wind.
Dimethicone is an ingredient in some infant colic drops e.g. Dentinox. There is no evidence that these work either. In any event, as you point out, colic is something that babies grow out of.
Thanks for the clarifications on dimethicone, I've updated the article to reflect this. IANAD :)
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
Great spot on the quote-mine, Martin. I know I keep saying this, but you really couldn't make it up
They just don't get it, do they?
Excellent work!
When I saw that BCA list (the obligatory, obfuscatory data-dump), I couldn't imagine wanting to check each item. I have adopted a policy of only looking in medical literature, and JMPT is not in that category (despite protestations from chiros).
Nice work, MJ - I think it's time for the BCA to fire their PR department again...
BTW my little baby (born last October) had terrible colic for months, during which period we sang him a lot of Flemish children's songs. Then quite recently, just after he turned 6 months old, the colic went away. Clearly this is knockdown evidence that singing Flemish children's songs to babies can cure them of colic - and the only reason you and I aren't being told this is because the truth has been suppressed by big pharma...
I'm with Richard above - it's almost as if the PR boys edited the Cochrane finding to "enhance its public appeal".
I think you did well just to be able to quote from the damned document that someone apparently thought should be password protected apparently to prevent copying!!!
It's taken me half an hour and finally some Googling "copy text protected pdf os x" to find a way to copy from the damned thing without actually retyping it.
Epic Fale!*
*In my personal opinion.
Yeah, I couldn't believe that the text was protected, I had to quote stuff by hand which was incredibly tedious.
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
In my browser tab the title of your post reads:
A Review of The BCA's Evi...
I though for a crazy moment the next letter would be l.
Excellent work Martin, I'm staggered by the BCA response. It smells very strongly of bovine excrement to me.
Great summary Martin. Someone should teach these guys how to conduct a systematic literature review. Even their cherrypicked "hot" evidence is extremely tepid.
Surely they cannot be unaware of all the contrary evidence that exists against the use of chiropractic for these conditions?
Only two conclusions are possible. They are either liars, or stupid.
Sorry, make that 3. They could be both.
Good review. I've commented on the bedwetting evidence. Interesting that they've missed one of the studies commonly cited by their own members.
Interesting that "plethora" now means 16. I always thought that it meant: overabundance, superabundance, excess. First "bogus" now this.
Yet another indication that the BCA may indeed be dishonest or inexcusibly ignorant or both is the fact they un-cherry-picked the following paper, perhaps the only well-made study available on chiropractic treatment for infant colic, which concluded that "Chiropractic spinal manipulation is no more effective than placebo in the treatment of infantile colic."
Olafsdottir E, Forshei S, Fluge G, Markestad T. Randomised controlled trial of infantile colic treated with chiropractic spinal manipulation. Arch Dis Child. 2001 Feb;84(2):138-41.
http://tinyurl.com/nbszy3
OK. The sums are getting difficult now. Have I got this right. One International jot = 29 papers, but one International plethora = 16 papers? So, one plethora = 1.8125 jots? You guys don't make it easy... ;-)
I'm sure you could use this to prove that 2 + 2 = 0, but it just doesn't add up, does it?
BTW, brilliant work and like the new format.
Looked for #23, found the abstract here, but can't get the full paper as it seems my institution doesn't subscribe to the JCCP. Can't think why!
From the conclusions section of the abstract Andrew linked to: "This pilot study can now serve as a starting point from which the chiropractic profession can begin to examine its role in the treatment of children with otitis media."
Excellent job finding that. Aside from anything else, it looks like another paper by people who don't understand the need to control a trial....
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
Dangerous /= real.
As much as I would like to take credit for Thinking Is Real's blogpost, it does cover up my lack of coverage of the BCA shenanigans :)
Still, despite not blogging so much at the mo, it's fun watching chiropractic self-destruct.
T
Are the BCA learning PR from the Scientologists? In the late 90's, the Scientologists seemed to have their big guns aimed squarely at their feet. They learned the hard way that you take on the whole internet at you peril.
Have the BCA embraced "operation foot-bullet"? Do they actually think that this is good scientific evidence? And how on earth can they report the Cochrane review that way with a clear conscience.
It looks like Mr. Singh's lawyers just got a load of support for Justice Eady's definition of "bogus"...
Highly impressive work Martin.
"It looks like Mr. Singh's lawyers just got a load of support for Justice Eady's definition of "bogus"..."
Now, that is an interesting thought.
The most wonderful thing is that their largest study - Klougart N, Nilsson N and Jacobsen J (1989) - specifically states in the conclusion that the authors are intending to undertake a double-blind controlled clinical trial of chiropraxy and colic to asssess how much of the effect they saw was placebo. That was 20 years ago - the absence of the evidence strongly suggests the evidence of absence!
The BCA say "For those wishing to learn more about some of the available research about the effectiveness and safety of chriopractic treatment for children with the symptoms referred to by Dr Singh in his Guardian article, they can begin by looking at the following"
I guess that makes safety relevant to them (but not to Singh), but it makes bed wetting (nocturnal enuresis) for which they almost achieve "weak evidence" irrelevant because Singh did not mention it.
I have seen no assertion from the BCA that this is the list of papers they presented to Singh in 2008 or to the court this year.
You missed one!
Number 11 : Hawk C, Khorsan R, Lisi AJ, Ferrance RJ, Evans MW. Chiropractic care for non-musculoskeletal conditions: a systematic review with implications for whole systems research. J Altern Complement Med. 2007 Jun; 13(5) 47980
Great work.
I was suprised that the other 28 include not a jot of evidence that Chiropratic can help beyond placebo with any of the conditions they discuss. I had expected a little, poor evidence, but there isn't even that. What's in this last one ?
The only plausible explanation is the BCA were 100% confident that Simon would cave-in, and settle.
APGaylard,
"Plethora" also has an archaic meaning of "a morbid condition..."
Responding to myself, Evidence Matters discusses #11 (Hawk et al.) with regard to asthma.
http://evidencematters.org/2009/06/18/british-chiropractic-association-a...
I can only get the abstract.
As a review paper, its more complex. The BCA quote reasonably from the abstract, though its 5th conclusion is "Chiropractic investigators, practitioners, and funding agencies should increase their attention to observational designs."
Evidence Matters says "The results for paediatric asthma indicate that the results from the better quality trials do not report clinically significant findings of well validated and familiar outcomes for assessing improvement in the management or experience of asthma."
The review covers only three RCT's relating to asthma, only two of which include children. One of these identified by Evidence Matters (Balon et al) said "the addition of chiropractic spinal manipulation to usual medical care for four months had no effect on the control of childhood asthma" and explicitly observes placebo / study effects "consistent with anecdotal observations and uncontrolled studies of alternative approaches to the management of asthma" For the review to be evidence in favour of Chiropractic treatment of children with asthma, it would need the other RCT not to be Nilsen Et Al, that found "no clinically important or statistically significant differences were found between the active and sham chiropractic interventions on any of the main or secondary outcome measures" but since Nilsen Et Al is in pubmed, it should be the other child asthma paper in Hawk et al.
The BCA list also fails to mention two asthma papers (both negative) and a Cochrane review of that subject (again negative).
1) Balon J, Aker PD, Crowther ER, Danielson C, Cox PG, O'Shaughnessy D, et al. A comparison of active and simulated chiropractic manipulation as adjunctive treatment for childhood asthma. New England Journal of Medicine 1998;339:1013-20.
2) Nielsen NH, Bronfort G, Bendix T, Madsen F, Weeke B. Chronic asthma and chiropractic spinal manipulation: a randomized clinical trial. Clin Exp Allergy 1995 Jan;25(1):8008.
3)Hondras MA, Linde K, Jones AP. Manual therapy for asthma (Review). The Cochrane Collaboration. John Wiley & Sons, Ltd. 2008. www.thecochranelibrary.com
#11 fell through the gaps when I put this together, apologies. But as davidp says, you can find it covered at Evidence Matters - http://evidencematters.org/2009/06/18/british-chiropractic-association-a...
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
@Edzard Ernst "The BCA list also fails to mention two asthma papers (both negative) and a Cochrane review of that subject (again negative)."
Great to have you here :) Yes, what's really quite striking is that after all these years, on their own terms and with careful cherry-picking, the best that they could manage was to cobble together 17 papers that in fact still don't support their conclusions.
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
Excellent work, Martin!
This is the sort of article that makes me proud of the 'blogosphere'.
Cheers!
John.
Paper 12 is severely underpowered for detecting serious adverse events. Even assuming that all adverse events were reported, all it can conclude is that the risk is not greater that 6 per 100,000. This is an order of magnitude higher than the previous worst estimate. While the BCA says this paper demonstrates that chiropractic is safe, in reality all it proves is that chiropractic is not much more dangerous than previously believed.
The analysis of the papers is an interesting exercise and I agree that this is the type of debate that we should be having.
However, whilst the evidence presented is "weak" it is hard to argue that it is non-existent. Weak, preliminary, poor etc it may be but it exists in the public domain. To ofer an opinion that the evidence does not supprt a view in a scientific journal is one thing but to state as a matter of fact that the evidence does not exist (not a jot) in a newspaper is something else. Additionally. by having even preliminary evidence it rather supports the view that the BCA has not promoted bogus treatments (in the true sense of the word bogus).
I fear that the exercise that you have gone through, whislt maybe scientifically valid (I can't be sure0, will represent a nail in the coffin for Singh.
I was initially of a similar mind myself - it did appear that the BCA had at least "a jot" of evidence but the more that is revealed about this so-called evidence base, the more convinced I am that they have indeed been promoting bogus treatments and have been acting in an intentionally deceptive way.
The manner in which they have cobbled together their "plethora" of evidence with blatant cherry-picking, inclusion of irrelevant papers on anti-inflammatory drugs, articles about osteopathy, a paper that was only published after Singh's article and worst of all, a total misrepresentation (i.e. a lie) of the conclusions of a Cochrane review - coupled with the advice from several chiropractic organisations, including the BCA, informing their membership to take down websites and patient leaflets that make claims unsupported by scientific evidence - have all brought me to the conclusion that Singh was quite right to assert that the claims of chiropractic to successfully treat these childhood ailments were absolutely bogus, even in the interpretation that Eady made up.
Oxford English Dictionary: "bogus" - not genuine or true.
Chambers Dictionary: "bogus" - false; not genuine.
I suppose a lot now depends on what happens with Singh's appeal regarding Eady's made up definition of bogus but if the lawsuit does proceed beyond this point, I wonder how much longer the BCA leadership and the members will give it before they try to limit the damage and withdraw the action or actively press for some sort of face-saving settlement. After all, it transpires they never wanted it to go to court anyway - they assumed the mere threat of a trial would be enough to bully Singh into a retraction and apology.
The libel laws are their to protect against malicious and false untruths but this rather depends on the veracity of the supposed defamation being disproven. It appears increasingly likely that Singh could actually mount a perfectly reasonable defence of truth, even with the meaning of terms decided upon by Eady.
Time will tell...
It seems clear that the evidence represents more than a jot but less than a plethora. Unfortunately for Singh this will have significant legal implications (and financial). For the BCA this may prove embarrassing but will not alter their legal position.
To prove that the evidence presented can be used to support the view that the BCA promote bogus treatments will be impossible. Remember that the analysis of the evidence presented so far cannot be regarded as being entirely unbiased and that interpretaion of scientific evidence is always open to debate.
I do not believe that they have ever claimed that their evidence was evidence was definitive.
@Coleman "Unfortunately for Singh this will have significant legal implications (and financial)"
Nope - this will have zero implications of any kind for the current case as it stands.
@Coleman "the analysis of the evidence presented so far cannot be regarded as
being entirely unbiased and that interpretaion of scientific evidence
is always open to debate."
Nah, you appear to have confused opinion with fact. It's not my opinion that these studies are inadequate - it is easily confirmable fact. A trial with no adequate control or an insufficient sample size is not evidence, period.
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
I am sorry to spoil your party but you are wrong. If Singh had said that the evidence was weak or inadequate, and I believe this is what he meant to say, I would agree with you. However, he said that the evidence did not exist.
If you can demonstrate that these studies do not or did not exist I will again agree with you.
It may be possible to show that these studies are weak, preliminary etc but this will not make them go away.
you state
"A trial with no adequate control or an insufficient sample size is not evidence, period."
when you really mean that it is not good evidence. Period.
@Coleman - You seem to have misunderstood what this case is about. The phrase "not a jot of evidence" isn't RElevant to this trial. I'd recommend reading some of the legal summaries to acquaint yourself with the actual ruling made.
But as it happens, I would be happy to characterise the evidence presented by the BCA as "not a jot".
Martin is the editor of layscience.net.
Follow Me!
RSS | Twitter
From a lay legal perspective, I would imagine that the term evidence implies something that is strongly suggestive of a causative link or strong correlation. Otherwise absolutely anything could be regarded as "evidence".
Perhaps we need a ruling on the legal meaning of evidence?
And just to follow, remember the BCA said: "It has never been the BCA’s case that the evidence is overwhelmingly conclusive. It is the BCA’s case that there is good evidence"
"Good" evidence? That is patently untrue.
Thanks for posting this information. I wonder if they can do anything for defiant children.
The BMJ has recently highlighted this issue.
http://www.bmj.com/cgi/content/full/339/jul08_4/b2783
http://www.bmj.com/cgi/content/full/339/jul08_4/b2782
http://www.bmj.com/cgi/content/full/339/jul08_4/b2766
How many people actually know what chiropractic therapy is about? I think it's important to mention that this a treatment for prevention of mechanical disorders of the musculoskeletal system, especially the spine. I personally think this therapy has a lot of healing perspectives as it becomes more and more popular.
Darry, Colorado Springs Chiropractor
I would agree with you. However, he said that the evidence did not exist.health science school
If you can demonstrate that these studies do not or did not exist I will again agree with you.Engineering degree | Online education school
The libel laws are their to protect against malicious and false untruths but this rather depends on the veracity of the supposed defamation being disproven.Online Criminal Justice school | Online computer Science school
Why has not the BCA responded to this blog? There is obviously a lot to be clarified.
My wife is planning to attend a chiropractor who recently gave a talk at the golf club. This raised my suspicions as I felt it was unprofessional self promotion for personal economic gain (€60 per hour) and not for the general good. I trawled through the internet to check the qualifications of Dr X and found that he was accredited by a national chiropractic association on the back of a degree in psychology. I felt the title of 'Dr' was a little premature on the basis of his qualifications (doctor of chiropractic or DC seems to be the normal title). I was surprised that given the controversy surrounding the science value of chiropractic that the WHO has issued a standard on educational requirements. On reading it I saw that one of its three objectives was to safeguard the safety of the various techniques.