What Are Evidence Based Medicine Zealots?
A co-author on the medical research we publish about stem cells sent me this paper this morning (see below). It’s from 2003, but it’s as relevant today as ever. Why? The medical research industrial complex (made up of universities, big pharma, and the FDA) has abandoned common sense by convincing physicians, science writers, and the public to buy the concept that any medical treatment or intervention needs randomized controlled trials (RCTs) before it can be used clinically. First, read the abstract below:
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials
BMJ 2003; 327 doi: http://dx.doi.org/10.1136/bmj.327.7429.1459 (Published 18 December 2003) Cite this as: BMJ 2003;327:1459
Objectives To determine whether parachutes are effective in preventing major trauma related to gravitational challenge.
Design Systematic review of randomised controlled trials.
Data sources: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists.
Study selection: Studies showing the effects of using a parachute during free fall.
Main outcome measure Death or major trauma, defined as an injury severity score > 15.
Results We were unable to identify any randomised controlled trials of parachute intervention.
Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.
While this paper out of Cambridge is Monty Python funny (I’ll post the even funnier acknowledgements below), it makes a very important point. Most of what’s done in medicine today is not evidence based in that there are no randomized controlled trials to support that the treatment is effective. While on the one hand, the lofty goal to make sure that we can test as many things as possible using this high bar is admirable and necessary, approaching this issue with a religious zeal is about as smart as approaching any problem in life that way-you lose sight of the goal. There are students, scientists, and members of the public that have taken this “RCT mantra” to the extreme. They often indirectly belittle sick patients who want to try non proven treatments, yet none of them are in this position of being ill (they are largely themselves the “worried well”).
How is the RCT system flawed? First, as above, a lofty goal to improve healthcare quality is helpful when it’s a goal and disabling when it becomes a standard that must be used without any thought given to common sense. As an example, using our current pace of medical research, all of the existing treatments in medicine won’t be tested this way in my nor your lifetime! What should we do with all of these patients that need to be treated? Perhaps the problem can be solved (as these Cambridge researchers suggest) by taking RCT zealots and enrolling them in a parachute trial? Second, RCTs designed and paid for by drug companies are at best poor surrogates for an objective test of a therapy. They are designed to show the therapy in the best possible light, often excluding many patients who would need the therapy in the real world. Finally, tight inclusion and exclusion criteria (who can and can’t participate in a study) often make the applicability of the research so narrow, that the unique patient sitting across from a doctor asking for help is not at all like the patients that were in the research study. This has the net effect of making the research study worthless in providing guidance on how best to help the patient. Finally, and perhaps humorous or sad, is that fact that we have no high level evidence that the “Evidence Based Medicine” movement actually improves real world healthcare outcomes or costs!
This town versus gown argument over how best to develop new medical technologies has been waging for awhile now. For example, while the U.S. research industrial complex has embraced the use of placebo trials where patients get fake treatment, the international community through the Declaration of Helsinki still frowns on them. The reasons are obvious, sick people often sacrifice their own lives on the altar of science. This has led to the kinder and gentler cross over study design (where everyone eventually gets treated). In addition, the tried and true medical discovery system that’s run by physicians who observe new therapies working in patients is the Yin to the Yang of RCTs. This type of new therapy invention process has lead to almost all modern surgeries and trauma care and often works better than the university lead process. For example, a simple gastric bypass surgery recently decimated all drug options to treat type 2 diabetes.
The upshot? Am I a research hater? No, on the contrary I have published as much research on the use of stem cells to treat orthopedic injuries as almost anyone on earth. We have 5 studies currently in the peer review process that should get published in the next 6 months and four RCTs that we’re personally funding. Having said that, I take evidence based medicine with the grain of salt that it was meant to be digested with and avoid swallowing hook, line, and sinker like a religious zealot.
Acknowledgements: Contributors GCSS had the original idea. JPP tried to talk him out of it. JPP did the first literature search but GCSS lost it. GCSS drafted the manuscript but JPP deleted all the best jokes. GCSS is the guarantor, and JPP says it serves him right.