Being a Research Leader – Correcting Bad Conclusions
Regenexx has been a leader in orthobiologics research ever since our first publication on mesenchymal stem cells in 2006. Most of the time that means that we publish our research findings, but sometimes that also means correcting bad research that gets published. Given that we just had the second type of publication in the prestigious medical journal Arthroscopy, let’s dive deeper into what that was about.
Our Research History
There is no group of clinics in the US or in the world that has published more original research on orthobiologics than Regenexx. How about the famous sports medicine clinics? Nope, not even a distant third. How about universities? Nope, not even a close second. In fact, there’s only one author that can give us a serious run for our money and that’s orthopedic surgeon Philippe Herigou in France.
Our first paper on mesenchymal stem cell use in orthopedics was published in 2006, when most medical providers didn’t know what a mesenchymal stem cell was. In fact, at that point, we were the only clinic on earth using culture-expanded stem cells in orthopedics. Since then, we’ve had dozens of peer-reviewed publications on the topic of orthobiologics appear in press.
Original Research vs. Correcting the Record
While much of what we publish is original research from our lab or about a specific clinical study, every once in a while, a paper appears in the research that’s published by another group of scientists with such a bizarre conclusion that it needs to be corrected. That happens when the authors of the paper can’t support the conclusion. Meaning that every research paper begins with collected data and then analysis of that data and then moves toward a conclusion supported by the data. However, sometimes, the conclusion isn’t supported by the data analysis. While that error is supposed to be caught in the peer review process, sometimes things slip through. Often when this happens I’ll just blog on the paper, but sometimes it needs a more formal written rebuttal.
Intra-articular Mesenchymal Stromal Cell Injections Are No Different From Placebo in the Treatment of Knee Osteoarthritis: A Systematic Review and Meta-analysis of Randomized Controlled Trials
Several months back I was contacted by colleagues who read a paper by a Chinese researcher named Wenli Dai (1). The paper was a meta-analysis, which means they took data from a number of different studies about using stem cells to treat knee arthritis and then pooled all of that information and analyzed it as if it were one study. Somehow, they reached the conclusion that using stem cells to treat knee arthritis didn’t work. Or did they?
When reading the Dai, et al paper in the journal Arthroscopy, a few HUGE issues jumped out. The authors:
- Lumped together the results of studies about apples and oranges
- Misidentified knee treatments as placebos
- Excluded research they were supposed to include
- Somehow managing to interpret 3 in 4 studies that showed improvement as an overall negative result
While getting into all of these issues in our published rebuttal would be intense (2), let’s dive into the first and most glaring error.
Apples and Oranges
As reviewed above, a meta-analysis is where you take many studies about the same treatment used in the same patients and you pull out the data and reanalyze it as if it were a single study. However, that means that there is one hard and fast rule. You can’t group data from studies that used different treatments.
Dai et al’s fatal error here can be your educational gain, as the paper shows that the authors knew less about orthobiologics than did most of my patients and people that read this blog. Why do I say that? They grouped together vastly different orthobiologics where the term “stem cell” has been used to describe the therapy and considered them all the same. This is what they grouped together:
- A low cell count bone marrow concentrate with PPP
- Stromal vascular fraction from fat
- Culture expanded bone marrow MSCs
- Culture expanded adipose MSCs
- Culture expanded umbilical cord MSCs
These treatments are VASTLY different.
First, we have tissue source. Cells taken from the bone marrow are different than those taken from fat which are again different than cells taken from umbilical cord tissue (3,4). This is like comparing apples to oranges to bananas because all are classified as fruits. Just tasting each you know that they are drastically different items.
Next, we have how the cells were handled. Bone marrow concentrate is centrifuged and then used. Stromal vascular fraction is digested with an enzyme and then centrifuged and then used. Culture-expanded cells are grown in an incubator for weeks before use, being handled and purified many times. Again, grouping these treatments as the same thing isn’t possible.
How Did the Authors and the Journal Make these Obvious Errors?
As you can see, the only thing the above treatments have in common is that someone, at some time, called them a “stem cell” therapy. So how did the authors and the journal get it so wrong? Basic education in regenerative medicine is VERY poor. The reality has to be that none of these smart people knew that these therapies could never be grouped. That’s likely because anything they learned about regenerative medicine and orthobiologics was at a weekend course and not part of their basic medical school, residency, or fellowship educational training.
The upshot? Now you see why we needed to get a published rebuttal to this misinformed meta-analysis. You also now see that the stem cell wild west isn’t just in the world of providers offering treatments, but sometimes extends into the smart people performing analyses and publishing research.
(1) Dai W. Leng X. Wang J. et al. Intra-articular mesenchymal stromal cell injections are no different from placebo in the treatment of knee osteoarthritis: A systematic review and meta-analysis of randomized controlled trials.
Arthroscopy. 2021; 37: 340-358
(2) Centeno C, Burnham R, Rowan P, Le A, Malanga G, Freeman M. Regarding “Intra-articular Mesenchymal Stromal Cell Injections Are No Different From Placebo in the Treatment of Knee Osteoarthritis: A Systematic Review and Meta-analysis of Randomized Controlled Trials”. Arthroscopy. 2021 May;37(5):1361-1362. doi: 10.1016/j.arthro.2021.02.026. PMID: 33896482.
(3) Mohamed-Ahmed S., Fristad I., Lie S.A., et al. Adipose-derived and bone marrow mesenchymal stem cells: A donor-matched comparison.
Stem Cell Res Ther. 2018; 9: 168
(4) Xu L. Liu Y., Sun Y., et al. Tissue source determines the differentiation potentials of mesenchymal stem cells: A comparative study of human mesenchymal stem cells from bone marrow and adipose tissue. Stem Cell Res Ther. 2017; 8: 275