Smithsonian.com

Acupuncture Barbie suffers from chronic pain. Photo: Deborah Leigh

A broad-based study of the medical effects of acupuncture released recently argues that the practice of “inserting needles at various places on the body to stimulate so-called acupoints” may have a beneficial effect for those dealing with chronic pain, such as migraines or arthritis, reports The New York Times.

The research, lead by Andrew Vickers with the Memorial Sloan-Kettering Cancer Center, pooled together and re-analyzed the results of 29 previous scientific studies on acupuncture, trials which represented more than 18,000 patients. The scientists found that acupuncture has a small mitigating effect on the level of pain felt by sufferers of chronic pain. At Science-Based Medicine, Steven Novella summarizes the results: When compared against people who were given no treatment for their pain, acupuncture patients pain levels were 35% lower. When compared against people who were given “sham acupuncture,” where fake needles are used which don’t actually penetrate the skin, acupuncture patients saw only a 5% reduction in pain levels.

Though the findings of reduced pain following acupuncture are statistically significant—the pain reduction isn’t the effect of random chance—doctor and pseudonymous blogger Orac writes at ScienceBlogs that the amount of pain reduction Vickers and his team found is essentially irrelevant.

What Vickers et al are arguing is that a change of 5 on a 0-100 pain scale… a subjective scale, is noticeable by patients. It’s probably not. There is a concept referred to as “minimally clinically important difference” (MCID) defined as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate…a change in the patient’s management.”

…

Indeed, Vickers et al labor mightily to try to convince readers that this tiny effect, if it exists, is not just statistically significant, but clinically significant. They doth protest too much, methinks.

Orac also sees a problem with the scientists’ research itself. He says the study would have been swayed towards finding a real medical effect of acupuncture because of something known as publication bias. There is a real trend in science that many researchers only actually bother to publish the research that works out. So, studies that draw on the published literature to make new claims, like this current acupuncture study, are drawing from a biased sample.

Steven Novella with Science-Based Medicine agrees:

The comparison between true acupuncture and sham acupuncture shows only a small difference, which is likely not clinically significant or perceptible. More importantly, this small difference is well within the degree of bias and noise that are inherent to clinical trials. Researcher bias, publication bias, outlying effects, and researcher degrees of freedom are more than enough to explain such a small difference. In other words – this data is insufficient to reject the null hypothesis, even if we don’t consider the high implausibility of acupuncture.

Novella ends his analysis of the study by saying,

The Vickers acupuncture meta-analysis, despite the authors’ claims, does not reveal anything new about the acupuncture literature, and does not provide support for use of acupuncture as a legitimate medical intervention. The data show that there is a large difference in outcome when an unblinded comparison is made between treatment and no treatment – an unsurprising result that is of no clinical relevance and says nothing about acupuncture itself.

More from Smithsonian.com:
Electrocybertronics