(Publisher's Note: A research project reviewed by HealthMall.com deals with a review of RCTs on manual therapy for asthma. The article may be seen at http://www.healthmall.com/mailarticle.cfm?type=article&id=409. The following is a response from  Dr. Anthony L. Rosner, FCER Director of Research and Education.)

Update: The above-referenced article is no longer available from the publisher.

Response to Manual Therapy for Asthma Review¹

Hondras' recently published systematic review of randomized clinical trials¹ addressed to manual therapy represents a sincere effort to summarize those investigations in what is commonly regarded as the gold standard of clinical research. That said, however, one has to remain particularly vigilant against accepting randomized clinical trials at face value, particularly in those instances involving physical interventions, in which the complete blinding of practitioners [and most likely patients as well] in the traditional RCT design is all but impossible.

Consider the following pieces of evidence which suggest that randomized clinical trials can be misinterpreted or even corrupted:

1.   One of the randomized clinical trials regarding the use of chiropractic in managing asthma which was published in The New England Journal of Medicine² and most likely accepted as one of two trials examining chiropractic in Hondra's study is fatally flawed by the following considerations, presented in detail as an object lesson so as to more fully acquaint the reader with the problems of properly designing and interpreting a randomized clinical trial:

a.   Lack of validity of the sham procedure:

With over 20 commonly used techniques and 100 procedures overall described for chiropractic, there is understandably a great deal of controversy as to what constitutes a proper sham or mimic treatment. Furthermore, with applications to no less than three regions of the patient having been described in the Balon study [gluteal, scapular, and cranial], there is a high probability that the sham procedure is invasive and overlaps to a large extent with the maneuvers chosen with the actual manipulation. This suspicion is strongly supported by a recently published clinical trial in a leading pediatrics journal to the effect that massage compared to a noncontact placebo produces significant improvements in lung functional tests, asthma symptoms, and stress indicators in two separate cohorts of children.³

The problem of sham procedures in the Balon study is compounded by the fact that nearly a dozen chiropractors had to be trained to perform such a procedure with no indication of standardization. The effect of all this is to minimize or obscure the therapeutic effect that might be observed in an actual adjustment.

The fact that all patients have been medicated may be necessary from an ethical point of view, but it would be expected to mask the beneficial effects that might have been observed from spinal manipulation. The reader must be cognizant of the fact that this trial reports little or no benefits in addition to standard medication.

c.   Vagueness of interaction with the patient:

The nature of personal interaction with the patient is ill-defined at best, dubious at worst No indication is given as to how the practitioner such as might be seen in the clinic interacts with the patient except to administer a satisfaction questionnaire. This leads to the additional intrigue as to how eligible patients as young as 7 years of age are to competently answer such questions as those pertaining to "feeling at ease, the skill and the ability of the chiropractor, and overall quality of care" that were administered in the trial.

d.   Improvements over baseline values in both treatments:

The fact that there was significant improvement by intervening with the patients is demonstrated by the declines at 2 months and 4 months of both daytime symptom scores and the number of puffs per day of a beta-agnonist, in addition to small increases of peak expiratory flow rates and pediatric quality of life scores in both groups. Such is to suggest that even in this trial there was significant improvement in the patients enrolled. What is not clear is which form(s) of intervention [global and/or manual] elicited responses. What is not shown by the data is that contact with the chiropractor fails to provide additional benefits in addition to medication in the management of childhood asthma. It is simply an outmoded concept to assume that simply the presence or absence of cavitation constitutes the difference between chiropractic and no treatment.

e.   Lack of complete representation of global symptoms:

Given the fact that the human diurnal cycle lasts 24 hours, I am mystified by the lack of data representing nighttime symptoms. In effect, we have been shown only half the complete picture in this study.

Balon's study reflects the challenges and problems of properly designing a clinical trial which involves more than simply ingesting pills which can be fully masked. In the application of manual therapies, practitioners cannot be blinded. The result in single-blind clinical investigations such as represented by the Balon study is that the authors rely solely upon the patients' incorrect answers to validate their ignorance as to what type of treatment they received. There is no allowance for the nuances of emotion or expectations of the therapist which are conveyed to the patient.

f.    Possible masking of effects by Type II error:

Even with its questionable design, the Balon study appears to demonstrate a tendency toward improvement in activity, symptoms, emotions, and overall quality of life in the manipulated as compared to the sham treated group. Statistical significance could not be demonstrated, however, presumably because the experimental groups employed in the trial were too small. Obscuring of significant results by improper experimental design or interpretation is known as a Type II error.

2.   Another highly visible clinical trial comparing three interventions in the management of acute low-back pain⁴ suffered from poor design⁵ and inappropriate statistical procedures⁶. Worse, it implied that a single intervention represented chiropractic care such that its clinical relevance was highly questionable.

Indeed, the Royal College of General Practitioners in a very recent systematic review of the literature designed to update the CSAG Guidelines of the United Kingdom⁷ has concluded that this trial neither adds nor detracts from the evidence base regarding appropriate interventions for low-back pain.⁸

3.   A meta-analysis has shown that contrasting interpretations can be obtained, depending upon which of 25 scales used to distinguish between high- and low-quality trials is actually employed.⁹

4.   A review of clinical trials comparing two antifungal agents has indicated that the apparent advantages of one of the instruments could have been obtained by manipulations of the design of most of the trials, in which the competing agent was inappropriately administered.¹⁰

5.   The weight of evidence produced by clinical trials may be overcalculated due to the fact that the clinical trials are overrepresented as duplicate, "sausage" publications by the same authors.^11-14

6.   Methodological scores attached to clinical trials create a misleading profile of high- and low-quality studies if they place too much emphasis upon sham procedures which we already know will seriously compromise controlled studies involving physical methods such as spinal manipulation if they are not true placebos. In other instances, the mere utterance of such terms as "blinded" or "randomized" in the title of the paper cited may be sufficient to glean points in the rating of clinical trials--even though such terms are never defined or qualified. The proper remedy in this instance would be to demote the trial ratings if such terms are inappropriately used.⁹

The point to realize here is that RCTs are subject to misinterpretation and outright abuse. Their generalization from a fastidious, defined laboratory setting is problematical. It is sometimes forgotten that the source of randomized clinical trials remains the sound, well-documented observations in the clinical setting. This has led no less an epidemiologist than David Sackett to conclude that there are essentially two pillars of sound clinical evidence, only one of which is experimentally derived from the RCT:¹⁵

"External clinical evidence can inform, but can never replace, individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision."

In light of these many arguments, I would maintain that reviews of clinical research should place far greater emphasis upon cohort studies and case series in its research goals rather than assume categorically that they provide inferior guidance to clinical decision-making than RCTs. It should be quite clear from this discussion that a well-crafted cohort or case series is far more informative than a flawed or corrupted RCT.

That said, one must then interpret such systematic reviews as Hondras' effort with extreme caution on the basis that one or more of its basic component RCTs is seriously flawed, such that the entire review might then have incorrectly evaluated the best clinical evidence available.

Anthony L. Rosner, Ph.D.
June 27, 2000

Top

REFERENCES:

¹Hondras MA, Linde K, Jones AP. Manual therapy for asthma. Cochrane Database Systematic Review 2000; 2: CD001992.

²Balon J, Aker PD, Crowther ER, Danielson C, Cox PG, O'Shaugnessy D, Walker C, Goldsmith CH, Duku E, Sears MR. A comparison of active and simulated chiropractic manipulation as adjunctive treatment for childhood asthma. New England Journal of Medicine 1998; 339: 1013-1020.

³Field T, Henteleff T, Hernandez M, Martinez E, Mavunda K, Kuhn C, Schanberg S. Children with asthma improved pulmonary functions after massage therapy. Journal of Pediatrics 1998; 32(5): 854-858.

⁴Cherkin DC, Deyo RA, Battie M, Street J, Barlow W. A comparison of physical therapy, chiropractic manipulation, and provision of an educational booklet for the treatment of patients with low back pain. New England Journal of Medicine 1998; 339: 1021-1029.

⁵Chapman-Smith D. Back pain, science, politics and money. The Chiropractic Report November 1998; 12(6).

⁶Freeman M, Rossignol A. A critical evaluation of the methodology of a low back pain clinical trial: A case study in misleading statistics. Journal of Manipulative and Physiological Therapeutics 2000; 23(5): in press.

⁷Rosen M. Back pain. Report of a Clinical Standards Advisory Group Committee on back pain. May 1994, London: HMSO.

⁸Royal College of General Practitioners, unpublished update of CSAG Guidelines [reference 2], 1999.

⁹Juni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. Journal of the American Medical Association 1999; 282(11): 1054-1060.

¹⁰Johansen HK, Gotzsche PC, Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. Journal of the American Medical Association 1999; 282(18): 1752-1759.

¹¹Rennie D. Fair conduct and fair reporting of clinical trials. Journal of the American Medical Association 1999; 282(18): 1766-1768.

¹²Gotzsche PC. Multiple publication of reports of drug trials. European Journal of Clinical Pharmacology 1989; 36: 429-432.

¹³Huston P, Moher D. Redundancy, disaggregation, and the integrity of medical research. Lancet 1996; 347: 1024-1026.

¹⁴Tramer MR, Reynolds DJM, Moore RA, McQuay HJ. Impact of covert duplicate publication on meta-analysis: A case study. British Medical Journal 1997; 315: 635-640.

¹⁵Sackett DL. Editorial: Evidence-based medicine. Spine 1998; 23(10): 1085-1086.

Top