To the Editors:

We read with interest the article by Meythaler, et al., entitled, "Amantadine to improve neurorecovery in traumatic brain injury-associated diffuse axonal injury: a pilot double-blind randomized trial" (J Head Trauma Rehabil. 2002;17[4]:300-313). The authors describe the results of a double-blind, placebo-controlled crossover trial of amantadine in subjects with acute traumatic brain injury (TBI), concluding that their trial "demonstrated drug-related improvement with no untoward safety effects." Dr. Horn, the issue editor, largely concurred, stating that "this is the first study to demonstrate the effectiveness of amantadine for TBI pharmacological rehabilitation with a rigorous evidence-based design (Preface. J Head Trauma Rehabil. 2002;17[4]:v-vi). We believe that several serious design flaws prevent any conclusions about the efficacy of amantadine.

First, the authors used a crossover design in which one group of subjects (G1) received amantadine during the first 6 weeks postinjury whereas a second group (G2) received placebo. Treatments were reversed for the next 6 weeks. Crossover designs are best used when the outcome variable is expected to return to pretreatment levels when treatment is stopped, and very difficult to use in the context of rapid (and irreversible) spontaneous recovery. Since one expects the pace of recovery to slow over time naturally, the group treated second will have already had substantial improvement by the time treatment begins and will have less room for further improvement. During weeks 1 to 6, on average, the range of possible Disability Rating Scale (DRS) scores was 15 points (i.e., 15 to 1). By the time G1 reached the placebo phase at week 7, the range of possible DRS scores had narrowed to only 5 points (i.e., 5 to 1). Moreover, the group treated first had better scores at the point of crossover, further limiting their ability to improve to an "equivalent degree" to G2 in the second phase. Moreover, because the DRS is an ordinal scale with unequal step sizes, comparing "rates" of improvement via difference scores is problematic.

One could, in principle, simply compare the outcomes at the end of the first phase and turn this study into a parallel groups design. However, this option is confounded by initial nonequivalence of the two treatment groups. G1's initial DRS score (M = 15.5) was significantly lower than G2's score (M = 21.7), indicating that G2 was more severely impaired than G1. G2 showed essentially the same degree of DRS improvement (9.4 points) as G1 across the initial 6 weeks while on placebo suggesting that spontaneous recovery may have accounted for the improvement noted in both groups during this period. In fact, group comparisons of rate of change in the first 6 weeks were not significant. One could strengthen this comparison by using initial DRS score as a covariate in the statistical analysis, though this remains somewhat problematic given the ordinal nature of the scale.

Treatment effectiveness studies typically require large samples to ensure comparability of the treatment groups through simple randomization. Group balance can be improved in smaller studies through stratified random assignment, assuming that the relevant prognostic factors are known and used in the stratification. Minor differences in prognosis can be controlled for statistically, depending on the nature of the measures used, as mentioned previously. Moreover, pilot studies of this type, conducted during rapid spontaneous recovery, are unlikely to provide interpretable results because the magnitude of "spontaneous" change is likely to be as large or larger than any treatment effect, and to be highly variable.

Although pilot studies are important, it is essential that their limitations be recognized to avoid overinterpretation of results. Some readers may view this study as demonstrating efficacy and will thus be reluctant to endorse a subsequent placebo-controlled trial, in which one group of patients would be "denied an effective treatment." Given that the authors have demonstrated reasonable safety, we strongly encourage the conduct of a larger scale parallel groups design study to definitively evaluate the efficacy of amantadine.