Use of the Multiple Sclerosis Functional Composite as an Outcome Measure in a Phase 3 Clinical Trial FREE

TO ADDRESS THE poor reliability and insensitivity to change over time of the available multiple sclerosis (MS) clinical rating scales, including the Kurtzke Expanded Disability Status Scale (EDSS),¹ the National MS Society's Clinical Assessment Task Force (NMSS Task Force) developed the MS Functional Composite (MSFC).^2- 5 The MSFC includes quantitative tests of leg function/ambulation (Timed 25-Foot Walk [T25FW]), arm function (9-Hole Peg Test [9HPT]), and cognitive function (Paced Auditory Serial Addition Test [PASAT]). Although vision was recognized as an important clinical dimension in MS, the measures of visual function for which longitudinal data were available were found not to be sufficiently sensitive to change to include in the initial version of the MSFC. In recent studies,⁶ contrast sensitivity showed promise as a measure of visual function for potential inclusion in the MSFC.

Several studies demonstrated that the MSFC correlated with disability as measured by the EDSS,^4,7- 8 disease course,⁷ and patient self-report measures of symptoms and quality of life.⁸ Preliminary studies^7,9- 10 suggested that magnetic resonance imaging measures of cranial lesion burden and atrophy correlated better with the MSFC compared with the EDSS. Preliminary studies^9- 10 also showed that the MSFC score in patients with early relapsing-remitting MS and the change in the MSFC score over 2 years were strongly predictive of brain atrophy and clinically significant disability 6 to 8 years later. Taken together, these results provided strong support for the validity of the MSFC.

Progression of neurologic disability as reflected by the MSFC score is the primary outcome measure in the ongoing multinational phase 3 trial of interferon beta-1a (Avonex) in patients with secondary progressive (SP) MS (International Multiple Sclerosis Secondary Progressive Avonex Controlled Trial [IMPACT]). Because it was anticipated that most clinicians and most patients with MS would be unfamiliar with the MSFC, standardized examiner training materials and patient testing procedures were developed in preparation for IMPACT. A pilot study¹¹ confirmed the effectiveness of training procedures for the MSFC examining technicians, the practicality of the MSFC testing protocol, and the excellent intrarater and interrater reliability of the MSFC. As expected, practice effects were evident in the initial testing sessions but stabilized by the fourth administration. Since practice effects potentially could obscure changes in disease status over time, these factors were used in designing the IMPACT protocol and in implementing MSFC testing.

Informed consent for participation in IMPACT was obtained after the potential risks and benefits of the study were reviewed with potential subjects.

Examining technicians at 42 study sites in the United States, Canada, and Europe, most without previous experience with the MSFC, were trained to administer the MSFC at a prestudy investigators' meeting. Formal training included a standardized didactic description of the MSFC background and testing procedures, review of the MSFC administration and scoring manual, viewing of a training videotape, and an interactive MSFC administration practice session with feedback. Examiners' knowledge and performance were assessed at the end of the training session to confirm familiarity with the MSFC procedures. Examiner training required approximately 4 hours.

The MSFC was administered to patients using a standardized protocol. The order of testing in a session was as follows: (1) T25FW (trial 1 and trial 2), (2) 9HPT (dominant hand: trial 1 and trial 2; and nondominant hand: trial 1 and trial 2), and (3) PASAT (3-second interstimulus interval and 2-second interstimulus interval).

Patient instructions for the T25FW, 9HPT, and PASAT were available in English, French, German, Dutch, Greek, and Hebrew. Two alternate forms for the PASAT were used. The order in which the 2 forms was used varied among patients and among sessions for an individual patient. The same examining technician at each study site administered the MSFC to a given patient at each study visit. To compensate for practice effects and to achieve a stable baseline before randomization, the IMPACT protocol incorporated 3 prebaseline testing sessions over 28 days before randomization. Examining technicians did not have access to the results of previous testing sessions once completed.

The MSFC score was derived from 3 components: (1) T25FW (the mean of the scores of the 2 T25FW trials), (2) 9HPT (the 2 trials for each hand were averaged, then converted to reciprocals, and the 2 reciprocals were then averaged), and (3) PASAT3 (the number correct on the PASAT with a 3-second interstimulus interval). The MSFC score was calculated as the mean of the z scores of the 3 components (T25FW, 9HPT, and PASAT3). A z score is a standardized score representing the number of SD units a given value is from a population mean and is calculated by subtracting the mean of the reference population from the test result, then dividing by the SD of the reference population. z Scores for the T25FW, 9HPT, and PASAT3 were calculated with reference to the pooled data set derived from the IMPACT baseline visit (session 4). For the T25FW and 9HPT, a higher raw score (time to complete the task in seconds) represents deterioration, whereas for the PASAT3, a lower raw score (number correct) represents deterioration. Consequently, in combining the 3 components into a single z score, the sign of the T25FW z score was reversed, and the z score of the inverse of the 9HPT time was used so that the direction of change was consistent across components. Thus, a decrease in the z scores of the 3 components and of the MSFC z score all represent deterioration in neurologic function.

Correlations among the MSFC, its components, and the EDSS were analyzed using Spearman rank correlations. The intraclass correlation coefficient was used to measure the session-to-session intrarater reliability of the MSFC.¹² The MSFC score for session 3 (the last prebaseline session) was compared with that of session 4 (the baseline session) as the primary measure of reliability. All statistical analyses were performed using SAS statistical software (SAS Institute Inc, Cary, NC).

A total of 436 subjects with SP MS and an EDSS score of 3.5 to 6.5 were enrolled in IMPACT. Baseline demographic and clinical characteristics are as follows:

Figure 1 illustrates the distribution of patients across the range of baseline EDSS scores. As expected, there was an overrepresentation of patients with EDSS scores of 4.0, 6.0, and 6.5, with a relative underrepresentation of patients with EDSS scores of 4.5 to 5.5, as has been seen in most population distributions for the EDSS.^13- 14

The z scores for the MSFC components were calculated with reference to the group means and SDs from the IMPACT baseline visit (session 4). The sample size estimate for IMPACT was based on an analysis of a cohort of 326 patients in the NMSS Task Force data set with SP MS and an EDSS score of 3.5 to 6.5. The values at baseline in IMPACT were comparable to those in the NMSS Task Force data set (Table 1), aside from the somewhat greater mean and SD of the T25FW score in the IMPACT baseline data set.

Practice effects on the MSFC, manifested as improved performance and decreased session-to-session variability, were evident in the first 3 prebaseline sessions but stabilized by the fourth session (Table 2). At the time of this analysis, pooled data blind to treatment group from the first postenrollment assessment at month 3 were available for 426 patients. The mean MSFC z score decreased slightly from 0.00 at baseline to −0.12 at month 3.

The 3 MSFC components contributed differentially to the practice effects observed for the overall composite (Figure 2). Progressive improvement in performance over the first 4 sessions was most evident for the PASAT3. The 9HPT exhibited less prominent practice effects, with most of the improvement occurring from the first testing session to the second. There were no discernible practice effects on the T25FW.

The MSFC demonstrated excellent intrarater reliability in IMPACT. The intraclass correlation coefficient for the MSFC for session 3 (the last prebaseline session) vs session 4 (the baseline visit) was 0.90. The intraclass correlation coefficient for the 4 sessions taken together was 0.87, despite the increased variability in the early testing sessions due to practice effects.

Table 3 provides the Spearman rank correlations among the MSFC, its components, and the EDSS at baseline. Similar to the analysis of the NMSS Task Force data set,⁴ correlation between the MSFC and the EDSS was moderately strong, even over the more restricted EDSS score range in IMPACT. Figure 3A illustrates the distribution of the baseline MSFC z scores as a function of baseline EDSS scores. For the patients as a group, there was nearly linear worsening in the mean MSFC z score from an EDSS score of 3.5 to 6.0. For an EDSS score of 6.5, there was substantially greater worsening and intersubject variability in the MSFC.

Correlations among the individual components of the MSFC were modest (Table 3), indicating that each measured an independent clinical dimension. The 3 components correlated comparably with the overall MSFC, indicating that each contributed information to the MSFC for the group as a whole. As expected, the T25FW was the MSFC component that correlated most strongly with the EDSS in this EDSS range. The 9HPT correlated moderately, and the correlation between the PASAT3 and the EDSS was weak. Figure 3B-D illustrates the distributions of the 3 MSFC components at baseline as a function of the EDSS. For the patients as a group, there was progressive worsening in the T25FW score from an EDSS score of 3.5 to 6.0, with relatively little variability in the T25FW score at each EDSS level. There was dramatic worsening and an increase in intersubject variability in the T25FW score at an EDSS score of 6.5. In contrast, the 9HPT and PASAT3 demonstrated substantial intersubject variability at all EDSS steps and a less prominent trend toward worsening with an increased EDSS score.

Quantitative assessment of neurologic impairment and disability in patients with MS is difficult because of the marked clinical heterogeneity exhibited by the disease, between patients and within individual patients over time. There is variability in the manifestations that are present, their severity, and the time course over which they develop. Also, some of the common manifestations of MS that can have substantial impact on quality of life, such as cognitive dysfunction, are difficult to quantify using the standard neurologic examination and the rating scales based on it. The EDSS¹ has been the most widely used clinical outcome measure in MS therapeutic trials. However, the EDSS has several well-recognized shortcomings.^14- 17 The MSFC developed by the NMSS Task Force is anticipated to have better reliability, advantageous psychometric properties, and greater sensitivity to change compared with the EDSS and other available MS clinical outcome measures.^2- 5 The results of previous studies and those presented herein demonstrate that the MSFC meets many of these goals.

It was anticipated that most clinicians and most patients with MS would be unfamiliar with the MSFC. Therefore, in preparation for IMPACT, standardized examiner training materials and a patient testing protocol were developed. A pilot study¹¹ confirmed the effectiveness of the examiner training procedures and the feasibility of the testing protocol in terms of ease of administration, the time required, and patient acceptance. The MSFC demonstrated excellent intrarater and interrater reliability in this small pilot study carried out at a single site. The MSFC again demonstrated excellent reliability in IMPACT, a multinational study with 42 participating sites, virtually none of which had prior experience with the measure. The reliability of the MSFC in IMPACT was substantially better than that previously reported for the EDSS in other trials.^18- 22

Because of previous clinical experience with quantitative measures and analysis of the NMSS Task Force pooled data set,⁴ practice effects on the MSFC were expected to occur. That is, performance and variability were anticipated to improve as subjects (and examiners) became familiar with the tasks. In the pilot study¹¹ carried out before IMPACT, practice effects appeared to stabilize by the fourth testing session. Therefore, the IMPACT protocol included 3 prebaseline testing sessions over 28 days before randomization. The experience in IMPACT confirmed that the MSFC is susceptible to practice effects. The substantial variability in the change from session to session before baseline suggested that the magnitude and time course of practice effects differed among patients. This potential variability could obscure underlying biological change and impede the detection of a treatment effect in the context of a clinical trial. Three prebaseline sessions appeared to be sufficient to compensate for practice effects on the MSFC, which stabilized by the fourth session. The pooled MSFC data from month 3 demonstrated neither substantial improvement, which would suggest further practice effects, nor substantial worsening, which would suggest "forgetting."

With the small sample size in the pilot study,¹¹ it was not possible to discern the relative contributions of the 3 MSFC components to the practice effects seen for the overall measure. With the larger sample size in IMPACT, it was clear that the primary source of practice effects on the MSFC was the PASAT3. Strong practice effects on the PASAT have been reported previously in normal and neurologically impaired subjects^23- 25 and in subjects with MS.²⁶ In IMPACT, modest practice effects on the 9HPT also were evident. We are unaware of any previous systematic study of variability over time or practice effects on the 9HPT in patients with impaired upper extremity function, including those with MS. No practice effects were seen on the T25FW in IMPACT. A previous study of 63 subjects with MS undergoing T25FW testing on 5 consecutive days also failed to detect any systematic trend suggesting practice effects (Steven Schwid, MD, written communication, March 1999). Although the main source of MSFC practice effects is the PASAT3, we recommend that prebaseline testing include all 3 components to familiarize subjects and examiners with the standardized MSFC testing protocol.

Two remaining key issues with the MSFC are its validity as a measure of disability in patients with MS and its utility as an outcome measure in clinical trials. Preliminary studies^7,9- 10 demonstrated good correlation between magnetic resonance imaging measures of cranial lesion burden and atrophy and the MSFC. Three previous studies^4,7- 8 have shown that the MSFC correlated with the EDSS, the most widely used measure of neurologic disability in MS clinical trials. The MSFC was shown to be worse in patients with SP MS compared with those with relapsing-remitting disease.⁷ Finally, the MSFC correlated well with patient self-report measures of symptoms and quality of life.⁸

The results reported herein for IMPACT provide further support for the validity of the MSFC from analysis of a fourth independent data set. First, the population values for the T25FW, 9HPT, and PASAT3 at baseline in IMPACT were comparable to those calculated for the cohort of patients with SP MS and an EDSS score of 3.5 to 6.5 in the NMSS Task Force data set used to estimate sample size for the trial. The somewhat greater mean and SD shown for the T25FW at baseline in IMPACT likely was due to the higher proportion of patients with EDSS scores of 6.0 and 6.5.

More important, correlations among the MSFC, its components, and the EDSS, which were collected prospectively in IMPACT, were similar to those previously reported by the NMSS Task Force based on retrospective analysis of existing data sets. Correlations among the 3 components of the MSFC were modest. Thus, although patients with MS tend to demonstrate parallel impairment in several domains, the T25FW, 9HPT, and PASAT3 clearly measure independent clinical dimensions of MS. As expected, the correlation was strongest between the T25FW and 9HPT, but neither correlated well with the PASAT3. The strengths of the correlations between the MSFC and its 3 components were roughly comparable, suggesting that all 3 contribute information to the MSFC for the patient population as a whole. The MSFC correlated moderately with the EDSS, supporting the convergent validity of the MSFC (correlation with another measure of neurologic disability) and its divergent validity (measurement of aspects of MS not covered by the EDSS). As expected, among the components of the MSFC, the T25FW correlated best with the EDSS. For an EDSS score between 3.5 and 6.5, the range used as an enrollment criterion for IMPACT, the EDSS is primarily an ambulation scale with the score determined solely by how far a patient can walk and the type of assistive device required. It was not surprising that walking speed and distance would be related. Correlation between the 9HPT and EDSS was less strong. Patients with motor impairment affecting ambulation also tend to have motor impairment in the arms. However, arm function does not affect EDSS scoring in the range studied in IMPACT. Correlation between the PASAT3 and the EDSS was weak. The EDSS, like all clinical rating scales based on the standard neurologic examination, measures cognitive dysfunction in patients with MS poorly throughout its range.

In summary, the results of the previously reported pilot study and the baseline data from IMPACT reported herein confirmed the excellent reliability of the MSFC when standardized procedures are used to train examiners and to assess patients. Incorporation of 3 prebaseline testing sessions compensated for practice effects on the MSFC. The baseline IMPACT results corroborated the results of previous studies correlating the MSFC and the EDSS. The pattern of correlations among the MSFC, its components, and the EDSS supported the validity of the MSFC. These results again indicated that the MSFC assesses aspects of neurologic function not measured by the EDSS, suggesting that it will be more sensitive to detect change over time and better able to demonstrate a therapeutic effect when one exists.

University of California Davis Medical Center, Sacramento: M. Agius, MD; J. Adams, RN; R. Beale; D. Richman, MD; N. Vijayan, MD; V. Wheelock, MD. University of Maryland School of Medicine, Baltimore: C. Bever, MD; K. Costello, RN; S. Dhib-Jalbut, MD; K. Johnson, MD; E. Katz, RN; H. Panitch, MD. Minneapolis Clinic of Neurology, Golden Valley, Minn: G. Birnbaum, MD; I. Altafullah, MD; G. Christenson, RN; K. Stillwell. University of Washington Medical Center, Seattle: J. Bowen, MD; A. Gianas, RN; E. Krause, MD; E. Yuen, MD. Rhode Island Hospital–Brown University, Providence: P. Calabresi, MD; L. Alderson, MD; G. Johnson, MD; P. Mills, RN; J. Quinless; J. Wilterdink, MD. Mayo Clinic, Scottsdale, Ariz: J. Carter, MD; J. Buckner; R. Caselli, MD; K. MacElwee, RN; K. Nelson, MD; J. Takata, MD. Mellen Center for Multiple Sclerosis Treatment and Research, The Cleveland Clinic Foundation, Cleveland, Ohio: J. Cohen, MD; D. Bolibrush; C. Hara-Cleaver, RN; R. Kinkel, MD; R. Rudick, MD; L. Stone, MD. University of Medicine and Dentistry–New Jersey Medical School, Newark: S. Cook, MD; D. Cadavid, MD; A. Jotkowitz, RN; Y. Maeda, MD; J. Quinless, RN. University of Colorado Health Sciences Center, Denver: J. Corboy, MD; J. Bainbridge, PharmD; J. LaGuardia, MD; H. Neville, MD; R. Taggert, RN; R. Wright, MD. Georgetown University Hospital, Washington, DC: H. Crayton, MD; D. Bartlett, RN; S. Cohan; T. Gustafson, RN; J. Richert, MD; C. Tornatore, MD. University of New Mexico Health Science Center, Albuquerque: C. Ford, MD; G. Graham; A. Kradochvil; J. Maldonado, MD. University of Pennsylvania, Philadelphia: S. Galetta, MD; L. Balcer, MD; F. Gonzalez-Scarano, MD; R. Grossman, MD; D. Kolson, MD; G. Liu, MD; M. Mills; D. Pfohl, RN; A. Pruitt, MD; A.-M. Rostami, MD, PhD; D. Silberberg, MD. University of Rochester, Rochester, NY: A. Goodman, MD; M. Petrie, RN; E. Schwid, RN; S. Schwid, MD; D. Shrirer, MD. Yale School of Medicine, New Haven, Conn: J. Guarnaccia, MD; J. Hayes; S. Novella, MD; H. Patwa, MD; M. Rizzo, MD; M. Shepard, RN; T. Vollmer, MD. University of Mississippi Medical Center, Jackson: R. Herndon, MD; J. Corbett, MD; R. Fredericks, MD; J. Pittman, PharmD; P. Reynolds, MD; M. Umberger, RN; R. Wier, RN. Buffalo General Hospital, Buffalo, NY: L. Jacobs, MD; R. Bakshi, MD; E. Gallagher, RN; S. Greenberg, MD; F. Munschauer III, MD; K. Murray, MD; K. Patrick; B. Weinstock-Guttman, MD. University of Southern California, Los Angeles: N. Kachuk, MD; L. Adobo, MA; C. Cooper; R. Cowan, MD; D. Ko, MD. MS Center at Carolinas Medical Center, Charlotte, NC: M. Kaufman, MD; A. Diedrich, MD; D. Lutz; R. Follmer, MD; S. Putman, MD; S. Presley. Alleghany University of the Health Sciences, Philadelphia: F. Lublin, MD; R. Elfont, MD, PhD; L. Kelly, PhD; M. Weber. Oregon Health Sciences University, Portland: M. Mass, MD; D. Bourdette, MD; R. Camicioli, MD; S. Cooper-Hanel; D. Griffiths, RN; R. Whitham, MD. Indiana University School of Medicine, Indianapolis: D. Mattson, MD, PhD; M. Farlow, MD; J. Fleck, MD; J. Hayes; D. Jackson, RN; R. Pourmand, MD. Maimonides Medical Center, Brooklyn, NY: A. Miller, MD; M. Brodbari; K. Bruining, MD; E. Drexler, MD; H. Elinzano, MD; M. Keilson, MD; T. LaRocca, RN, BSN; L. Morgante, RN, MSN; L. Sciarra, RN, MSN; R. Wolintz, MD. University of California at Los Angeles MS Center: L. Myers, MD; R. Baumhefner, MD; S. Craig, RN; R. Klutch; N. Sicotte, MD. Ohio State University, Columbus: K. Rammohan, MD; D. Higgens, MD; J. Kissel, MD; D. Lynn, MD; A. Sifford, RN; A. Slivka, MD; J. Warner. Vanderbilt University MS Center, Nashville, Tenn: S. Sriram, MD; S. Hunter, MD, PhD; H. Moses, Jr, MD; F. Niaz, MD; J. Simmons, RN; K. Reece, RN. MS Center at Shepherd Center, Atlanta, Ga: W. Stuart, MD; E. Awad, MD; D. Court, RN; R. Gilbert, MD; E. Hedaya, MD; S. Morgan; D. Stuart, MD. Washington University, St Louis, Mo: J. Trotter, MD; D. Cross, MD; D. Derrington, MD; J. Lauber, RN; C. Martinez, LPN. University of California at Irvine: S. Van den Noort, MD; R. Babcock, RN; P. Fotinakes, MD; Y. Qin; G. Thai, MD. Yale University, New Haven: T. Vollmer, MD; G. Blanco; J. Guarnaccia, MD; T. Halverson; E. Kane; S. Markovic-Plese, MD; L. Marshall; S. Novella, MD; H. Patwa, MD; G. Richerson, MD; M. Rizzo, MD. University of Alabama at Birmingham: J. Whitaker, MD; K. Bashir, MD; B. Layton, RN; L. Nabors III, MD; A. Nicholas, MD, PhD; R. Slaughter, MD; K. Whikehart. University of Texas at Houston Health Science Center: J. Wolinsky, MD; S. Brod, MD; E. Cerreta, RN; W. Lindsey, MD; C. Weisbrodt, RN. Hopital Notre-Dame, Montreal, Quebec: P. Duquette, MD; G. Bernier, MD; P. Cossette, MD; R. Dubois, RN; J. Poirier. Ottawa General Hospital, Ottawa, Ontario: M. Freedman, MD; S. Christie, MD; C. Freedman, BMT (PT); R. Nelson, MD; H. Rabinovitch, MD; U. Webb, RN. University of Toronto MS Clinic, St Michael's Hospital, Toronto, Ontario: P. O'Connor, MD; J. Fleming; P. Fleming, RN; T. Gray, MD; M. Hohol, MD; P. Marchotti, MD. University of Western Ontario Hospital, London, Ontario: G. Rice, MD; T. Bentall, RN; G. Ebers, MD, DPhil; M. Hopkins; P. Mandolfino, MD; M. Nicolle, MD; D. Wingerchuk, MD. Neurological Center Quellenhof, Bad Wildbad, Germany: A. Foit, MD; R. Ascheron; A. Fauser, MD; J. Fernholtz; A. Immesberger; M. Riexinger; R. Roth. Stadtisch Kliniken Osnabruck, Germany: P. Haller, PhD, MD; D. Lammers; S. Stove, MD; A. Terwey, MD; S. Windhagen. Medizinische Hoschule, Hannover, Germany: F. Heidenreich, MD; H. Becker; K. Fricke, MD; R. Hilse, MD; N. Kohler, MD; A. Kracke, MD; R. Lindert, MD; S. Maniak, MD; S. Marckmann, MD. Athens University, Athens, Greece: M. Dalakas, MD; C. Kilidreas, MD; A. Rombos, MD; C. Taskanikas, MD; C. Voumvourakis, MD. Haddassah Hebrew National University, Jerusalem, Israel: O. Abramsky, MD; A. Askenazi, MD; A. Karmi, MD; D. Karussis, MD; A. Linitsky, MD; M. Mor, MD. VU Ziekenhuis, Amsterdam, the Netherlands: C. Polman, MD, PhD; J Castelijns, MD; B. Jelles, MD; N. Kalkers, MD; J. Killestein, MD; T. Schweigmann; B. Uitdehaag, MD. Chefarzt der Marinanne-Strauβ-Klinik, Milchberg, Germany: N. Konig, MD; H. Albrecht, MD; W. Feneberg, MD; C. Kutschker, MD; W. Pollmann, MD; M. Stark, MD. (For each site, the principal investigator is listed first.)

Accepted for publication November 27, 2000.

This study and the development of the MSFC manual (a manual describing MSFC testing and scoring procedures, which is available through the National MS Society) were supported by Biogen, Inc, Cambridge, Mass.

Presented at the annual meeting of the American Academy of Neurology, Toronto, Ontario, April 22, 1999.

We thank Richard A. Rudick, MD, for reviewing the manuscript.

Corresponding author and reprints: Jeffrey A. Cohen, MD, The Mellen Center-U10, The Cleveland Clinic Foundation, 9500 Euclid Ave, Cleveland, OH 44195 (e-mail: cohenj@ccf.org).