Science and Ethics of Sham Surgery: Title and subTitle BreakA Survey of Parkinson Disease Clinical Researchers FREE

What is the appropriate control condition for testing a neurosurgical intervention for the treatment of Parkinson disease (PD)? The use of placebo or sham-surgery controls—in which, for example, burr holes are created without further experimental intervention—in randomized trials of surgical interventions is becoming more common in PD research^{1 - 4} but not without controversy.⁵ The controversy may be further magnified when the intervention involves gene-transfer therapies, an area that has received its own share of ethical scrutiny.⁶ A phase 1 study of gene transfer for PD is already under way,⁷ and other studies will likely follow. Thus, we may soon be facing the question of how best to conduct a gene-transfer efficacy trial for PD.

Aside from questions surrounding informed consent, the controversy over sham-surgery controls involves 2 distinct but related issues. The first is whether a randomized clinical trial with a sham-surgery control is the scientifically optimum method to test novel neurosurgical interventions for PD. Because of significant placebo responses in PD treatment trials,^{8 - 9} many argue that a sham-control trial is the scientific ideal^{10 - 12} while others believe alternative designs are adequate.¹³ Second, is a sham-control condition ethically permissible in PD research? Specifically, are the risks and burdens to the subjects in randomized clinical trials with sham-surgery controls reasonable in relation to the potential benefit to the subjects and to society, as current federal regulations (45 CFR 46.111.a.2) require? Given that the degree of invasiveness varies with the kind of sham condition at issue, which sham conditions optimally minimize the risk of harm to subjects without sacrificing scientific value?

We surveyed experts in PD clinical research regarding their views about the science and ethics of an efficacy study testing a hypothetical novel gene-transfer technique with promising phase 1 results.

We surveyed the voting investigator members of the Parkinson Study Group (PSG), a nonprofit cooperative group of PD experts in the United States and Canada. The PSG consists of investigators who are employed or otherwise engaged by academic or research institutions. The PSG currently has approximately 85 sites, including more than 350 active investigators, coordinators, and scientists. At the time of the survey, there were 168 voting investigator members.

The survey was conducted over the Internet and can be reviewed online at http://www.supersurvey.com/customers/mbh/rochester/. The hypothetical scenario consisted of 2 main topics. First, the respondents were asked to suppose that, following preliminary rodent and primate studies, a successful phase 1 study of a novel gene-transfer therapy for PD had been conducted on 8 subjects. They were told that the experiment appeared to reduce a subject’s awake time spent in the “off” state, “as well as reducing time spent with troubling dyskinesias. There were no significant short-term (6-month) negative effects.” Second, they were asked to compare 2 potential randomized control trial designs for a follow-up study involving 50 subjects, the goal of which was to test the efficacy and safety of the gene-transfer therapy. The sham-control study design would compare gene-transfer surgery plus optimal pharmacological treatment vs sham surgery plus optimal pharmacological treatment (blinded study), in which sham surgery consisted of drilling 2 full-thickness burr holes. The open-control study would compare gene-transfer surgery plus optimal pharmacological treatment vs optimal pharmacological treatment alone (unblinded study). For the sham-control study, the respondents were told that if the gene-transfer surgery turned out to be safe and effective, “the participants who undergo sham surgery will be offered the gene transfer procedure using the predrilled holes.”

The survey questions are presented in Table 1, Table 2, and Table 3. Except for the questions regarding quantitative estimates of efficacy and false-positive rates, the respondents were invited to explain their answers in open-ended format. An earlier version of the survey was pretested at 2 scientific meetings.

The survey was conducted from April 5, 2004, through April 26, 2004. The e-mail list obtained from the PSG contained more than the investigator voting members, but it was decided a priori that the primary analysis (presented in this article) would include the investigator voting members only. The respondents were sent 2 additional reminder e-mails. The e-mails contained an embedded link to the survey and a unique password. We used an online survey company (SuperSurvey; Tercent, Inc, Lake Oswego, Ore). The survey took 10 to 15 minutes to complete.

We tabulated frequency data for categorical answers and means for quantitative answers. We analyzed the qualitative data from the open comments as follows. We (S.K. and R.W.) read through all the comments together and devised a coding scheme that we (R.H. and S.F.) later modified. Three of us independently coded each comment; the final code was assigned when at least 2 of 3 of us agreed. The few that did not get agreement at least from 2 of 3 of us were resolved by joint discussion. Some respondents made lengthy comments that required more than 1 code per comment.

The University of Michigan (Ann Arbor) and the University of Rochester (Rochester, NY) reviewed the protocol and deemed this survey exempt from institutional review board review.

Of the 168 eligible individuals, 103 responded to the survey, for a 61.3% overall response rate. Not everyone answered every question, but no question had a response rate lower than 55%. Mean ± SD age was 48 ± 8.3 years, and 77% were men. Most respondents made comments, for a total 439 comments; 2 or better agreement among 3 coding judges occurred in 91% of the comments.

Table 1 summarizes the responses to scientific questions surrounding the use of sham surgery as compared with open controls. The scientific preference for sham-surgery controls was nearly unanimous (91/94 or 96.8%). The most common comments were that a sham-control design better controls for placebo effect (57/79 or 72% of comments mentioned this), that a sham-control design has less potential for bias (13/79 or 16%), and that results of an efficacy study would not be definitive without a sham control (9/79 or 11%).

Almost every respondent (96.8%) felt that an open-control study risked greater likelihood of falsely concluding that an ineffective intervention was effective. The respondents felt that an open-control study would involve, on average, a 33.0% false-positive rate. Also, these investigators felt that based on the hypothetical vignette of the phase 1 study results, there was a 40.3% chance that the gene-transfer intervention would in fact turn out to be “truly effective.”

Table 2 summarizes the responses regarding the ethical permissibility of the sham-control trial and of the open-control trial. One half of the respondents answered that an open-control design should probably not or definitely not be allowed. Of those who held this view and made comments, 24 (61%) of 39 respondents were concerned about either false-positive results (33%, 13/39) or inconclusive results (28%, 11/39), and 10 (26%) of 39 respondents felt that despite the lower risk profile of an open-control study, the risks of an open-control design were not justified because of the decrease in scientific benefits.

As for the sham-control study, there was strong (93.5%) support for its permissibility. Various reasons were given in the comments: it is the only way to determine true efficacy (13/67, 19%), general statements to the effect that it is a better design (15/67, 22%), risks to subjects are outweighed by benefits to society and science (12/67, 18%), and the risk-benefit ratio for subjects is acceptable (7/67, 10%). Another 8 (12%) of 67 comments reflected a conditional approval: they would be in favor of allowing sham-control design but only after further preliminary testing.

Different types of sham surgery have varying degrees of invasiveness.^{1 - 4} What do the respondents believe about whether various types of sham surgery are justified by the potential benefits to science and society? The responses split along the line of whether or not the sham condition would involve penetration of the brain matter (Table 3). About 90% of the respondents felt the risks of burr holes without penetration of the dura were justifiable. A minority (less than 22%) of respondents believe that the risks associated with sham conditions that involve brain penetration are justifiable. These 2 groups differed in their open-ended comments. Those who are reluctant to endorse sham surgeries beyond burr holes were mainly concerned about placebo effects due to expectancy effects in the patient and bias in the clinical raters. Nearly all comments (53/55, 96%) made by those who do not favor sham conditions that penetrate the brain matter reflected this point: “The primary goal is to guard against placebo effects, investigator bias, and similar sources of bias. Insertion of a probe and related manipulations, which would considerably raise the chance of a serious complication (hemorrhage), are really controls for nonspecific effects of surgery. Preclinical work should be sufficient to defuse these concerns.” However, those in support of more invasive sham conditions felt that nonspecific clinical effects of brain manipulation should be controlled for as well. Twelve (57%) of 21 comments made by those in favor of going beyond burr holes reflected this position, eg, “Any of these [sham procedures] might be scientifically justified if the appropriate question were seriously raised (eg, does empty viral vector affect symptoms of PD?).”

A vast majority of PD investigators believe that a sham-control design is scientifically superior to an open-control design in efficacy trials of gene transfer for PD. A similarly large majority believe that the sham-surgery control condition is ethically permissible while half of the respondents question the ethics of an open-control trial. The respondents showed considerable concern about false-positive results and seemed to agree that the sham-control condition is needed to definitively show that the intervention is effective.

These sentiments may represent a shift from a decade ago¹⁴ that is at least in part due to the recent history of placebo-controlled surgical trials in PD: clinical experience and early open trials seemed to indicate benefit, only to be proven ineffective and even possibly harmful when tested in sham-surgery controlled trials.^{1 - 4} Prior to these negative studies, cell-transplant procedures as treatments for PD had been offered at many centers throughout the world.¹⁰

There is disagreement about the degree of invasiveness of the sham condition that is justifiable. Most seem to believe that controlling for placebo effects and potential bias is scientifically sufficient. One attractive feature of the sham-control condition of a recent cell transplant trial was the minimization of risk by not penetrating the dura.¹⁰ Since then, however, a placebo condition involving penetration into the brain (indwelling intraventricular catheter with periodic placebo injections) has been used.⁴ The question of degree of invasiveness of the sham condition should receive more attention and discussion.

What were the underlying quantitative assumptions behind the respondents’ views? Despite being told that a preceding phase 1 trial had been successful, the respondents’ estimate of the likelihood that the gene transfer described in the survey was “truly effective” was 40%. This figure is comparable with data reported by the Food and Drug Administration regarding drug development trials and does not seem unduly optimistic.¹⁵ Our respondents would view a positive open-control study to have a one-third chance of being a false-positive study. Given that the usual randomized clinical trial is powered to tolerate a 5% chance of falsely concluding that an experimental intervention is effective, the degree of skepticism about open-control efficacy trials is notable.

This survey has limitations. The PSG is predominantly US-based, and the results may not reflect the views of clinical researchers in the rest of the world.¹⁴ We also did not investigate whether the researchers’ responses were related to their own experience with conducting sham surgical trials.

Although scientists may have special insight into the scientific design and the degree of risk that may exist in sham surgical trials, the value-based policy judgments about the degree of subject and societal benefit needed to outweigh the research risks to subjects involves multiple stakeholders. In the future, it will be important to compare the perspective of the scientists with the perspectives of potential research subjects (as special stakeholders in the debate) and the general public (representing a broader societal perspective) to aid our society’s attempt to reach an informed ethical consensus on this important issue.

Correspondence: Scott Y. H. Kim, MD, PhD, Bioethics Program, 300 North Ingalls St, 7C27, Ann Arbor, MI 48109 (scottkim@umich.edu).

Accepted for Publication: February 4, 2005.

Author Contributions:Study concept and design: Kim, Frank, Holloway, Zimmerman, and Kieburtz. Acquisition of data: Frank, Wilson, and Kieburtz. Analysis and interpretation of data: Kim, Frank, Holloway, Wilson, and Kieburtz. Drafting of the manuscript: Kim, Frank, Holloway, and Wilson. Critical revision of the manuscript for important intellectual content: Kim, Frank, Holloway, Zimmerman, Wilson, and Kieburtz. Obtained funding: Kim, Holloway, and Kieburtz. Administrative, technical, and material support: Frank, Zimmerman, Wilson, and Kieburtz. Study supervision: Kim and Holloway.

Funding/Support: This study was supported as a subproject of U54-NS45309 from the National Institutes of Health, Bethesda, Md (Howard Federoff, Principal Investigator). Dr Kim is also supported by grant MH64172 from the National Institutes of Health.

Acknowledgment: We thank the members of the Parkinson Study Group who generously participated in this survey.