Trials used as basis for cancer drug approval show deficits in design, analysis

Randomized controlled trials (RCTs) make up most of the studies forming the basis of European Medicines Agency (EMA) approval of new cancer drugs between 2014 and 2016, but nearly half of these appear to be at high risk of bias because of deficits in their design, conduct of analysis, reports a study.

“Regulatory documents and the scientific literature had gaps in their reporting,” the researchers said. “Journal publications did not acknowledge the key limitations of the available evidence identified in regulatory documents.”

The EMA approved 32 new cancer drugs between 2014 and 2016 based on 54 pivotal studies, of which 41 (76 percent) were RCTs and 13 (24 percent) were either non-RCT or single-arm studies. Of the RCTs, 39 had available publications and were included in the analysis. Only 10 RCTs (26 percent) measured overall survival (OS) either as primary or coprimary endpoint, while the rest assessed surrogate measures such as progression-free survival and response rates. [BMJ 2019;366:l5221]

Almost half of the RCTs (n=19; 49 percent) were deemed to be at high risk of bias for their primary outcome. The most common domains leading to such judgment were concerns about missing outcome data (n=10) and measurement of the outcome (n=7). There were fewer RCTs evaluating OS as the primary endpoint at high risk of bias compared with those evaluating surrogate efficacy endpoints (20 percent vs 55 percent, respectively).

Considering available data in regulatory documents and the scientific literature separately showed varying overall risk of bias judgments for eight RCTs (21 percent), reflecting reporting inadequacies in both sources of information. Additional deficits for 10 drugs (31 percent) were identified beyond the domains captured in risk of bias assessments, namely magnitude of clinical benefit, inappropriate comparators and nonpreferred endpoints, which were not disclosed as limitations in scientific journals.

“Policymakers, investigators and clinicians should carefully consider risks of bias in pivotal trials that support regulatory decisions, and the extent to which new cancer therapies offer meaningful benefit to patients,” the researchers said.

These findings also stress the need to improve the design, conduct, analysis and reporting of cancer drug trials. [Lancet Oncol 2016;17:e560-567]

Design features of pivotal trials are shaped by regulatory agencies and their evidence requirements, which is why regulatory action is warranted to ensure that pharmaceutical manufacturers routinely assess their products in RCTs that collect data on meaningful outcomes, according to the researchers. [Milbank Q 2017;95:261-290; BMJ 2015;351:h5542; BMJ 2011;343:d4849]

In a linked editorial, Barbara Mintzes, associate professor of the School of Pharmacy and Charles Perkin Centre at University of Sydney, and Agnes Vitry, senior lecturer of the School of Pharmacy and Medical Science at University of South Australia, highlighted the need to raise standards of trials and regulation to “ensure real benefit for patients.” [BMJ 2019;366:l5399]

“Uncertainty and exaggeration of the evidence that supports approval of cancer drugs causes direct harm if patients risk severe or fatal adverse effects without likely benefit, or forgo more effective and safer treatments,” they wrote. “Inaccurate evidence also leads to intangible harms if it encourages false hope and creates a distraction from needed palliative care.”

The current cross-sectional analysis examined design characteristics (randomization, comparators and endpoints), risk of bias (arising from the randomization process, deviations from intended interventions, missing outcome data, measurement of the outcome and selection of the reported result) and reporting adequacy (completeness and consistency of information in trial protocols, publications, supplementary appendices, clinical trial registry records, and regulatory documents) of pivotal RCTs of cancer drugs approved by the EMA.