Alan Scott and Denis McMillan, PAREXEL International04.05.19
Late-stage clinical research remains a risky and costly business despite the reasonable expectation that risks would decrease as products advance successfully through the earlier phases of development. But, in fact, about half of Phase III trials fail.
Trials fail for a variety of reasons, including flawed science or study design, suboptimal dosing, poor study execution, slow patient enrollment, high dropout rates and, finally, a failure to demonstrate clinical or commercial value.
Most companies conduct at least some form of feasibility testing on their trial designs because they know that changes and miscalculations can cause delays and cost money. For example, the direct cost to implement a single protocol amendment averages approximately $500,000 in unplanned expenses and adds 61 days to project timelines. But a rudimentary glance at feasibility is not enough.
The surest way to mitigate and even avoid many of the risks of later-stage studies is to methodically and comprehensively examine the feasibility of a study’s design before it starts.
Once you’ve determined what’s feasible, who’s listening?
Feasibility analysis exists to inform clinical decision-makers about a variety of risks. But identifying and quantifying them is not enough. Risk analysis should stimulate action. And to drive action, feasibility analysis should be informed by quality data from multiple sources. There are many reasons why clinical trials fail to meet timelines, and feasibility-testing must look at all aspects of operationalizing a trial (see Table 1).
Table 1. Key Elements of a Systematic Risk-Based Feasibility Analysis
Accessing and interpreting the vast number of potential data sources that can aid in vetting a trial’s design is not always straightforward. Assessing the credibility of each data source is crucial and requires deep expertise. The increasing complexity of modern trial designs—which include, for example, narrow genetic eligibility criteria that require screening and involve administering technologically-advanced, expensive treatments—requires increasingly sophisticated analytic methods to determine possible impacts on a trial’s success. Adding to this difficulty, feasibility analyses are almost always conducted under tight time constraints.
The ability to quantify and communicate risk empowers sponsors to mitigate it. Every solution discovered through feasibility analyses will be a compromise between market access, time, cost and science. Feasibility data should both motivate and reassure decision-makers. Suggesting that a study design “might be challenging” is one thing; putting a monetary value on what the challenges may cost a sponsor is a surer way to get the attention of senior management.
The primacy of human data
There is no one approach to feasibility analysis that will fit all trials. And the best data to gauge feasibility, as well as the best analytic tools to obtain that data, depend on the risks of the specific protocol, especially as they concern patients.
For example, recently a major regulatory agency requested that a firm developing a drug for a serious neurologic condition conduct a placebo-controlled trial to demonstrate efficacy. The agency-mandated study design assumed, among other things, that patients had no therapeutic alternatives because none had gained formal regulatory approval. In fact, families had multiple off-label therapeutic options—some drugs approved for other conditions had shown efficacy and were being offered to patients on a routine basis—meaning that a placebo-controlled trial would have difficulty recruiting and retaining patients. It possibly also would be unethical. The firm engaged Parexel to conduct an arm’s-length analysis of the protocol’s viability. We interviewed patients and physicians, key opinion leaders, regulatory experts and bioethicists from a wide range of institutions about the trial’s design and how the treatment landscape had changed in recent years. We were able to quantify the negative impact of such a design on enrollment and retention and offer a data-driven argument that illustrated why a placebo-controlled trial was not feasible.
How the patient population is defined also has a direct impact on how well a study can be executed in a real world setting. For instance, if the entry criteria are too tight, patients may be too hard to find. Sponsors must therefore strike a balance between enrolling: 1) a strictly controlled, homogeneous population to reduce the level of noise in the data and increase the probability of an efficacy finding; and 2) enough patients fast enough to hew to development timelines, which could necessitate broadening entry criteria.
How many patients will make it through pre-screening, informed consent, screening and all the way to randomization? If, for every 100 patients screened, the sponsor ends up with one randomized patient, that will determine how many patients need to enter the top of the trial’s patient funnel.
And what is the burden of patient recruitment on site resources and is this adequately compensated for in the research funding that sites receive? How do we motivate the principal investigator to pay for additional resources?
Deeper understanding and quantification of the target patient population can reduce uncertainty. For example, a symptom monitoring assessment provides a picture of the prevalence of particular symptoms in a population—symptoms that may be crucial for patient trial eligibility. Electronic health data or prescription fulfillment records may be used to acquire a snapshot of a sample of patient’s current treatments and disease history which could play a critical part in the trial design and speed recruitment.
Practical designs are grounded in what motivates patients and physicians
A recruitment funnel analysis is both art and science: estimates need to be made in light of how patients, their families and physicians make their decision on whether to participate in a trial, or not.
Several tools can be used to understand how those choices are made. At Parexel, we rely on medical, therapeutic area and operational experts within our own organization, and we seek advice from external experts, opinion leaders, and experienced principal investigators and their staff within our global alliance network of 350 sites.
Engaging with investigators, patients and study site nurses who represent a critical, and often untapped reservoir of practical information and expertise provides invaluable insight. Sponsors can and should modify their protocols to address findings from surveys of these communities. In some cases, the design of a protocol may remain unchanged, but the geographic footprint of the trial can be improved.
For example, a recent Alzheimer’s disease study required a lumbar puncture (LP) procedure to obtain a cerebral spinal fluid sample (CSF). When a trial requires an invasive procedure of this kind it’s important to have deep knowledge of standards of care and cultural idiosyncrasies. Different countries have different levels of acceptance for certain interventions. In some European countries, LPs are widely accepted and routine; in other countries, they are rare and unfamiliar. If a study includes a non-negotiable burden—in this case, CSF sampling was needed to measure a biomarker hypothesized to be an indicator of disease progression—by understanding cultural standards, sponsors can choose sites in geographies that convey the best chance of recruitment success.
To gain an appreciation of the patient (and family/caregiver) perspective, Parexel routinely conducts a patient burden analysis of study designs, which involves asking questions such as:
Knowing what you don’t know is valuable in managing risk
Done correctly, analytical pressure-testing of a trial’s protocol enables sponsors to identify and eliminate unrealistic or impractical elements of the design, thereby reducing the risk of waste, missing development deadlines or outright failure. Even identifying uncertainties is worthwhile before a trial starts, giving sponsors insight as to where problems may arise and time to develop mitigation strategies.
Ultimately, feasibility testing should drive clinical decision-makers to put contingency plans in place that would not exist otherwise. And that will increase a late-stage trial’s chance of success.
Trials fail for a variety of reasons, including flawed science or study design, suboptimal dosing, poor study execution, slow patient enrollment, high dropout rates and, finally, a failure to demonstrate clinical or commercial value.
Most companies conduct at least some form of feasibility testing on their trial designs because they know that changes and miscalculations can cause delays and cost money. For example, the direct cost to implement a single protocol amendment averages approximately $500,000 in unplanned expenses and adds 61 days to project timelines. But a rudimentary glance at feasibility is not enough.
The surest way to mitigate and even avoid many of the risks of later-stage studies is to methodically and comprehensively examine the feasibility of a study’s design before it starts.
Once you’ve determined what’s feasible, who’s listening?
Feasibility analysis exists to inform clinical decision-makers about a variety of risks. But identifying and quantifying them is not enough. Risk analysis should stimulate action. And to drive action, feasibility analysis should be informed by quality data from multiple sources. There are many reasons why clinical trials fail to meet timelines, and feasibility-testing must look at all aspects of operationalizing a trial (see Table 1).
Table 1. Key Elements of a Systematic Risk-Based Feasibility Analysis
Accessing and interpreting the vast number of potential data sources that can aid in vetting a trial’s design is not always straightforward. Assessing the credibility of each data source is crucial and requires deep expertise. The increasing complexity of modern trial designs—which include, for example, narrow genetic eligibility criteria that require screening and involve administering technologically-advanced, expensive treatments—requires increasingly sophisticated analytic methods to determine possible impacts on a trial’s success. Adding to this difficulty, feasibility analyses are almost always conducted under tight time constraints.
The ability to quantify and communicate risk empowers sponsors to mitigate it. Every solution discovered through feasibility analyses will be a compromise between market access, time, cost and science. Feasibility data should both motivate and reassure decision-makers. Suggesting that a study design “might be challenging” is one thing; putting a monetary value on what the challenges may cost a sponsor is a surer way to get the attention of senior management.
The primacy of human data
There is no one approach to feasibility analysis that will fit all trials. And the best data to gauge feasibility, as well as the best analytic tools to obtain that data, depend on the risks of the specific protocol, especially as they concern patients.
For example, recently a major regulatory agency requested that a firm developing a drug for a serious neurologic condition conduct a placebo-controlled trial to demonstrate efficacy. The agency-mandated study design assumed, among other things, that patients had no therapeutic alternatives because none had gained formal regulatory approval. In fact, families had multiple off-label therapeutic options—some drugs approved for other conditions had shown efficacy and were being offered to patients on a routine basis—meaning that a placebo-controlled trial would have difficulty recruiting and retaining patients. It possibly also would be unethical. The firm engaged Parexel to conduct an arm’s-length analysis of the protocol’s viability. We interviewed patients and physicians, key opinion leaders, regulatory experts and bioethicists from a wide range of institutions about the trial’s design and how the treatment landscape had changed in recent years. We were able to quantify the negative impact of such a design on enrollment and retention and offer a data-driven argument that illustrated why a placebo-controlled trial was not feasible.
How the patient population is defined also has a direct impact on how well a study can be executed in a real world setting. For instance, if the entry criteria are too tight, patients may be too hard to find. Sponsors must therefore strike a balance between enrolling: 1) a strictly controlled, homogeneous population to reduce the level of noise in the data and increase the probability of an efficacy finding; and 2) enough patients fast enough to hew to development timelines, which could necessitate broadening entry criteria.
How many patients will make it through pre-screening, informed consent, screening and all the way to randomization? If, for every 100 patients screened, the sponsor ends up with one randomized patient, that will determine how many patients need to enter the top of the trial’s patient funnel.
And what is the burden of patient recruitment on site resources and is this adequately compensated for in the research funding that sites receive? How do we motivate the principal investigator to pay for additional resources?
Deeper understanding and quantification of the target patient population can reduce uncertainty. For example, a symptom monitoring assessment provides a picture of the prevalence of particular symptoms in a population—symptoms that may be crucial for patient trial eligibility. Electronic health data or prescription fulfillment records may be used to acquire a snapshot of a sample of patient’s current treatments and disease history which could play a critical part in the trial design and speed recruitment.
Practical designs are grounded in what motivates patients and physicians
A recruitment funnel analysis is both art and science: estimates need to be made in light of how patients, their families and physicians make their decision on whether to participate in a trial, or not.
Several tools can be used to understand how those choices are made. At Parexel, we rely on medical, therapeutic area and operational experts within our own organization, and we seek advice from external experts, opinion leaders, and experienced principal investigators and their staff within our global alliance network of 350 sites.
Engaging with investigators, patients and study site nurses who represent a critical, and often untapped reservoir of practical information and expertise provides invaluable insight. Sponsors can and should modify their protocols to address findings from surveys of these communities. In some cases, the design of a protocol may remain unchanged, but the geographic footprint of the trial can be improved.
For example, a recent Alzheimer’s disease study required a lumbar puncture (LP) procedure to obtain a cerebral spinal fluid sample (CSF). When a trial requires an invasive procedure of this kind it’s important to have deep knowledge of standards of care and cultural idiosyncrasies. Different countries have different levels of acceptance for certain interventions. In some European countries, LPs are widely accepted and routine; in other countries, they are rare and unfamiliar. If a study includes a non-negotiable burden—in this case, CSF sampling was needed to measure a biomarker hypothesized to be an indicator of disease progression—by understanding cultural standards, sponsors can choose sites in geographies that convey the best chance of recruitment success.
To gain an appreciation of the patient (and family/caregiver) perspective, Parexel routinely conducts a patient burden analysis of study designs, which involves asking questions such as:
- What are the demographics of the patients in the study and will it impact their habits? Are they elderly and therefore may have difficulty leaving home? Young adults who can travel easily? Or, small children who must be supervised by parents?
- What is life like with this disease? Do patients have frequent medical appointments and/or hospitalizations, similar to trial site visits and assessments, or very few medical interventions, making trial-related testing and procedures more out of the ordinary?
- What will it mean to undergo this procedure/intervention/assessment visit every week for 12 weeks when the patient and his or her family may be struggling with a frightening or discouraging disease diagnosis?
- How far away from sites do most patients live, and what kind of transportation obstacles might they confront?
Knowing what you don’t know is valuable in managing risk
Done correctly, analytical pressure-testing of a trial’s protocol enables sponsors to identify and eliminate unrealistic or impractical elements of the design, thereby reducing the risk of waste, missing development deadlines or outright failure. Even identifying uncertainties is worthwhile before a trial starts, giving sponsors insight as to where problems may arise and time to develop mitigation strategies.
Ultimately, feasibility testing should drive clinical decision-makers to put contingency plans in place that would not exist otherwise. And that will increase a late-stage trial’s chance of success.
