Kristin Brooks, Contract Pharma04.13.16
Efforts to streamline drug development in today’s complex R&D landscape often include the use of technology. Quality clinical trial data is crucial in identifying promising drug candidates and eliminating failures early. As part of an effort to improve the conduct and efficiency of clinical trials, CRO Analytics and the Association of Clinical Research Professionals (ACRP) partnered to create a validated assessment for identifying and measuring key drivers of clinical trial quality from the perspective of investigator sites. The data and findings generated through the collaborative development of Site Voice, a cloud-based platform designed to gather information from clinical research sites, will be presented at the ACRP Executive Summit on Site Strategies on April 16th. Data from the development research is referenced below.
The research provides insights into the quality of clinical trials and best methods to improve them. These findings aim to help Sponsors and CROs better collaborate with investigator sites and develop partnerships by providing objective, valid and reliable data, and identifying and prioritizing specific areas for improvement.
CRO Analytics provides a validated performance data collection system for clinical trials, with access to benchmarking, key performance drivers and predictive analytics. ACRP, whose mission is to promote excellence in clinical research, is a nonprofit organization that supports clinical research professionals through membership, training and development, and certification.
Peter B, Malamis, chief executive officer and founder of CRO Analytics, discusses Site Voice and the findings, as well as challenges around cloud-based clinical operations, clinical performance optimization, and how Sponsors and CROs better collaborate with investigator sites. –KB
Contract Pharma: What is Site Voice and how does it work?
Peter Malamis: Site Voice is designed to improve patient recruitment, retention, and improve site alignment with study partners. The application captures the real-time assessments of the sites as to how the study team is performing on the study. This site perspective then serves as a type of ‘early warning system’ to identify impending failures and causes, and help sites, sponsors, and CROs target underperforming areas that lead to delays and missed milestones.
To accomplish this, data is collected from multiple site personnel via 3-5 minute online assessments during the trial. The data provides an understanding of what sponsors and CROs need to do to meet the needs of sites and study milestones.
Finally, the responses provided by sites are statistically validated through a predictive modeling methodology, led by Dr. Michael Howley. This ensures the final assessments that are a part of Site Voice collect what is most important from the sites view of clinical trial quality, and also enables benchmarking across disparate study types, as well as predictive analytics.
CP: Based on the analysis, what trends are evolving within clinical performance optimization?
PM: Our research shows two critical drivers of performance areas of interest from the site perspective: communication and the protocol. Within communication, sites believe a high quality trial is driven by providing timely information from their research partners; being available for questions, and being helpful and flexible in resolving problems. Sites want protocols that are appropriately complex, well organized, have realistic endpoints and inclusion/exclusion criteria. They also value a well-managed amendment process.
While there are other drivers of performance, communication and protocol have the greatest impact. Within each of these key drivers, remember, there are five or six other sub-drivers that are moving the needle on performance. We’re still analyzing the data from our research, but several priorities of the sites are emerging in addition to these. What’s encouraging is that these appear to track with previous research, but because of the methodology, we are able to drill-down to more actionable insights.
CP: What are main challenges around cloud-based clinical operations and how can they be addressed?
PM: Complexity and inter-operability are the challenges with cloud-based clinical operations. Sites told us that they typically have to log on to five to seven different systems for a single clinical trial. Each one of these systems has a different formatting and way they would like the information entered. This degrades productivity and efficiency of the trials. It’s also very frustrating for the site personnel.
Overall, technology is the fourth most important driver of quality from the site perspective (behind communication, the protocol, and budgeting). The research showed that to deliver a high quality trial, the technology used needed to be useful, easy to use, and have a single point of access. To the extent that cloud-based technologies can accomplish this, sites view them positively.
CP: How can Site Voice improve data quality?
PM: Think about Site Voice as a way for the sites to tell you, in real-time, all the things that are currently degrading the data quality (and also the quality of your clinical trial). This is why doing the research is so important! We have gone through to identify all of the factors that detract from quality. We have also scientifically validated that these are the factors that you need to watch. Instead of having to call all of the site personnel regularly to know what’s happening in your clinical trial, you can rely on site voice to integrate all of those perspectives and feed it to you.
The heart of our approach is the factors that impact data quality and all other functions typically tracked by measurement of time and quantity. Whether it’s poor data quality or some other underperforming operational metric, the question that must be asked is “why”? Where did the breakdown occur? Was it the result of the protocol itself in some way? Training? Communication? Site Voice helps identify performance relative to these core elements. Site Voice enables faster root cause analysis to get at the “why” of underperformance and lead you to the “what” and “how” with regard to improving it. Our research identified the critical Quality Performance Indicators (QPI) of overall trial performance but also measures those services and skills impacting all trial functions.
CP: In helping Sponsors and CROs better collaborate with investigator sites and site networks, what specific areas need improvement?
PM: The issues breakdown into five major areas:
· Protocol – sites want ‘mature’ protocols. In other words, they need to be able to plan for the study, construct a budget, and even estimate a screen failure rate from the protocol. When the protocol is dramatically changed, they should have a budget for the re-training and re-work necessary.
· Budgeting – this one is easy: Sites just want to be paid fairly for their work.
· Initiation – sites need information and preparation in order to function properly. They need good information and people who understand the study to explain it to them. Get them their supplies on-time.
· Monitors/CRAs – sites want monitors who are facilitators, not auditors. They want people who can help them conduct a better study.
· Queries and Closeout – queries should be reasonable and resolved fairly. Sites need easy and convenient closeout procedures and drug reconciliations.
Our research shows there is more than enough room for improvement in very specific, critical areas, Furthermore, it shows that CROs have more ground to make up than sponsors. Specifically, we identified five QPI from the viewpoint of sites: Communication, Protocol, Budget/Contracting, Technology, and Monitors. In other words, these areas are most directly linked to high quality trials from the site perspective. These are where sponsors and CROs have to excel if they are going to improve performance. On 1-10 scale with ten being the highest, the score low score was 6.6 while the highest was 7.7. In addition, in every single area, CROs scored lower than sponsors. In some cases, substantially lower. (See Charts 1-3 below)
For more than two decades, Peter Malamis has strategically guided the growth and development of service firms supporting biopharma clinical research and commercialization. Most recently, as Vice President and General Manager at the MAPI Group, he helped lead the acquisition and integration of Registrat, Inc. in a transaction that created the largest global CRO exclusively devoted to post-marketing research. Previous to MAPI, as President of US Operations for the CRO Premier Research, Mr. Malamis led that group to the highest growth rates and profitability it had ever achieved. Prior to Premier, he held executive positions at SCIREX, Galt Associates, MEDTAP International, PAREXEL International, and State & Federal Associates.
The research provides insights into the quality of clinical trials and best methods to improve them. These findings aim to help Sponsors and CROs better collaborate with investigator sites and develop partnerships by providing objective, valid and reliable data, and identifying and prioritizing specific areas for improvement.
CRO Analytics provides a validated performance data collection system for clinical trials, with access to benchmarking, key performance drivers and predictive analytics. ACRP, whose mission is to promote excellence in clinical research, is a nonprofit organization that supports clinical research professionals through membership, training and development, and certification.
Peter B, Malamis, chief executive officer and founder of CRO Analytics, discusses Site Voice and the findings, as well as challenges around cloud-based clinical operations, clinical performance optimization, and how Sponsors and CROs better collaborate with investigator sites. –KB
Contract Pharma: What is Site Voice and how does it work?
Peter Malamis: Site Voice is designed to improve patient recruitment, retention, and improve site alignment with study partners. The application captures the real-time assessments of the sites as to how the study team is performing on the study. This site perspective then serves as a type of ‘early warning system’ to identify impending failures and causes, and help sites, sponsors, and CROs target underperforming areas that lead to delays and missed milestones.
To accomplish this, data is collected from multiple site personnel via 3-5 minute online assessments during the trial. The data provides an understanding of what sponsors and CROs need to do to meet the needs of sites and study milestones.
Finally, the responses provided by sites are statistically validated through a predictive modeling methodology, led by Dr. Michael Howley. This ensures the final assessments that are a part of Site Voice collect what is most important from the sites view of clinical trial quality, and also enables benchmarking across disparate study types, as well as predictive analytics.
CP: Based on the analysis, what trends are evolving within clinical performance optimization?
PM: Our research shows two critical drivers of performance areas of interest from the site perspective: communication and the protocol. Within communication, sites believe a high quality trial is driven by providing timely information from their research partners; being available for questions, and being helpful and flexible in resolving problems. Sites want protocols that are appropriately complex, well organized, have realistic endpoints and inclusion/exclusion criteria. They also value a well-managed amendment process.
While there are other drivers of performance, communication and protocol have the greatest impact. Within each of these key drivers, remember, there are five or six other sub-drivers that are moving the needle on performance. We’re still analyzing the data from our research, but several priorities of the sites are emerging in addition to these. What’s encouraging is that these appear to track with previous research, but because of the methodology, we are able to drill-down to more actionable insights.
CP: What are main challenges around cloud-based clinical operations and how can they be addressed?
PM: Complexity and inter-operability are the challenges with cloud-based clinical operations. Sites told us that they typically have to log on to five to seven different systems for a single clinical trial. Each one of these systems has a different formatting and way they would like the information entered. This degrades productivity and efficiency of the trials. It’s also very frustrating for the site personnel.
Overall, technology is the fourth most important driver of quality from the site perspective (behind communication, the protocol, and budgeting). The research showed that to deliver a high quality trial, the technology used needed to be useful, easy to use, and have a single point of access. To the extent that cloud-based technologies can accomplish this, sites view them positively.
CP: How can Site Voice improve data quality?
PM: Think about Site Voice as a way for the sites to tell you, in real-time, all the things that are currently degrading the data quality (and also the quality of your clinical trial). This is why doing the research is so important! We have gone through to identify all of the factors that detract from quality. We have also scientifically validated that these are the factors that you need to watch. Instead of having to call all of the site personnel regularly to know what’s happening in your clinical trial, you can rely on site voice to integrate all of those perspectives and feed it to you.
The heart of our approach is the factors that impact data quality and all other functions typically tracked by measurement of time and quantity. Whether it’s poor data quality or some other underperforming operational metric, the question that must be asked is “why”? Where did the breakdown occur? Was it the result of the protocol itself in some way? Training? Communication? Site Voice helps identify performance relative to these core elements. Site Voice enables faster root cause analysis to get at the “why” of underperformance and lead you to the “what” and “how” with regard to improving it. Our research identified the critical Quality Performance Indicators (QPI) of overall trial performance but also measures those services and skills impacting all trial functions.
CP: In helping Sponsors and CROs better collaborate with investigator sites and site networks, what specific areas need improvement?
PM: The issues breakdown into five major areas:
· Protocol – sites want ‘mature’ protocols. In other words, they need to be able to plan for the study, construct a budget, and even estimate a screen failure rate from the protocol. When the protocol is dramatically changed, they should have a budget for the re-training and re-work necessary.
· Budgeting – this one is easy: Sites just want to be paid fairly for their work.
· Initiation – sites need information and preparation in order to function properly. They need good information and people who understand the study to explain it to them. Get them their supplies on-time.
· Monitors/CRAs – sites want monitors who are facilitators, not auditors. They want people who can help them conduct a better study.
· Queries and Closeout – queries should be reasonable and resolved fairly. Sites need easy and convenient closeout procedures and drug reconciliations.
Our research shows there is more than enough room for improvement in very specific, critical areas, Furthermore, it shows that CROs have more ground to make up than sponsors. Specifically, we identified five QPI from the viewpoint of sites: Communication, Protocol, Budget/Contracting, Technology, and Monitors. In other words, these areas are most directly linked to high quality trials from the site perspective. These are where sponsors and CROs have to excel if they are going to improve performance. On 1-10 scale with ten being the highest, the score low score was 6.6 while the highest was 7.7. In addition, in every single area, CROs scored lower than sponsors. In some cases, substantially lower. (See Charts 1-3 below)
For more than two decades, Peter Malamis has strategically guided the growth and development of service firms supporting biopharma clinical research and commercialization. Most recently, as Vice President and General Manager at the MAPI Group, he helped lead the acquisition and integration of Registrat, Inc. in a transaction that created the largest global CRO exclusively devoted to post-marketing research. Previous to MAPI, as President of US Operations for the CRO Premier Research, Mr. Malamis led that group to the highest growth rates and profitability it had ever achieved. Prior to Premier, he held executive positions at SCIREX, Galt Associates, MEDTAP International, PAREXEL International, and State & Federal Associates.