For the past several years, I have participated in a continuing education course on preclinical study monitoring at the annual meeting of the American College of Toxicology. My contribution to this popular offering primarily consists of sharing some of my real-life experiences of when things haven’t gone as planned on preclinical studies. The whole idea in the sharing these experiences is to raise the awareness of the course participants so that, hopefully, they can watch for similar situations in their own preclinical studies and intervene before a concern grows into a significant problem.

During my presentation, we discuss issues ranging from unanticipated toxicology findings to regulatory issues to communication lapses to operational failures. For example, one of my favorite experiences to share is about the time at a CRO when my host could not find the room that contained the study that I was asked to monitor. I probably only increased his embarrassment when I asked, “So, do you visit the room often?” Either this individual was directionally challenged or we were both visiting the study room for the first time. Needless to say, the experience didn’t leave me feeling full of confidence at the start of my day.

Let me make some disclaimers. I actually like CROs. I like the people in CROs. I respect those who work in CROs. I have good friends who work in CROs. Over the years, some have suggested that I focus too much on what goes wrong and don’t spend enough time on what goes right at CROs. I readily acknowledge that much of the time, preclinical studies are at CROs are done in accordance with the study protocol and no issues are encountered at all. One of the downsides of this business, whether the preclinical research is conducted at a pharma company or at a CRO, is that operational excellence is seemingly seldom acknowledged, while people seem to come out of the woodwork to point fingers and express their opinions when something goes wrong operationally. In some organization, the operations staff may feel pressured when unexplained toxicity occurs. The possibility of human error must be considered and excluded as a root cause when trying to understand unanticipated toxicology findings. Unfortunately, operational errors do occur, infrastructure issues may arise, and periodically, regulatory issues are encountered.

The likelihood of encountering these issues may be greater at some CROs than at others based on historical performance, but at the end of the day, they can occur in any preclinical operation at any time. My job as a consultant is to mitigate risk. For more than 30 years, I have seen what can go wrong in tox studies and I try to work with CROs to make sure that these same issues don’t happen on my sponsors’ studies. I accept the pushback from the CRO industry that study-related issues don’t happen frequently but, as I told one individual, I would stop writing about operational concerns if they stopped happening. In the end, sponsors that experience operational issues on a study don’t care that this was “a rare occurrence”; they just know that their studies were screwed up and they want to know as soon as possible what the CRO is going to do to make it right. Hopefully, the sponsor will not have to repeat their study. Repeating studies costs time and money. The challenge in trying to mitigate risk is that you never know what you are going to encounter. For example . . .

In July, the FDA recently sent a letter to clients of the Houston, TX-based bioanalytical lab of preclinical CRO Cetero Research, informing them that data from bioequivalence and bioavailability studies conducted between April 2005 and June 2010 may have been falsified. According to the agency, “the pattern of misconduct was serious enough to raise concerns about the integrity of the data Cetero generated during the five-year time frame.” As a result, clients who use that data for pending NDAs or ANDAs may have to re-do testing.

In a further statement, the FDA noted, “It is unlikely that these concerns relating to data integrity will affect the overall safety and efficacy of drugs already on the market. FDA is taking this precautionary measure to make sure the data underlying the approval decision is completely reliable. At this time, there is no evidence of problems with the safety, quality, purity or potency of drugs already approved and marketed.”

Cetero’s management reacted strongly to the letters and press releases from the FDA. In a press statement, management noted that the company “remains fully committed to maintaining the quality and integrity of the data collected in each of its facilities, including the Houston, TX, bioanalytical laboratory.” The FDA’s moves were “difficult to understand,” the statement added. Also,

Cetero initiated its own internal investigation of its Houston bioanalytical laboratory over two years ago when it discovered the recording of inaccurate day/time data by a small number of research chemists in its Houston facility. Cetero proactively contacted the FDA to self-report its preliminary findings, as well as seek agency feedback on its comprehensive investigation plan. At that time, Cetero clients were also contacted to make them aware of the situation. The Untitled Letter does not accept the results of our rigorous scientific analysis and discredits the Company’s 1,200 dedicated and experienced employees. The research conducted on behalf of our pharmaceutical sponsors can be, and has been, properly validated.

(A PDF of Cetero’s full statement can be found here and the FDA’s notices can be found here.)

Okay, here’s my unsolicited commentary. First, the FDA did not discredit Cetero’s 1,200 employees. In my opinion, those Cetero employees who falsified data discredited the company and the site management team, which is a shame because I am sure the vast majority are good, honest people. Secondly, the “it was those few bad apples not the rest of us” argument diminishes the public commitment for accepting full and unyielding responsibility for significant regulatory non-compliance. Third, you can blame the FDA all you want, but it was the CRO’s employees that caused this problem and, in the end, you are not likely to win a war of words with the FDA. 

Finally, let me just say that research operations are more complex than many realize. Press releases that are authored by those without operational background may have good soundbites, but they often lack credibility among those who do understand research operations and FDA interactions. I can’t say that the CRO’s response was the work of a PR agency, but I implore any CRO involved in an issue like this to make sure the operational perspective is out in front.

I think it is fair to say that what has been alleged above represents a sponsor’s worst nightmare. As far as we know, instances of data falsification and manipulation are very rare occurrences, but those sponsors that may have to repeat studies don’t care about probabilities.

What does all of this mean? Incredibly, approximately 30 years after the FDA’s Good Laboratory Regulations were put in place to improve the quality and reliability of preclinical data, we are still seeing an instance of data falsification and manipulation. Why? Although technology has improved dramatically and we now have years of regulatory experience, preclinical research is and always will be conducted by humans. Contrary to the aspirations of some people, we are not perfect. In the world of preclinical research, people are the ultimate risk factor! Volumes of SOPs, elaborate processes, complex study protocols and volumes of training documentation mean nothing if an employee in a pharmaceutical company or a CRO chooses not to follow established procedures.

While the aforementioned example of data falsification represents a seeming intentional divergence from operational procedures, other circumstances can impact data quality as well. Accordingly, sponsors and managers of preclinical operations should consider that as long as humans conduct research, there is no such thing as a truly controlled experiment. That concept may be hard to accept but doing so may actually set the stage for a more successful outsourcing experience. Beyond the science, processes, regulations and company rules, what can sponsors and research managers do to mitigate risk and diminish the chance of operation errors? Here are a few tips:

• Listen: Those that conduct preclinical research may understand operations better than their managers. These individuals may sense operational challenges before they happen. If you are a sponsor or a manager, listening to and acknowledging the recommendations of your staff may improve your operations.

• Watch: If you are a sponsor, try to assess the quality of the operational work environment. Does the staff feel like they matter? Do functional areas work together as a team? How is the morale in the workplace? Research managers can ask themselves the same questions but they need to also consider if their assessment will be unbiased. A research operation with low morale or poor cooperation between functional areas is breeding ground for quality issues.

• Get to know who is conducting your research: If you are a sponsor, spend some time building relationships with those who are working on your studies. This is especially important on longer-term studies. Don’t hesitate to share information about the drug and how it could benefit patients. It has been my experience that technical staff are even more diligent if you show your appreciation for their effort and ask for their feedback about your study.

• Understand weekend work responsibilities: Weekends represent prime opportunities for operational errors because it is likely that less familiar technical staff will be responsible for study activities. Ask how essential study information will be conveyed to weekend workers. Will the study director be available for questions? What are the policies for reporting operational issues on weekends?

• Nurture relationships with other functional areas: Does the veterinary staff understand the study design and potential adverse effects? Was a veterinary intervention plan discussed before the study started? How is the relationship with the Quality Assurance group? There is nothing wrong with a civil discussion about regulatory interpretation, but after 30 years, battles between operational areas and QA, to me, are suggestive of an organizational leadership problem. Furthermore, an antagonistic relationship with Quality Assurance could suggest to the operational staff that it is acceptable to not fully follow established procedures.

• Perform due diligence: I once had a scientist from a biotechnology company tell me that he assumed that operational quality was a given when he was deciding where to place their studies. As we saw above, just because procedures and regulations are in place, that doesn’t necessarily mean they are always followed. If you are assessing a research operation prior to or during the conduct of your studies, any concerns that you find may ultimately determine whether your outsourcing experience is successful or not. If you are a research manager, trusting your operation staff is critical, but it never hurts to test your own operations to make sure that they are as effective as you think they are. Organizational changes, outdated processes, and overconfidence are the seeds of future operational issues. After two years of slow demand and cutting costs at CROs, it may now be more critical for sponsors to conduct due diligence and study monitoring to help the CRO staff catch operational issues before they occur.

While many will tout organizational experience, most research managers do acknowledge that operational errors will occasionally occur. They understand that people are the ultimate risk factor and maybe they also understand that preclinical research operations are never truly under control. Preclinical operational errors can occur at any time at any place despite best intentions. In the end, I will reiterate again that this is a relationship-based business. The best risk mitigation strategy is to truly partner with your CRO and maintain active involvement throughout the outsourcing experience.

Steve Snyder is a consultant with more than 25 years of experience in preclinical toxicology as an outsourcing customer and provider. He can be contacted at info@outsource-support.com.