As part of the Prescription Drug User Fee Amendments of 2007 (PDUFA IV), the Food and Drug Administration (FDA) agreed to specific commitments to enhance and modernize the drug safety system. The FDA committed to identify pharmacoepidemiologic safety study best practices and to develop a guidance describing these practices. In February, the FDA published a draft guidance document for industry and FDA staff entitled, “Best practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies using Electronic Healthcare Data Sets.” The draft guidance document provides recommendations for documenting design, analysis and results of such pharmacoepidemiologic safety studies to optimize FDA’s review of protocols and final reports submitted for these types of studies. This draft guidance document fulfills FDA’s PDUFA commitment of providing best practices for pharmacoepidemiologic safety studies.

FDA Primary Goals forDraft Guidance Document

FDA has explained that the primary purpose of the guidance isthreefold:

1) provide consistent guidance for industry to use when submitting reports to FDA and protocols for pharmacoepidemiologic safety studies so study protocols and study reports contain sufficient information to permit thorough review,

2) provide a framework for FDA to use when reviewing and interpreting these submissions, and

3) provide consistent guidance for FDA to use when conducting pharmacoepidemiologic safety studies.

Note that FDA specifically did not address realtime active safety surveillance studies; the agency believes this field is still evolving and therefore, it is not currently possible to recommend best practices. Additionally, FDA did not provide a framework for determining the appropriate weight of evidence from pharmacoepidemiologic data in the overall assessment of drug safety. FDA feels that the data provided by pharmacoepidemiologic studies is a separate aspect of the regulatory decision-making process.

Background

FDA regulatory review and decision-making on drug safety issues has multiple aspects. First, evidence is reported regarding an “association” between a drug and an adverse event. Usually the evidence of this association comes about from one of the following data sources: (1) randomized controlled trials, (2) spontaneous adverse event case reports, or (3) pharmacoepidemiologic safety studies. The evidence derived from these data sources is integrated and evaluated by an FDA multidisciplinary team to make a determination about the relationship between the potential adverse effect and the drug. Based on the outcome of this analysis, FDA will determine whether regulatory action is required and whether the drug’s sponsors should be required to communicate additional important safety information to medical providers, patients, and other stakeholders.

The draft guidance document focuses on one data source in the safety assessment described above: pharmacoepidemiologic safety studies. These are observational studies designed to assess the risk attributed to drug exposure and to test pre-specified hypotheses. In observational studies, the investigator does not control the therapy but observes and evaluates the results of ongoing medical care. FDA specified in the draft guidance document that observational studies include case-control, cohort, and case-crossover studies.

The draft guidance document focuses on providing best practices specifically applicable to pharmacoepidemiologic safety studies using electronic healthcare data sets. These are organized sets of healthcare data or collections of files available via electronic format, which can be used analytically to conduct pharmacoepidemiologic safety studies. FDA recognizes the benefit of electronic healthcare data sets in that they present the possibility of studying safety issues quickly in real-world healthcare environments involving large populations of real patients. To reap the benefits of this type of safety information, the draft guidance document distills best practices into general considerations, data sources, study design, and analyses. Each section is summarized in turn below.

General Considerations

The first section directs investigators to submit protocols to FDA before study initiation and final reports upon completion of pharmacoepidemiologic safety studies. The draft guidance documents explains that each protocol and final report should contain a Study Summary. The critical elements to present in the study summary include: (1) scientific goals (i.e. study objectives), (2) study design, (3) study population and time period of study, (4) data sources, (5) drug exposures of interest, (6) drug safety outcomes of interest, (7) methods of control for sources of bias, (8) brief and balanced description of results, and (9) public health impact.

Investigators should also include background information and study approach considerations. In the draft guidance document, FDA explains the “Background” should include a brief description of prior evidence or suspicions prompting the study initiation, the strength and weaknesses of previous studies on the issue, and some general information about the therapeutic class and use of the study drug. In the “Study Approach Considerations” section, FDA emphasizes that investigators should expound on this rationale of study design, selection of databases, and analysis plan as each pertains to the pre-specified hypotheses. In addition, FDA encourages investigators to describe alternative study approaches they considered before selecting the proposed approach, and explain why the alternative study approaches were neither feasible nor optimal in answering the specific study inquiries.

The draft guidance elaborates that the protocol for pharmacoepidemiologic studies should also include “Study Team Expertise and Credentials.” This section should include a description of the expertise and credentials of the study team, including their level of experience in using specific data sources utilized in the study. FDA cited the importance of this section, emphasizing that “an experienced, balanced study team with appropriate expertise is crucial to the successful execution of a safety study.” The “Interpretation and Findings” section is intended to summarize the key results of the study including the main measure of effect. However, FDA notes that statistical significance can be easy to achieve in large electronic data sets and, alone, does not determine the importance of the finding in and of itself. Moreover, the agency reminds the audience of the importance of considering clinical significance when interpreting findings. Therefore, the confidence interval should be provided in the interpretation and findings section.

Best Practices � Data Sources

In addition to the general considerations outlined above, the draft guidance document provides best practices for data sources used in pharmacoepidemiologic safety studies. FDA does not encourage the use of one type of data source. Instead, it suggests investigators select data sources that are appropriate to address the specific hypotheses. It is important for investigators to demonstrate a complete understanding of an electronic healthcare data source. As FDA explains, this is critical because current healthcare data systems were created for purposes other than drug safety investigations, and therefore, it is important to understand the data source limitations when relied upon for drug safety investigations. Data sources from other countries can be utilized, but FDA explains it is important to describe cultural and population differences between the country and the U.S., including background information about the healthcare system, a description of prescribing and utilization practice, and an explanation of how all factors might affect study results in a U.S. population.

FDA encourages the use of explicit inclusion and exclusion criteria for study population selection, and the use of multiple data sources and populations when possible to verify, validate and replicate findings. Another issue is quality assurance (QA) and quality control (QC) procedures used by the data holders. Indeed, FDA provided six areas investigators should address in evaluating how QA and QC procedures could affect data integrity.

Lastly, investigators should define the study timeframe. FDA states that the study timeframe should “span from the beginning of the look back period, when the investigator looks back in the database before the drug exposure to ascertain baseline patient covariate data” to the end of the exposure risk window. Also, investigators must demonstrate that the timeframe they selected is appropriate to address the specific hypothesis of the study.

Study Design and Analyses

As in the Data Sources section, FDA does not endorse a specific type of study design in the draft guidance document. FDA suggests the investigator start from the study questions of interest and then determine which data sources and study design are most appropriate to address the questions. The draft guidance document also emphasizes the importance of selecting an appropriate comparator group for pharmacoepidemiologic safety studies. FDA is encouraging the use of multiple comparator groups when feasible because this can enhance the validity of the safety studies. In addition, the draft guidance document specifies that it is ideal to choose a comparator group taking a drug used to treat the same disease, with the same level of severity, for the same time period as the subjects exposed to the drug of interest for the particular study. For pharmacoepidemiologic safety studies it can be appropriate to use historical comparators, but it is important for the investigator to explain the rationale behind the comparator group and to address potential limitations of this comparator group.

The draft guidance document also emphasizes the risk of inadequately addressed confounding, which can threaten the validity of all pharmacoepidemiologic safety studies. Therefore, investigators should describe the process used to identify potential confounders and provide the scientific rationale for handling them. FDA noted that “confounding by indication or channeling” can be particularly problematic in pharmacoepidemiologic safety studies because this might lead to the appearance of an association between a drug and a safety outcome when the associationis actually due to the underlying disease for which the drugis prescribed. The draft guidance document contained much more detail regarding study design that is not summarized above. Refer to the draft guidance for more detail in selecting a study design.

In study analyses, investigators should include the pre-specified analysis plan. The plan should specify primary and secondary analyses, if any. The final analysis of the study should include both adjusted and unadjusted results. As addressed above, to avoid confounding, FDA emphasizes that investigators should ensure that appropriate statistical techniques are used to address confounding. The statistical techniques and diagnostics should be described in the analysis plan. As described in the draft guidance document, it is important that investigators develop a plan to assess and handle missing and uninterpretable data.

Finally, FDA re-emphasized that it encourages industry to inform FDA of all pharmacoepidemiologic safety studies, to submit plans and protocols for such studies before study initiation, and to submit comprehensive final reports with detailed methods and results to FDA in a timely manner. The draft guidance document outlined more specific guidelines and best practices, which are only briefly summarized above. To review the guidelines in more detail, a copy of the draft guidance can be obtained from FDA’s website (www.fda.gov).

Colleen Heisey is a partner in the Washington, D.C. office of Hunton & Williams LLP (www.hunton.com) in the Firm’s Food and Drug Practice. She can be reached at cheisey@hunton.com.Ms. Wheeler is an associate in the firm’s Food and Drug Practice.