The FDA has announced its intention to standardize quality metrics as one tool, but not the only tool, for prioritizing levels of effort and scrutiny for drug manufacturers. The FDA wishes to use standard metrics to compare relative quality results between similar establishments, companies, and products.

This has been driven in part by resource needs to address issues arising from: Drug shortages; complex supply chains; proliferation of foreign manufacturing; aging facilities; counterfeiting; and a whole host of other issues that may impact quality of the supply chain of a drug.

So what problem is the FDA really trying to solve? In October 2005 Janet Woodcock of the FDA stated that a “maximally efficient agile flexible pharmaceutical manufacturing sector that reliably produces high-quality drug products without extensive regulatory oversight” is the goal.

However, since she made that statement issues such as data integrity, supply-chain integrity drug shortages and an increase in the number of lagging key indicators such as complaints and recalls have occurred. Now, these may be a result of greater effectivity in reporting, or they can be a true increase in the overall risk of the drug product portfolio. In any case, these are all driving a number of changes within the FDA as well as other health authorities around the world.

FDA Quality Metrics Authority
The 2012 FDASIA act, section 711 added text to section 501 of the FD&C Act clarifying that, for the purposes of paragraph 501(a)(2)(B), the term “current good manufacturing practice” includes the implementation of oversight and controls over the manufacture of drugs to ensure quality, including managing the risk of an establishing the safety of raw materials, materials used in the manufacturing of drugs, and finished drug products. Section 706 of FDASIA also added a new subparagraph to Section 704(a) of the FDCA which states that: “Any records or other information that the Secretary may inspect under [Section 704] from a person that owns or operates an establishment that is engaged in the manufacture, preparation, propagation, compounding, or processing of a drug shall, upon the request of the Secretary, be provided to the Secretary by such person, in advance of or in lieu of an inspection, within a reasonable timeframe, within reasonable limits, and in a reasonable manner, and in either electronic or physical form, at the expense of such person.”

While section III of the new guidance provides significant details on the FDAs legal authority to request and use quality metrics, it is important to note thar organizations have always been required to: Understand the sources of variation; detect the presence and degree of variation; understand the impact of variation on the process and ultimately on product attributes; and control the variation in a manner commensurate with the risk it represents to the process and product.

The quality metrics initiative is the FDA’s attempt to provide formality and means to utilize this data in amore effective manner.

What are quality metrics?
Quality metrics are simply an objective measure of the quality of the product or process. A quality metric allows a user to quantify the quality of a specific aspect of the product or process by comparing it to a criteria or predetermined value, or to similar products, process or sites.

Importance of quality metrics and risk management
By now we all know the definition of risk. Risk is a combination of the severity of harm and the probability of occurrence of that harm. Quality metrics are an inherent component in our industry as an evaluation of the overall risk profile of its products and processes and manufacturing sites. By segmenting product types and indications and getting access to occurrence rate type of data, the overall risk profile will become much clearer to the agency. This clarity will provide the framework for more objective audits and compliance programs as well as application of agency resources to reduce the potential for drugs shortages and other patient impact type events.

The use of quality metrics in industry
Quality metrics are used in other industries as a means to focus energies in places that will be able to apply the biggest bang for the buck. Most manufacturing operations already capture many of the metrics the FDA wishes to have reported. Within the industry, these tools are being used in a very similar manner for internal and external industry audit programs as the FDA wishes to apply to manufacturers. So, the content should not be unfamiliar. Additionally, quality metrics are being utilized on the patient side to improve the patient experience as well as overall healthcare outcomes. The U.S. Department of Health and Human Services has provided a potential list of metrics for providers to consider as they implement IT infrastructure and quality-improvement programs. Specifically, “each metric helps provide a more complete picture of the health care quality status of an organization.”

There are different types of quality metrics:

• Access measure assesses the patient’s ability to obtain timely and appropriate health care;
• Outcome measure is the patient’s health status after receiving health care services. It can be used to evaluate the quality of care to the extent that health care services influence the likelihood of desired health outcomes;
• Patient experience measure aggregates reports of patients about their observations of and participation in health care;
• Process measure assesses the actual health care service provided to, or on behalf of, a patient; and  
• Structure measure describes a feature of a health care organization or clinician relevant to its capacity to provide health care (e.g., nurse to patient ratio, number of beds).

FDA Quality Metrics initiative
The FDA’s intention with regards to quality metrics is fairly straightforward in its objectives, but perhaps not quite yet in design. Per the FDA’s Russell Wysdyk, one of the key goals of the quality metrics initiative is to keep the program simple and to use it for improved objectivity in inspections. The FDA is looking for an objective measure of the quality of the product or process where quality is the fitness for intended use. The FDA also wishes to have an objective measure of the quality of the site as well as an objective measurement of the effectiveness of the quality systems that are used to manufacture pharmaceutical products. The FDA would capture data from industry and then calculate metrics from what is calculated. On paper and as written it is hard to argue those points.

Who is covered?                   
Per the current July 2015 draft, the following groups are covered as a part of the quality metrics initiate: Owners and operators that are required to register under section 510 and are engaged in the processing, preparation, propagation, compound in or processing of a cover drug product or API used in the manufacturing of a drug product. This includes, contract labs, contract sterilizers and contract packagers.

Quality Metrics Reporting
What does the FDA wish to be reported? A one-year period of data reported thought the FDA Electronic Gateway for the following 10 objective measures are being proposed:

• The number of lots attempted of the product;
• The number of specification-related rejected lots of the product, rejected during or after manufacturing;
• The number of attempted lots pending disposition for more than 30 days;
• The number of out-of-specification (OOS) results for the product, including stability testing;
• The number of lot release and stability tests conducted for the product;
• The number of OOS results for lot release and stability tests for the product that is invalidated due to lab error;
• The number of product quality complaints received for the product;
• The number of lots attempted which are released for distribution or for the next stage of manufacturing the product;
• If the associated annual product reviews (APRs) or product quality reviews (PQRs) were completed within 30 days of annual due date for the product; and
• The number of APRs or PQRs required for the product.

What happens if your organization does not report?
Since the quality metrics initiative is in draft form requirements for submission are not required at this time. But that doesn’t mean that organizations shouldn’t be engaged in Joppa Dallas’s activities in the future planning of these. Especially for contract manufacturing organizations (CMOs). These companies will have multiple clients and owners coming to them with potentially different requests. Getting ahead of the curve on this is incredibly important. And when the initiatives does go live, failure to report requested data may push the organizations risk level into the FDA’s prioritization of inspections and may lead to an earlier inspection. Worse yet, products associated with that site may be deemed adulterated under section 501 of the FDCA.

Industry Feedback
Many industry organizations provided support and engagements to the FDA prior to and after the July 2015 publication of the Request for Quality Metrics draft guidance document. A public meeting was held August 24, 2015 to, in the words of the FDA, “Gain stakeholders perspective on various aspects of the development and planned implementation of a quality metrics program.” (Recordings, industry presentations, and speaker list for the meeting can be found here at www.fda.gov/Drugs/NewsEvents/ucm456083.htm)

There is an agreement from the industry association group that worked with the FDA to to try and develop workable metrics. The consensus from the group is that these metrics should be:

• Universal—able to be used by many diverse, global, drug manufacturers with varying drug substance and drug product technologies and dosage forms;
• Not overly burdensome—not take resources away from existing quality related activities and focus, they should be things the industry is all ready measuring;
• Be useful—effective in meeting the objective of determining quality performance;
• Be quantifiable and objective—avoid potential for individual interpretation and bias;
• Be difficult to game or unreasonably manipulate—in other words we did not want people to just work to the metric ignoring the overall objective of the achievement of product quality; and
• Do not create unintended consequences—focusing on quality metrics for the sake of collecting quality metrics data is not useful for the FDA, the industry or the patient population.

Groups such as the PDA and ISPE have advocated for a reduced list as pilot. The following were suggested:

• Confirmed Out of Specification (OOS) Rate for each product, including both drug substances and drug products, calculated as total number of confirmed OOS results per total number of specification tests;
• Recall Rate by Product, calculated as number of recalls per total drug product lots. FDA would need to standardize the enumeration process for counting recalls; and
• OOS Rate by Site, calculated as the rate of confirmed OOS across all products produced at a manufacturing site.

These were suggested, as universal definitions exist for the above. Most of the industry groups suggested additional clarity and definitions some of the data requested. This is a fair point. If sites and product types are to be compared, the manner in which the data is defined and calculated needs to be common, or the end result is useless.

Industry also suggested trending of the data, perhaps as a part of Stage CPV reporting and analysis. The goals of both programs are quite similar and there are potential overlaps in the types of data collected. 

Impact
Impact to industry could potentially be minimal. Organizations that already have mature collection of data that is already required as a part of regulations have a leg up on other organizations that do not. Combining the requirements for quality metrics with existing APR and stage III requirements could reduce the overall effort on all of these requirements. Smart application of effort good reporting structure will also be important in implementation. However, organizations that currently do not have a good handle on these will feel a significant impact. 

References

• Quality Metrics Initiative.  Contract Pharma Contracting and Outsourcing Conference New Brunswick, NJ Dr Mike Long 18 Sept 2015
• Quality Metrics – PDA FDA Conference Applied Statistics Interest Group Presentation, 28 September 2015
• FDA Guidance Document “ Request for Quality Metrics” July 2015
• FDA Quality Metrics Public Meeting. FDA and Industry presentation http://www.fda.gov/Drugs/NewsEvents/ucm456083.htm

---

Dr. Mike Long, is senior director of consulting services at ValSource. He can be reached at mlong@valsource.com.