The Analytical Method Control Strategy

When testing Quality Control (QC) laboratory analytical samples the goal is to determine with the greatest degree of accuracy and precision the chemical and physical attributes of that sample. It is understood that there is an inverse relationship between the number of analytical sample measurements/preparation and the uncertainty of measurement in that, as the number of independent, replicate measurements and preparations increases, the standard error of the mean (associated with the analytical method) decreases. Therefore, one approach to reducing the measurement of uncertainty for quantitative analysis is to generate multiple sample measurements which can be achieved through either (or a combination thereof) multiple sample preparations (to account for variability associated with sample preparation, e.g., extraction, dilution, digestion) or multiple measurements of the same sample solution (accounting for variability associated with instrument precision). 

NIST Technical Note 1297, 1994 edition¹ references the combined standard uncertainty of measurement U_c which combines the individual standard uncertainties whether arising from randomized or systematic (non-random) sources (within the NIST Technical note this is referred to as the law of propagation of uncertainty). Taking such an approach the relationship between uncertainty of measurement and standard error of mean and bias uncertainty for an analytical method can be represented as follows:

Standard error of the mean (SEM) = StdDev (SD) for Sample Measurement / SQRT n (where n = sample size)

Standard combined uncertainty U_c = SQRT (SEM² + b²) where b is the uncertainty associated with bias measurement. 

Expanded uncertainty U = k U_c where k refers to a coverage factor  

When considering the above, it should be recognized that commonly the uncertainty of measurement is predominately driven by the SEM. As for the bias, it is only the uncertainty associated with the bias measurement that contributes U_c (if the accounts for any systematic, bias effect) If the method does not account for the bias, then U will need to consider the bias value along with the associated uncertainty of bias measurement. 

If one attempted to reduce the SEM by increasing the sample size (n) then the natural question is how many preparations/measurements should be made for my analytical method? The answer to such a question should be driven by the requirements of the analytical method which can be defined within the analytical methods target profile (ATP). A key component of the ATP is the uncertainty of measurement (U) where the ATP should define the requirements of U (which will consider the specification that the sample is being tested against). 

For the bias component of (U) this should be ascertained and quantified through method development (as per ICHQ14) where Analytical Quality by Design (AQbD) will identify/characterize those method parameters/settings that impact bias along with defining the elements of the Analytical Control Strategy (ACS) to reduce their influence to an acceptable level. Examples of such measures would be optimizing sample preparation conditions to ensure quantitative recovery of the analyte, minimizing matrix influence etc. Analytical development should define the ACS where Analytical Procedure Performance Qualification (APPQ) will confirm the ACS suitability (by testing the ACS against protocol defined acceptance criteria for the various analytical method attributes) and ongoing suitability of the ACS demonstrated through continuous performance verification (CPV). Analytical development along with APPQ and CPV represents Analytical Lifecycle Management (as per USP<1220>) which also includes change management where any change to an analytical method would require assessment of the potential impact to effectiveness of the ACS (recognizing those elements of the ACS that are established conditions (EC) necessitating regulatory notification for any change). For example, a change to an analytical method’s reference standard lot number would commonly be associated with a bridging/comparability study to assess any impact to the ACS. 

In situations where the ACS has minimized any influence of bias, which could be addressed through a correction within the method (when bias has been demonstrated to be consistent through the method’s reporting range), the expanded measurement of uncertainty primary contributor is the method’s randomized error. Analytical method development would need to identify those impactful method settings/parameters which impact the analytical method’s randomized error and define the associated ACS to minimize their influence. This would include establishing the number of sample preparations and measurements to achieve an acceptable level of SEM along with those other elements of the ACS to control systematic and random errors such as those associated with instrument performance, operator technique, environmental fluctuations and sample preparation variability. An example of an element within an ACS to mitigate the impact of error of measurement associated with instrument performance, sample preparation etc., would be the incorporation of an internal standard into a method. Statistical tools such as ANOVA can be utilized to partition the total variability associated with an analytical method into components, allowing an understanding of what components have the greatest contribution to the pooled sample standard deviation, recognizing that method precision is normally evaluated at three levels: repeatability, intermediate precision and reproducibility.

So, returning to the question of how many sample preparations/measurements are needed for an analytical method is obviously case dependent. Ultimately, the aim is to drive the analytical method’s U to a level that meets the requirements of the ATP, which in turn is driven by the Quality Target Product Profile (QTPP). Increasing sample size (n) is one approach to reduce SEM, however, it must be recognized that there are limitations/practical considerations as you are reducing the SEM by the SQRT of n and that you would also want to reduce the StdDev associated with the Sample Measurement. With that in mind, Analytical Lifecycle Management (ALM) needs to achieve a complete understanding of the analytical method bias and SD influences to ensure the analytical control strategy is targeting those that are most impactful to achieving the ATP requirements. The ALM program needs to be risk based where, via knowledge management, an ACS is afforded which focuses on the most significant method bias/SD influencers. For example, it would be inappropriate to address an unacceptable level of SEM by increasing the number of sample preparations/measurements to an impractical level when a more significant and effective reduction could be afforded by incorporating an internal standard or increasing sample load/volume to reduce the SD. 

A component of ALM is to continually monitor the ongoing effectiveness of the ACS through CPV which can include a periodic holistic assessment of those investigations that were associated with the execution of the test procedure. Such test method investigations can be associated with those that were initiated due to exceeding test method defined criteria (e.g., agreement between multiple sample preparations). Such test method criteria will be established through method development and PPQ, that when exceeded, reflect an atypical outcome necessitating an investigation. In addition to test method defined criteria there will commonly be method system suitability criteria (from the analysis of reference standard and/or system suitability solutions) which together represent a key component of the method’s ACS. Any resulting investigation will include a laboratory component, but for scenarios where the sample agreement criteria was exceeded, a manufacturing component of the investigation could be required (if a laboratory root cause was ruled out indicating a potential manufacturing cause). This will be particularly important if one of the sample preparations is Out of Specification (OOS), which will be discussed in the next paragraph. The goal of such investigations is to determine if the root cause is associated with the respective control strategies: for the analytical method, the ACS, but also the manufacturing control strategy (if it was suspected that the lack of agreement between multiple preparations was deemed reflective of the quality of the sample that was tested). Ultimately, the aim through CPV is to ensure an effective control strategy is in operation that delivers against the requirements of the ATP and QTPP. 

A dilemma that faces a QC laboratory is “if my analytical method requires multiple preparations/multiple measurements (to minimize error of measurement), what action do I take if one of those preparations/results are outside of the specification (OOS) but then the reported result (which is based upon an average) is within specification?” Within the FDA’s May 2022 OOS guidance (Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production),  there is reference to two different scenarios, where the first addresses averaging results from multiple sample preparations from the original submitted analytical sample and then the second scenario addressing averaging results from the same final sample preparation. For the former scenario the OOS guidance states the following:

“….where a series of assay results (intended to produce a single reportable result) are required by the test procedure and some of the individual results are OOS, some are within specification, and all are within the known variability of the method, the passing results are no more likely to represent the true value for the sample than the OOS results.  For this reason, a firm should err on the side of caution and treat the average of these values as an OOS result, even if that average is within specification.”²  

An important point to note with the above excerpt (and as discussed earlier), there needs to be an understanding of the known variability of the method and that this is reflected in the analytical control strategy such that if the results are not within the expected variability, an investigation should still be initiated (even if all are within specification) and that would be addressed via a test method (sample preparation agreement) criterion. The resulting investigation will focus on determining if the cause for exceeding the test method criterion reflects either (or both) an issue with the effectiveness of the analytical or process control strategy. If an OOS result is also associated with exceeding the sample agreement criterion, the investigation should determine if the cause for exceeding the agreement is also associated with the cause for the OOS. 

For the latter scenario the OOS guidance states:

“….an HPLC test method may specify both acceptance criteria for variability and that a single reportable result be determined by averaging the peak response from a number of consecutive, replicate injections from the same test vial.  In these cases, and given the acceptance criteria for variability are met, the result of any individual replicate in and of itself should not cause the reportable result to be OOS.”² 

The above excerpt is stating that the method is averaging the injection responses to derive a reported result for that sample preparation, and that if the effectiveness of the analytical control strategy has been demonstrated, then the reported result is valid. So again, in such situations the analytical control strategy should include a criterion for the expected variability of multiple measurements from the same sample preparation as determined through method development/PPQ. If this is exceeded, then an investigation will be initiated. If the variability criteria were met but one of the injections responses represented an OOS result, then it is highly likely that the reported result is out of trend (OOT) as the U for that result would overlap with the regulatory specification reflecting a false negative/positive risk which would necessitate an investigation. This will be discussed further below.

It is common to ask what is considered an acceptable level for U. Acceptability is normally based on assessing the level of U in relation to the specification range for that analyte and determining the % value of U in relation to that specification range. Commonly a value of 12.5% of the specification range is used as a rule of thumb, but caution should be applied with taking such an approach. 

U = (Q^max – Q^min) / 8 where Q^max and Q^min represent the material specification range

When defining the acceptability of the method’s U, one must reflect on the risk of making an incorrect quality decision from the testing of the manufacturing sample due to the method’s U. As such, when establishing the U in the context of the ATP, there needs to be an assessment of the risk that the analytical method will generate a false positive or negative result during production support. This goes back to the earlier comment of understanding the production process capability. In Figure 1 below, scenarios 2 and 3 represent a false positive/negative risk.³

To assess such a risk, one needs to understand the method’s U, but as discussed earlier, there also needs to be an understanding of the expected capability of the manufacturing process towards that attribute. For those manufacturing processes that are less capable (where the quality of the generated material rides close to the specification), there is more emphasis on developing analytical methods with lower U value to minimize the risk that when testing the manufacturing sample, the generated result’s U value overlaps with the specification (representing a false positive/negative risk). To minimize the risk of making an incorrect quality decision, i.e., incorrect batch disposition decision, companies will commonly implement internal specifications (such as OOT limits that were mentioned earlier) which will prompt further action such as initiating an investigation. This is represented below in Figure 2 as the transition zones.⁴

 It is recognized that QC tests a submitted analytical sample as it is not practical to test the parent batch/lot in its entirety, and as such, one of the primary requirements of the submitted sample is to represent (as closely as possible) the parent batch/lot (for the attributes that are to be tested). With that in mind, there needs to be an understanding of how that attribute is dispersed through the parent batch and thus the ease of obtaining a representative sample for that attribute (and any associated risk with the sampling approach). If it is an attribute where there is a risk of it being heterogeneously distributed, for example, an impurity that is formed during lyophilization, then obtaining a representative sample is more of a challenge versus sampling for a homogeneous attribute. In recognition of that risk, when sampling for a heterogeneous attribute, one can employ stratified sampling along with an increased sample size (versus sampling of a homogenous attribute). Such sampling approach needs to be reflected in the associated analytical method, for example, where one would commonly test the representative sample from each stratified sampling point to assess the uniformity of distribution. This becomes particularly relevant for the testing of PPQ batches where it is recognized that a higher level of sampling commonly occurs (versus routine production), where one of the goals is to establish the capability of the associated process control strategy. After the PPQ batches, it may be possible to justify a lower level of sampling where, for example, testing is limited to the worse case sampling sites, and the focus of that testing is to confirm the ongoing suitability of the process control strategy. A word of caution regarding the testing of composite samples—this needs to be justified with consideration of the nature of the attribute which includes demonstrating that the attribute is homogenously distributed. Obviously, if the sampling plan is to demonstrate UOD (per USP<905>), then this is not congruent with testing a composite. If, through PV, it has been demonstrated that the attribute is homogeneously distributed (and this is also scientifically justified), then the sampling plan will focus on obtaining a representative sample and testing of a composite sample maybe justified. 

Conclusion

Generating quantitative sample test data with an acceptable level of U is a fundamental requirement of the ATP (for a quantitative method) and should be the goal of the method’s ACS where there is an understanding of all the sources of random and non-randomized error (and those controls to mitigate their effect). When defining the acceptability of the method’s U it is recommended that there is consideration for the risk of generating false positive / negative data (whereby there is an awareness of the capability of the associated manufacturing process).

References

1. NIST; Barry N. Taylor and Chris E. Kuyatt; “NIST Technical Note 1297, 1994 Edition, Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results”; September 1994;  https://emtoolbox.nist.gov/Publications/NISTTechnicalNote1297s.pdf

2.   FDA; “Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production – Guidance for Industry”; May 2022 (Revision 1); https://www.fda.gov/media/158416/download

3. Eurachem/CITAC Guide; “Use of Uncertainty Information in Compliance Assessment”; Second Edition; 2021; https://www.eurachem.org/images/stories/Guides/pdf/MUC2021_P1_EN.pdf

4. Burgess, Christopher et al; Pharmacopeial Forum; “Fitness for use: Decision rules and target measurement uncertainty”; 42(2); January 2016; https://www.researchgate.net/publication/298822306_Fitness_for_use_Decision_rules_and_target_measurement_uncertainty

Paul Mason, Ph.D., is an Executive Director at Lachman Consultant Services, Inc. with more than 20 years of pharmaceutical industry experience. His background spans Quality Control, Analytical Development, CMC submissions, and scientific support for complex FDA review issues across sterile parenteral, API, and oral solid dosage forms.