Gary E. Ritchie, Contributing Writer04.01.21
Near Infrared (NIR) Spectroscopy has become a mainstay in the pharmaceutical as well as most other industries as an analytical technique because of its rapid, non-destructive, and non-invasive sampling capabilities. With sufficient training in spectroscopy and Chemometrics, it is relatively easy to deploy at various points throughout the sample lifecycle for both qualitative and quantitative measurements.
It was not always this way. Stories abound at the dawn of “modern” NIR in the 1970s of instruments purchased, people trained, samples obtained, and money allocated to fund ambitious projects using NIRS to solve problems quickly and cheaply—relative to a reference laboratory approach.
Years later, these same instruments were mothballed, people lost their positions, and executives turned their nose up towards the technology because of these initial failed attempts at the outset. No doubt these initial failures were the result of educated scientists failing to do the first step of any new research: the literature search. If they had, they would have learned to appreciate and really understand the complexity and nuances of the NIR technique as compared to other spectroscopic modes.
So that future analysts take heed of the philosopher George Santayana, who said, “Those who cannot remember the past are condemned to repeat it,” this article outlines and provides the critical elements gleaned from the history of NIR that should be paid attention to in order to be successful at applying the lessons learned from the past.
Because the pharmaceutical industry is regulated, all analytical techniques used for pharmaceutical analysis are required to comply with CGMPs (Current Good Manufacturing Practices) in the U.S. and with other regulations in other countries. “cGMP regulations for drugs contain minimum requirements for the methods, facilities, and controls used in manufacturing, processing, and packing of a drug product. These regulations assure that a product is safe for use and that it has the ingredients and strength it claims to have.”1 The cGMP and similar requirements for analytical methods developed from NIR are found in many guidances across the globe.2,3,4,5,6,7 Their differences vary only slightly and for the most part follow the basic validation approaches for analytical methods used for pharmaceutical drug substance and product development because they have been harmonized through the International Council for Harmonization (ICH).8
NIR characteristics can be viewed as three legs of a triangle (Figure 1):
Figure 1. Analytical Method Triangle
A fourth component recognized is analyst knowledge—education qualification and training. Meeting the regulatory requirements for an analytical method requires that critical parameters for instrument and method performance be evaluated. Similarly, samples must be evaluated for their appropriate properties and response for NIRS measurements.
But, what was done prior to these ubiquitous standards to show that NIR measurements were valid? The historical record documenting the discovery, investigation, and use of infrared and later-on near-infrared phenomena provides the fundamental proofs on which NIR measurements are based today.
It is important to realize that the logic, hence the science used during the historical phase, became the foundation upon which the principles of the modern NIR experiment rests. The following discussion cites the characteristics of NIR explored by the early investigators that verify and validate the use of NIR for measuring the physical and chemical properties of matter.
Error Analysis
Rocco Difoggio stated that error is associated with all analytical measurements. Error from NIR measurements arise primarily from the calibration results determined from the NIR measurements combined with the laboratory values obtained from the reference method. This combined error constitutes the bias (difference) observed between the calibrated NIRS and compendial reference method.
Secondary contributions arise from user requirements (i.e., constraints from environmental conditions, sample presentation, etc.), spectral noise, and artifacts from the reference values propagate through calibration model, each contributing to the experimental bias. It is incumbent on the analyst to assess, understand, validate, and control all sources of error.9,10
NIRS errors are manifested as authentic (a value or a spectrum associated with an acceptable bias), an outlier (a value or spectrum associated with a bias greater or less than the acceptable bias), or a residual (the difference of an unknowns predicted value and spectrum from the model). Historically, error was attributed to instrument (mechanical and electronic), computation, sample positioning, and measurement, measurement from a single determination from one “representative” sample measurement, and/or the presence of moisture.11
Accuracy
Norris and Williams demonstrated that effects from sample mean particle size, particle size distribution, temperature, and moisture from the NIR results vary greatly.12 Determining the sample properties of not only those being determined, but also of those that directly or indirectly effect the sample placement and measurement, must be shown and accounted for if the results are to be accurate and the bias acceptable from the reference value. Norris later improved the instrument for measuring diffuse reflectance NIR spectra by devising a single beam instrument leading to increased signal-to-noise and significant improvement of the accuracy of the NIR measurement.13
Precision
The reproducibility of NIR measurements was demonstrated for the first time by Abney and Festing.14,15 Not only were accurate thermographs of line spectra obtained from ethyl iodide and chloroform, but they were reproducible anytime and anyplace. Most convincing however was the demonstration proving the existence of heat rays. By fixing a (0, 0) reference from the center of the yellow wavelength, and by making many thermographs, they were able to establish the mean of a great number of thermographs so that on average, the mean of three measurements proved useful for demonstrating the presence of heat rays due to heat only, and not those which are usually associated with chemical effects. Modern NIR spectrophotometers are capable of achieving precision in the range from 0.004 to 0.5% when measuring OH. NH, or CH groups of unknown compounds.16
Sample
It is important to note that historically, earlier measurement of samples was done entirely on pure chemical compounds, and it was only later on, well into mid 1900s that substances ranging from forage, soil, and animal physiology were being studied. Later on, its use expanded into other markets such as energy, food, textiles, beverage, pharmaceuticals, medical and more. Certain caveats have been developed when measuring materials for their NIR spectra. An assessment of the results from a well-planned Design of Experiment (DoE) against these critical factors is necessary if valid (known errors, accurate and precise) results are to be obtained.
Instrument(s)
Kay’s remarks at the Pittcon 1954 symposium are succinct regarding requirements for NIR instrumentation.16 NIR Instruments should be assessed for their signal-to-noise ratio and wavelength or wavenumber resolution. Low signal-to-noise contributes significant error to a calibration. The main contribution of noise comes from the light source as photon noise. Regarding wavelength or wavenumber resolution, Griffiths et al.17 present data on the importance of understanding wavelength or wavenumber resolution variance regarding measurements made in near-infrared diffuse reflection mode when using high resolution FT-NIR instruments versus low resolution scanning monochromators.
To Kay’s credit, Griffith et al. note that differences due to (a) observed wavelength shifts; (b) the difference in the number of resolved absorption bands caused by differences in the resolution between grating and Fourier–transform (FT) NIR instruments; and (c) differences in the number of absorption bands among standards, will have significant effects on the accuracy of NIR measurements. In addition, Griffiths et al. point out that NIR instruments effect on the accuracy of the wavelength scale of NIR spectrometers are determined by:
These instrument characteristics should be assessed and defined in any user requirement before making an instrument selection.
Analyst
Analytical chemistry education and laboratory training alone is not enough to be successful at using NIR as a solution for solving analytical problems. Besides the fact that NIR spectra are not suitable for analysis directly, since they are mainly comprised of hydrogenic bonds (OH, CH, NH) from nonspecific sources, another requirement must be met for this highly specialized branch of spectroscopy to be useful: that being knowledge of chemometrics.
Chemometrics is “…concerned with the application of mathematical and statistical techniques to extract chemical and physical information from complex data.”18 Meeting all of the requirements would allow an analyst to pull together knowledge from chemistry, spectroscopy and chemometrics to perform a sequence of steps for obtaining both qualitative and quantitative information about materials. Because the use of NIR and chemometrics consumes a lot of data in order to arrive at a solution, another requirement is that the analyst must be able to keep track of and organize a set of samples, taking the measurements without making errors, and the sampling of spectra should be performed by the same person entering the chemical data and operating the computer programs to do the analyses in order to minimize or eliminate human error.19
Conclusion
They say that “Twenty-twenty” is hindsight. What the future has wrought with microprocessors, miniature, handheld spectrophotometers, free, open-source software and Google is the expectation that information can now be had instantaneously and at no cost to the user. This may be true for some applications, but, it is not true for NIR technology for several reasons:
Sufficient training must not only include that the characteristics of NIR are thoroughly reviewed and understood as outlined in this paper, but also chemometrics and principles arising from the Theory Of Sampling (TOS). Doing so will ensure that instruments, personnel and NIR programs flourish and remain funded.
References
Gary E. Ritchie, MS, is an internationally recognized expert in pharmaceutical analysis with a focus on vibrational spectroscopy and multivariate analysis, process analytics and quality assurance. Gary’s experience includes increasing responsibilities in quality control, technical services, research and development and new technologies with Schein Pharmaceuticals and Purdue Pharma. Gary was appointed Scientific Fellow for Process Analytical Technology and Liaison to the General Chapters, Pharmaceutical Waters and Statistics Expert Committee’s from 2003 through 2008 for the United States Pharmacopeia (USP). Gary currently consults, providing analytical and quality solutions for the pharmaceutical industry.
It was not always this way. Stories abound at the dawn of “modern” NIR in the 1970s of instruments purchased, people trained, samples obtained, and money allocated to fund ambitious projects using NIRS to solve problems quickly and cheaply—relative to a reference laboratory approach.
Years later, these same instruments were mothballed, people lost their positions, and executives turned their nose up towards the technology because of these initial failed attempts at the outset. No doubt these initial failures were the result of educated scientists failing to do the first step of any new research: the literature search. If they had, they would have learned to appreciate and really understand the complexity and nuances of the NIR technique as compared to other spectroscopic modes.
So that future analysts take heed of the philosopher George Santayana, who said, “Those who cannot remember the past are condemned to repeat it,” this article outlines and provides the critical elements gleaned from the history of NIR that should be paid attention to in order to be successful at applying the lessons learned from the past.
Because the pharmaceutical industry is regulated, all analytical techniques used for pharmaceutical analysis are required to comply with CGMPs (Current Good Manufacturing Practices) in the U.S. and with other regulations in other countries. “cGMP regulations for drugs contain minimum requirements for the methods, facilities, and controls used in manufacturing, processing, and packing of a drug product. These regulations assure that a product is safe for use and that it has the ingredients and strength it claims to have.”1 The cGMP and similar requirements for analytical methods developed from NIR are found in many guidances across the globe.2,3,4,5,6,7 Their differences vary only slightly and for the most part follow the basic validation approaches for analytical methods used for pharmaceutical drug substance and product development because they have been harmonized through the International Council for Harmonization (ICH).8
NIR characteristics can be viewed as three legs of a triangle (Figure 1):
- Instrument Qualification
- Method Validation
- Sample Design of Experiment (DoE)
Figure 1. Analytical Method Triangle
A fourth component recognized is analyst knowledge—education qualification and training. Meeting the regulatory requirements for an analytical method requires that critical parameters for instrument and method performance be evaluated. Similarly, samples must be evaluated for their appropriate properties and response for NIRS measurements.
But, what was done prior to these ubiquitous standards to show that NIR measurements were valid? The historical record documenting the discovery, investigation, and use of infrared and later-on near-infrared phenomena provides the fundamental proofs on which NIR measurements are based today.
It is important to realize that the logic, hence the science used during the historical phase, became the foundation upon which the principles of the modern NIR experiment rests. The following discussion cites the characteristics of NIR explored by the early investigators that verify and validate the use of NIR for measuring the physical and chemical properties of matter.
Error Analysis
Rocco Difoggio stated that error is associated with all analytical measurements. Error from NIR measurements arise primarily from the calibration results determined from the NIR measurements combined with the laboratory values obtained from the reference method. This combined error constitutes the bias (difference) observed between the calibrated NIRS and compendial reference method.
Secondary contributions arise from user requirements (i.e., constraints from environmental conditions, sample presentation, etc.), spectral noise, and artifacts from the reference values propagate through calibration model, each contributing to the experimental bias. It is incumbent on the analyst to assess, understand, validate, and control all sources of error.9,10
NIRS errors are manifested as authentic (a value or a spectrum associated with an acceptable bias), an outlier (a value or spectrum associated with a bias greater or less than the acceptable bias), or a residual (the difference of an unknowns predicted value and spectrum from the model). Historically, error was attributed to instrument (mechanical and electronic), computation, sample positioning, and measurement, measurement from a single determination from one “representative” sample measurement, and/or the presence of moisture.11
Accuracy
Norris and Williams demonstrated that effects from sample mean particle size, particle size distribution, temperature, and moisture from the NIR results vary greatly.12 Determining the sample properties of not only those being determined, but also of those that directly or indirectly effect the sample placement and measurement, must be shown and accounted for if the results are to be accurate and the bias acceptable from the reference value. Norris later improved the instrument for measuring diffuse reflectance NIR spectra by devising a single beam instrument leading to increased signal-to-noise and significant improvement of the accuracy of the NIR measurement.13
Precision
The reproducibility of NIR measurements was demonstrated for the first time by Abney and Festing.14,15 Not only were accurate thermographs of line spectra obtained from ethyl iodide and chloroform, but they were reproducible anytime and anyplace. Most convincing however was the demonstration proving the existence of heat rays. By fixing a (0, 0) reference from the center of the yellow wavelength, and by making many thermographs, they were able to establish the mean of a great number of thermographs so that on average, the mean of three measurements proved useful for demonstrating the presence of heat rays due to heat only, and not those which are usually associated with chemical effects. Modern NIR spectrophotometers are capable of achieving precision in the range from 0.004 to 0.5% when measuring OH. NH, or CH groups of unknown compounds.16
Sample
It is important to note that historically, earlier measurement of samples was done entirely on pure chemical compounds, and it was only later on, well into mid 1900s that substances ranging from forage, soil, and animal physiology were being studied. Later on, its use expanded into other markets such as energy, food, textiles, beverage, pharmaceuticals, medical and more. Certain caveats have been developed when measuring materials for their NIR spectra. An assessment of the results from a well-planned Design of Experiment (DoE) against these critical factors is necessary if valid (known errors, accurate and precise) results are to be obtained.
- Selectivity: the analyte must be a known and documented near-infrared absorber or otherwise fully characterized as such.
- Specificity: the method must be shown to be free from contaminants and specific for the analyte.
- Linearity: a suitable range should be established in order to determine the linearity of response.
- Recovery: the critical experimental step (procedure) should be explored for retrieving the analyte.
- Repeatability: the measurement step (method) should be described clearly, concisely and completely so that it can be performed repeatedly with minimum variability.
- Suitability: the instrument requirements are provided (S/N, wavelength / wavenumber range and resolution, # of scans co-add, data acquisition rate), and instructions given for handling the sample during measurement [i.e. temperature effects].
- Sample Lifecycle: specific requirements as to the reference and sample storage, handling, preparation and presentation.
- Experimental: a method for obtaining the spectral result is described.
- Results: the units for the results are provided and expressed.
Instrument(s)
Kay’s remarks at the Pittcon 1954 symposium are succinct regarding requirements for NIR instrumentation.16 NIR Instruments should be assessed for their signal-to-noise ratio and wavelength or wavenumber resolution. Low signal-to-noise contributes significant error to a calibration. The main contribution of noise comes from the light source as photon noise. Regarding wavelength or wavenumber resolution, Griffiths et al.17 present data on the importance of understanding wavelength or wavenumber resolution variance regarding measurements made in near-infrared diffuse reflection mode when using high resolution FT-NIR instruments versus low resolution scanning monochromators.
To Kay’s credit, Griffith et al. note that differences due to (a) observed wavelength shifts; (b) the difference in the number of resolved absorption bands caused by differences in the resolution between grating and Fourier–transform (FT) NIR instruments; and (c) differences in the number of absorption bands among standards, will have significant effects on the accuracy of NIR measurements. In addition, Griffiths et al. point out that NIR instruments effect on the accuracy of the wavelength scale of NIR spectrometers are determined by:
- the spectrometer resolution (sometimes called the spectral bandwidth, SBW), i.e., the optical resolution of the instrument;
- the shape of the instrument line shape (ILS) function of the spectrometer (often called the spectral slit function for grating spectrometers);
- the symmetry and extent of overlap of the bands in the spectrum of the wavelength standard;
- the choice of the peak-picking algorithm used to assign peak maxima; and
- vignetting of the beam by the diffuse reflection optics.
These instrument characteristics should be assessed and defined in any user requirement before making an instrument selection.
Analyst
Analytical chemistry education and laboratory training alone is not enough to be successful at using NIR as a solution for solving analytical problems. Besides the fact that NIR spectra are not suitable for analysis directly, since they are mainly comprised of hydrogenic bonds (OH, CH, NH) from nonspecific sources, another requirement must be met for this highly specialized branch of spectroscopy to be useful: that being knowledge of chemometrics.
Chemometrics is “…concerned with the application of mathematical and statistical techniques to extract chemical and physical information from complex data.”18 Meeting all of the requirements would allow an analyst to pull together knowledge from chemistry, spectroscopy and chemometrics to perform a sequence of steps for obtaining both qualitative and quantitative information about materials. Because the use of NIR and chemometrics consumes a lot of data in order to arrive at a solution, another requirement is that the analyst must be able to keep track of and organize a set of samples, taking the measurements without making errors, and the sampling of spectra should be performed by the same person entering the chemical data and operating the computer programs to do the analyses in order to minimize or eliminate human error.19
Conclusion
They say that “Twenty-twenty” is hindsight. What the future has wrought with microprocessors, miniature, handheld spectrophotometers, free, open-source software and Google is the expectation that information can now be had instantaneously and at no cost to the user. This may be true for some applications, but, it is not true for NIR technology for several reasons:
- It is not a primary mode of analysis. The results are obtained by correlating NIR spectra with values or properties of a reference method used to make original measurements of the samples under study. Learning the nature of sampling is the first necessary step and is paramount for a successful NIR program.20
- Raw NIR spectra contain both physical and chemical properties about the sample and may require some form of spectral pretreatment, which require a basic understanding of spectroscopy and the chemometric tools used to perform the spectral transformations.
- Chemometrics involves the use of a plethora of algorithms for analyzing the spectra and reference data. The mathematical relationships required to analyze the data depend on understanding the nature of the problem (i.e., qualitative or quantitative), the scope of the problem (i.e, data reduction, clustering, regression analysis) and the instrument mode required to name some of the key factors and reasons for researching the history of NIR before plowing ahead.21
Sufficient training must not only include that the characteristics of NIR are thoroughly reviewed and understood as outlined in this paper, but also chemometrics and principles arising from the Theory Of Sampling (TOS). Doing so will ensure that instruments, personnel and NIR programs flourish and remain funded.
References
- https://www.fda.gov/drugs/pharmaceutical-quality-resources/current-good-manufacturing-practice-cgmp-regulations
- Pharmaceutical Analytical Sciences Group (PASG), NIR sub-Group Guidelines for the Development and Validation of Near-Infrared Spectroscopy Methods in the Pharmaceutical Industry, PASG, 2002
- Guideline on the use of near infrared spectroscopy by the pharmaceutical industry and the data requirements for new submissions and variations Revision 2 January 2014
- GUIDANCE DOCUMENT, Development and Submission of Near Infrared Analytical Procedures MARCH 2015 Draft Not for implementation. Contains non-binding recommendations
- ASTM E1790-04(2016)e1, Standard Practice for Near Infrared Qualitative Analysis, ASTM International, West Conshohocken, PA, 2016
- ASTM E1655-17, Standard Practices for Infrared Multivariate Quantitative Analysis, ASTM International, West Conshohocken, PA, 2017
- USP <1119> Near-Infrared Spectroscopy, USP 43-NF (38)
- ICH Harmonized Tripartite Guideline Validation Of Analytical Procedures: Text And Methodology Q2(R1) March 2005
- Difoggio, R., “Examination of Some Misconceptions About Near-Infrared Analysis”, Applied Spectroscopy, Volume 49, Number 1, pp. 67-75, 1995
- Difoggio, R., Applied Spectroscopy, “Guidelines for Applying Chemometrics to Spectra: Feasibility and Error Propagation,” Volume 54, Number 3, pp. 94A–113A, 2000
- Coblentz, W. W., “Supplementary Investigations of Infra-Red Spectra Part V Infra-Red Reflection Spectra, Part VI” “Infra-Red Transmission Spectra, Part VII Infra-Red, Emission Spectra”, The Carnegie Institution of Washington, 1908
- Norris K.H. and Williams P.C., “Optimization of Mathematical Treatments of Raw Near-Infrared Signal in the Measurement of Protein in Hard Red Spring Wheat. I. Influence of Particle Size”, Cereal Chem. 61(2), 158–165, 1984
- Norris, K., Early History of near Infrared for Agricultural Applications, NIR News 3(1), 12-13, CNIR Publications 1992
- Bakerian Lecture “On the Photographic Method of Mapping the least Refrangible End of the Spectrum,” by Captain W. de W. A b n e y, R.E., F.R.S., Phil. Trans., 1880
- Abney, R.E. and Festing, R.E., On the influence of the atomic grouping in the molecules of organic bodies on their absorption in the infra-red region of the spectrum, Phil. Trans. R. Soc. Lond. 172, p. 889, 1881
- Kaye, W., “Near-infrared spectroscopy; A review. I. Spectral identification and analytical applications”, Spectrochimica Acta, 6, 257-287, 1954
- Griffiths, P. R., Ritchie, G. E., Casay, G. A., Jones, B. J., Pharmacopeial forum 38(1) “Stimuli to the Revision Process: Wavelengths of Calibration Standards for Near-Infrared Diffuse Reflection Spectrometry”, 2012
- Wold, S., Spline functions, a new tool in data-analysis, Kemisk Tidskrift, 1972
- Norris, K., Early History of near Infrared for Agricultural Applications, NIR News 3(1), 12-13, CNIR Publications 1992
- Esbensen, K. H., et al, Ed., Introduction to the Theory and Practice of Sampling, IM Publications, 2019
- Ciurczak, E. W., Ed., 4th Edition Handbook of Near-Infrared Analysis, 2021
Gary E. Ritchie, MS, is an internationally recognized expert in pharmaceutical analysis with a focus on vibrational spectroscopy and multivariate analysis, process analytics and quality assurance. Gary’s experience includes increasing responsibilities in quality control, technical services, research and development and new technologies with Schein Pharmaceuticals and Purdue Pharma. Gary was appointed Scientific Fellow for Process Analytical Technology and Liaison to the General Chapters, Pharmaceutical Waters and Statistics Expert Committee’s from 2003 through 2008 for the United States Pharmacopeia (USP). Gary currently consults, providing analytical and quality solutions for the pharmaceutical industry.
