| Author | Message | ||
     Richard Kramer (kramer) Advanced Member Username: kramer Post Number: 24 Registered: 1-2001 |
I agree with the concept of a "validation space" or "validated area of measurement" but would you not consider that such a term or phrase would be defined by the linearity, range, specificity, accuracy, precision, LOQ, LOD, etc? ---- Mark, A good question to raise. I think the direct answer to your question is "no." Validation space may be simply defined as the multivariate region populated by the samples in the data set(s) which are used to validate the calibration. (Note: I use the term "populated by" rather than a term such as "spanned by" because a validation data set need not populate a validation space thoroughly-- there can be sub regions of the validation space which are devoid of validation samples. The calibration cannot be considered validated within any such unpopulated sub regions.) The performance specifications to which the calibration is validated are specified by the user and are based on the requirements which the calibration must satisfy in order to be suitable for the intended purpose (ideally defined within a risk-based context). As such, characteristics such as LOD, LOQ, precision, etc. are completely decoupled from the concept of validation space. Using FDA/PAT terminology, a validation space can be considered as a particular kind of design space. The significance of validation space is centrally important. I would suggest (and ASTM E2617 essentially states) that it is not possible to responsibly deploy a multivariate calibration without checking each and every candidate unknown sample, prior to applying the calibration to the sample, in order to confirm that it was adequately represented by the samples in the data set(s) used to validate the calibration. In other words, it is essential to confirm that an unknown sample resides in the calibration's validation space, before using the calibration on that sample. If this step is not performed, then there is no basis to claim that the calibration is valid for that sample!!! It would be very difficult to confirm that a candidate unknown sample lies within a calibration's validation space if validation space is not part of the routine lexicon, yet this seems to be the case within the ICH universe. It was also the case in the PAT universe until the promulgation of ASTM E2617, which, as you can guess by now, I regard as a critically essential standard where deployment of multivariate calibrations is concerned. | ||
| sunchunxiao (sun_chunxiao) New member Username: sun_chunxiao Post Number: 3 Registered: 11-2008 |
Everyone, is specificity carried out only for qualitation, not for quantification�� Are LOD and LOQ not discussed in NIRS method in common case? What is robustness carried out for? NIRS method? models? or equipment? And, Gabi, I have sent emails to you as follow adress: [email protected], however,I never recieve from you. My email: [email protected]. Waiting for your emails! | ||
| Mark Murphy (qamarkm1) Junior Member Username: qamarkm1 Post Number: 7 Registered: 6-2008 |
Richard, I agree with the concept of a "validation space" or "validated area of measurement" but would you not consider that such a term or phrase would be defined by the linearity, range, specificity, accuracy, precision, LOQ, LOD, etc? Mark | ||
| Charles E. Miller (millerce) Member Username: millerce Post Number: 15 Registered: 10-2006 |
While at a project meeting many years ago, I recall idly using the term �robustness� to convey the concept of the ability of a predictive model to operate effectively when interrogated with extreme observations. I was promptly chastised by a senior statistician, who mentioned that this term is �reserved� for a whole class of statistics (Robust Statistics) that deals with methodologies that are resistant to extreme observations. In fact, Robust Statistics is not confined to calculating the median (vs. mean) of a set of observations - it has long included regression: least median squares vs. least (mean) squares. Furthermore, Robust PCA, Robust PCR, and even a robust version of PLS (IRPLS) have been developed more recently, and have been found to be quite useful for �cleaning� calibration datasets that have large numbers of extreme observations. So, I kind of doubt anyone will be able to expunge the terms �robust� or even �robustness� from the multivariate analysis lexicon anytime soon. Nonetheless, this usage of the term appears to be restricted to statistical/modeling methodologies, and not to the models themselves. I�ve thought about (and heard of) other terms that attempt to describe the model property in question- including �relevance�, �range of operation�, and �brittleness�. Cheers- Chuck | ||
| Richard Kramer (kramer) Advanced Member Username: kramer Post Number: 23 Registered: 1-2001 |
but your rant is misplaced because the current specifications were developed, and intended to apply, to what were essentially univariate methods of pharmaceutical analysis: wet chemistry, chromatography, electrochmistry, etc. --- Howard, I'm not campaigning against the application of "robust" in a univariate context (although you rightly point out that they are also somewhat deficient in that context) but against the DANGEROUS MISAPPLICATION of the term in a multivariate context. This misuse is completely independent of the history of the term, and my criticism of such misuse makes no statement, positive nor negative, about any definition authored in a univariate context. ---- Since you feel strongly about "robustness", you should pick up the cudgel for that specification, to make it more quantititative. But simply ranting about how terrible the situation is, is not productive, and does nothing to improve it. ---- Howard, I think I've done the opposite of "simply ranting." I've quite specifically suggested that "robust" should be banished from use in a multivariate context and should, instead, be replaced by a proper definition and explanation of the concept of "validation space." | ||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 273 Registered: 9-2001 |
Richard - I think you say these things because you're not familiar with the history of the topic. I'm not saying you're wrong; I think you're right, but your rant is misplaced because the current specifications were developed, and intended to apply, to what were essentially univariate methods of pharmaceutical analysis: wet chemistry, chromatography, electrochmistry, etc. Granted, terms such as "robustness", like "linearity", were specified, but even in a univariate context were as vague then as they are now. This is largely because the people who formulated them knew of no way to properly specify them, or even define them, even though they had a (rather vague, to be sure) idea of what they wanted. Proper definitions, in both cases, may require multivariate definitions, especially when intended to be applied to multivariate analytical methods. But you can't fault the people who developed the current specifications; they did the best they could under the limitations they were faced with, including their limited knowledge. They had no multivariate analytical methods, and had no way to know how to apply mutivariate math. So the alternative was to have nothing. Better that some of the specifications were imperfect, and even vague, than not to have any specifications at all. "Don't let the perfect become the enemy of the good" applies here, I think. How many people would have been killed if there were no specifications at all, do you think? Now we know more, so we should be able to improve on the specifications. As you know, I've been trying to do that for "linearity", despite heavy resistance to my attempts. Since you feel strongly about "robustness", you should pick up the cudgel for that specification, to make it more quantititative. But simply ranting about how terrible the situation is, is not productive, and does nothing to improve it. \o/ /_\ | ||
| Richard Kramer (kramer) Advanced Member Username: kramer Post Number: 22 Registered: 1-2001 |
Coming directly from the ICH document Q2(R1) - Validation of Analytical Procedures: Text and Methodology, robustness is defined as follows: "The robustness of an analytical procedure is a measure of its capacity to remain unaffected by small, but deliberate variations in method parameters and provides an indication of its reliability during normal usage." --- Mark, Thank you for citing this document. It directly illustrates the point I'm trying to make about how meaningless this term is, to the point of dangerousness. There is nothing in this definition of robustness which is of any practical significance. What does "unaffected" mean? What does "small" mean? How is one to express the measure of the "reliability"? Most serious of all, the issue of "robustness" with respect to an unknown candidate for analysis is completely ignored. This definition seems to be hinting at recommendations on how a method should be validated, but it provides no meaningful guidance, neither within the definition, nor in the elaboration which occurs later in the document (section 8). In this document, robustness seems to be substituting for the role which should, instead, be filled by the concept of validation space. Robustness is a very poor, completely inadequate, substitute for the concept of validation space. Without the concept of validation space, there is no way to ensure that a calibration can be validly applied to a given candidate unknown sample. Robustness, as defined in the document, doesn't address that important topic. I find is truly shocking that a purportedly authoritative document, titled "Validation of Analytical Procedures: Methodology," completely ignores the fact that it is meaningless to validate an analytical procedure absent the concept of qualifying each unknown on which the procedure is to be applied to ensure that the unknown is qualified for analysis by the procedure as validated. On this topic, I would recommend reading ASTM E2617 "Standard Practice for Validating Empirically Derived Mulitvariate Calibrations." Unfortunately, since it is a copyrighted document, I cannot attach it here. I understand that this ICH document doesn't purport to suggest practices but merely claims to discuss "the characteristics for consideration during the validation of the analytical procedures." However, validation space is certainly one of the most crucial characteristics in this context. Rather than cover validation space in a meaningful way, the document, instead, substitutes the virtually useless but warm and fuzzy, feel good term, robustness. It is not hard to imagine that, relying on this ICH document, a company might deploy a calibration which is believed to be "robust" and inadvertently use it to produce substandard product by relying on incorrect analytical results provided by this "robust" method upon analysis of an unqualified unknown which, undetected, was outside of the validation space of the calibration. I think that the community is ill served by this dangerous, misplaced focus on "robustness," which is virtually meaningless as defined and described, while the essential characteristic of an analytical method, its validation space, is completely ignored. | ||
| Ravi Kalyanaraman (kalyanar) New member Username: kalyanar Post Number: 4 Registered: 8-2009 |
You can delibrately change the resolution of the instrument, maybe change the source intensity etc if possible and demonstrate that the method still works. If the method does not work with these changes, you need to specify the experimental boundaries within which the method is applicable. The purpose of performing robustness for your method is to determine this experimental boudary. Just a suggestion. | ||
| Mark Murphy (qamarkm1) Junior Member Username: qamarkm1 Post Number: 6 Registered: 6-2008 |
Richard, Coming directly from the ICH document Q2(R1) - Validation of Analytical Procedures: Text and Methodology, robustness is defined as follows: "The robustness of an analytical procedure is a measure of its capacity to remain unaffected by small, but deliberate variations in method parameters and provides an indication of its reliability during normal usage." I would consider robustness to be only a small part of the validation of a method, and in considering unknown samples being tested using a validated method, the linearity, precision and range of the method will determine the method's suitability for the analysis of the samples at hand and therefore restrict it's use accordingly. In knowing that temperature and humidity can have measurable effects on NIR analysis, I would suggest that the initial reviewer is correct in suggesting that robustness be evaluated as the determined results could be unnaturally biased, and as you have said... lives are at stake! The full ICH document can be found at the following link and provides a full glossary of the definitions used as well as good details on each of the parameters involved in validating a method in the main body of the document. http://www.ich.org/LOB/media/MEDIA417.pdf Regards, Mark | ||
| Richard Kramer (kramer) Advanced Member Username: kramer Post Number: 21 Registered: 1-2001 |
Posted by sunchunxiao (sun_chunxiao) on Sunday, September 27, 2009 - 4:03 am: I have submitted an artical ��Determination of potency of heparin API by NIRS), and one reviewer gave the followed advises: In order for this method to be considered as a compendial method, the authors would need to demonstrate the following: . . . c)robustness ---- Is anyone else as bothered as I am that a reviewer would use the term "robustness" in this way? What is the meaning of robustness supposed to be? Is it supposed to mean that the calibration can be expected (within specified confidence limits) to deliver a specific level of analytical performance when applied to unknown samples which were adequately represented in the data set used to validate the calibration (i. e. within the validated space for that calibration)? In that case, this should be called "validation," which has an exact meaning, rather than "robustness," which does not. Is it supposed to mean that the calibration can be expected to work 'properly' on unknown samples which lie outside the validated space for that calibration? In that case, this is a totally improper, and potentially highly dangerous, concept since the calibration is invalid, in every sense of the word, for application to samples which lie outside of the validated space for the calibration (which is why I bracketed 'properly' in single quotation marks since there is no acceptable meaning to the word in this context). Either of these two possible meanings for "robustness" has no legitimacy in the context of empirical calibrations, especially so when lives are at stake! Is there some other, appropriate, meaning of "robustness" which escapes me? | ||
| Gabi Levin (gabiruth) Advanced Member Username: gabiruth Post Number: 21 Registered: 5-2009 |
Dear Sun, Can we communicate directly? Since you are using our AOTF - I am afraid that public discussion will be considered by the community as propaganda for Brimrose and this is not the purpose of this forum. If you write directly to my mail as shown below I will continue into more details. Many thanks, Gabi Levin [email protected] | ||
| sunchunxiao (sun_chunxiao) New member Username: sun_chunxiao Post Number: 2 Registered: 11-2008 |
Thanks to eveyone! Gabi, now I give you responses to your questions: 1. Do you wish to quantify the API in the product or contamination levels by some contaminants? No, I don't. I just want to determine potency of heparin API by AOTF-NIRS, and potency is a parameter indicating anticoagulant activities of heparin. 2. If it is the API itself, what is the permissible range of % that is acceptable for this API? According to USP, potencies of heparin API should not less than 150 IU/mg, its range is usually from 150 to 180 IU/mg. 3. If it is contaminants - how many, and what is the permissible range for each. DS is a common contaminants in Heparin API, and it is less than 1% in common cases, and it don't exert effect on potency of heparin. However, as an adulteration, OSCS can affact potency of heparin, but there are no OSCS in our heparin API. The reviewer want me to "demonstrate specificity for heparin in the presence of process impurities (e.g. DS) and potential and known adulterants (e.g. OSCS)" Should I need add OSCS to heparin API samples? 4. What is the form of the product? clear liquid? suspension? crystals? amorphous powder? lyophilized? They are powders and were comminuted by a factory bucker in about 1 minute so the grain size of the powders was about 180 ��m. And this experiment was carried out with Brimrose Luminar 5030 AOTF-NIR spectrometer. Best Wishes! | ||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 272 Registered: 9-2001 |
Sun - most regulatory authorities require that analytical methods be validated, that is, to demonstrate that the methods are accurate, consistent and reliable. While the advent of new paradigms such as the FDA's PAT initiative may change this, the current paradigm requires that the method have the properties that you listed, and more. There is a two-part article demonstrating and thoroughly discussing how that was done for one analytical method: "Validation of a near-infrared transmission spectroscopic procedure, Part A: validation protocols"; J. Pharm. Biomed. Anal.; 28; p.251-260 (2002) "Validation of a near-infrared transmission spectroscopic procedure, Part B: Application to alternate content uniformity and release assay methods for pharmaceutical solid dosage forms"; J. Pharm. Biomed. Anal.; 29; p.159-171 (2002) You may also want to examine a validation protocol standard for mutivariate modelling recently approved by ASTM, but I don't recall the exact designation. You should be able to find it on the ASTM web site, www.astm.org \o/ /_\ | ||
| Gabi Levin (gabiruth) Intermediate Member Username: gabiruth Post Number: 20 Registered: 5-2009 |
Dear Sun, I can start by asking some questions - which will help me to be more specific in my advice and hopefully more useful. The guidalce is very good, but it includes so many aspects that it is sometimes more difficult to find what is suitable for your case. 1. Do you wish to quantify the API in the product or contamination levels by some contaminants? 2. If it is the API itself, what is the permissible range of % that is acceptable for this API? 3. If it is contaminants - how many, and what is the permissible range for each. 4. What is the form of the product? clear liquid? suspension? crystals? amorphous powder? lyophilized? There is a reason for my questions - and they all stem from experience with several forms of API from real production processes. The actual requirements for each case maybe somewhat different, depending on the case. For example - Limit of Detection is completley irrelevant for a case where a contaminant is allowed from 0 to 1% (and there are such cases). I could help you by posting a message here which could benefit everyone, and I could also respond directly, whcihever you wish. Gabi Levin [email protected] | ||
| Sheelagh Halsey (shalsey) Advanced Member Username: shalsey Post Number: 21 Registered: 12-2006 |
There are plenty of literature references on validating NIR methods for pharmaceutical use. Probably the most referred to is the EMEA guidance which can be found at: http://www.emea.europa.eu/pdfs/human/qwp/330901en.pdf Regards Sheelagh | ||
| sunchunxiao (sun_chunxiao) New member Username: sun_chunxiao Post Number: 1 Registered: 11-2008 |
I have submitted an artical ��Determination of potency of heparin API by NIRS), and one reviewer gave the followed advises: In order for this method to be considered as a compendial method, the authors would need to demonstrate the following: a)sensitivity-limit of detection and limit of quantitation b)specificity-demonstrate specificity for heparin in the presence of process impurities potential and known adulterants c)robustness d)accuracy-authors attempted this with 3 batches. However, they will need to demonstrate spike recovery. However, little literatures told how to demonstrate those parameters, can everyone give me some useful advises?Thank you! |