| Author | Message | ||
     Nuno Matos (Nmatos) |
Dear All, PASG introduces repeatibility to validate qualification models in the next terms: "This is not normally required for identification methods. For qualification methods, repeatability is addressed in order to demonstrate that the acceptance thresholds established provide reliable discrimination between acceptable and unacceptable materials; the approach is therefore conceptually different for NIR methods compared with conventional methods." What should be the translation? | ||
| hlmark |
Nuno - the difference is between quantitative methods and qualitative methods. "Repeatability" for a quantitative method is straightforward: the numerical measure of the agreement of the final answers, among multiple analyses. For a qualitative method, the problem you run into with that concept is that even though calculations are used to ascertain the identity of a material, the final answer is not a numerical quantity, it is a statement: "The material is X". In this situation, the only thing you can say about the statement is whether it is right or wrong. If it's wrong, it is difficult to assign a number to "how much" it is wrong. If it's right, "repeatability" is almost meaningless, because all statements that "The material is A" are essentially identical. What can be done, post-facto, is to calculate the number (or the fraction of times) the material is correctly identified as A when it is A, and the fraction of times it is correctly identified as something else when it is not A. Ideally, both those numbers should be 100%. But if the material is not A, it can be "not A" in many different ways. It is possible to step back from the final identification and do some sort of "repeatability" calculations on the numerical values leading to the identification, but I'm somewhat uncomfortable with that. For example, calculating something like a standard deviation of Mahlanobis distances may provide a small number for the S.D., but that could be misleading since different readings could be the same "distance" from the centroid of the A data, but be in different directions. That would make the data points really be much further apart than the calculation indicated. Howard \o/ /_\ | ||
| Bruce H. Campbell (Campclan) |
Nuno, Another way to look at qualitative results is in parallel with statistics, which indeed much of qualitative analysis results are. That is, statistics never tries to give an absolute answer but instead what the probability of the answer is of being "correct." This is reflected in the common use of three Mahalanobis distances used to verify or deny identity. These three distances have a certain probability someplace in the high ninety percent range (but I don't know what that probability is). Classically, positive identification is given by agreement between two different identity tests on the same material. This also has a probability of being correct, but I don't know this probability either. Bruce |