| Author | Message | ||
     Donald J Dahm (djdahm) Junior Member Username: djdahm Post Number: 6 Registered: 2-2007 |
Howard, I agree with what you say, but I can't help notice how much of our discussion deals with "how to twiddle the knobs" of the Chemometrics rather than trying to understand what's going on. I've started a new thread that I hope will allow some further discussion on this issue. Don | ||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 77 Registered: 9-2001 |
Don - in principle I agree with you. The problem you run into, though, is that in the majority of cases (or at least a majority of cases of interest) the spectrum of the analyte, and the changes in the spectrum of the analyte, are smaller than either or both of the following: 1) the variations due to the changes in the scatter of the samples 2) the variations due to changes in the other components of the sample This is very often the case, and is essentially always the case when the analyte of interest is one of the minor components. That does not mean trace analysis necessarily; this has been true since the beginning of NIR as a practical analytical technique, in the measurement of protein in wheat, the first "killer app" for NIR (bad term, actually, since that was what gave NIR life). Protein is generally present in wheat at levels from about 8-18%, a respectable amount but still a minor component. Moisture is present at about 10-14%, but due to its higher intrinsic absorbance has a much larger effect on the spectrum than the protein content does, at the same level. Starch is present as about 65-80% of the composition; that, by itself, makes it the major component. There's about 5-10% combined fiber, vitamins, minerals, and other minor ingredients. Despite all this, the measurement of protein in wheat is, to this day, probably the most successful application of NIR, both scientifically and economically, and that's due solely to the use of chemometric techniques to "pull" the protein signal out from all those other interferences, that domoinate the spectrum. What you propose is great when you're dealing with samples that are clear liquids, or other simple cases. Then you can actually relate NIR measurements to the underlying absorption coefficient and other physical phenomena the same way you can in mid-IR, UV, visible, etc. But the limiting fact in NIR measurements has always been the inability to reliably and properly correct for the effect of optical scatter, due to the lack of proper physical theory to explain it. If we could do that, then we could reduce the problem to an equivalent case analogous to clear liquids and apply all those other methods with the same reliability as we can there. \o/ /_\ | ||
| Donald J Dahm (djdahm) New member Username: djdahm Post Number: 4 Registered: 2-2007 |
Sorry, this is a very late posting. I don't know if this will contribute anything or not, but since I quit writing the "Theoretically Speaking�" column, I need an outlet for my hostility. Way down in this thread Eric said: "The sense of my request was actually to compare two measurement techniques (diffuse reflectance and transmission)...." We all have our prejudices, and mine is that we go back to basics to answer questions like this instead of screwing around with the Chemometrics. (As you can tell, it's not hard for me to get grumpy about the subject.) My approach is to ask (and perhaps explore experimentally before the selection): 1) Which of the two gives you a light intensity that can be reliably measured, and which changes significantly as a function of the concentration we were measuring? The desire to have a convenient sample presentation sometimes makes our decision in the specific for us, but the above requirement is always with us. 2) In the absence of any difference in the two in the above: Which of the two will yield a change as a function of concentration that is most nearly linear? [Or can be made to be linear by a pre-treatment that is theoretically sound. (Remember log (1/T) [or log (1/R)] is a pretreatment.) And consistent with the "no screwing around with Chemometrics" these pretreatments should not depend on the sample set. Each sample is treated separately.] Here transmission will probably win out. Once you have the Spectroscopy right, the Chemometrics will work better, because they won't have to compensate for second rate data. I think we can say the following. 1) There are always experimental arrangements restrictions that must be dealt with. Remember back in your first Spectroscopy course when we were introduced to the transmission spectroscopy of liquids. We had to pick an arrangement (select the wavelength, and path length) that gave us a light intensity that could be reliably measured, and which changed significantly as a function of the concentration we were measuring. Balancing this criterion against the desire to have a convenient sample presentation sometimes makes our decision in the specific for us, but the question is always with us. 2) As a secondary criterion, we hope that the above change as a function of concentration is relatively linear. | ||
| venkatarman (venkynir) Senior Member Username: venkynir Post Number: 35 Registered: 3-2004 |
Pierre Thank you, Kindly give the web address of the article or Please post Pdf of the article I am reading N.M Faber paper and other IUPAC technical Report . | ||
| Pierre Dardenne (dardenne) Intermediate Member Username: dardenne Post Number: 19 Registered: 3-2002 |
Hi, Do try to find the article of Tom Fearn again : NIR News Vol 15 N�14 (2004) Pierre | ||
| venkatarman (venkynir) Senior Member Username: venkynir Post Number: 34 Registered: 3-2004 |
Hello NIR friends , please brief about RMSEC,RMSECV & RMSEP. All the publication speaks about it , but it is confusing. When to use RMSEC and How to relate RMSECV and RMSEP. | ||
| Eric LALOUM |
Dear colleagues, What test would you recommend to compare the performance of two NIR calibration ? For instance (all other things being comparable, like number of sample, factors, dof...) what ratio between two SEC (or SECV, SEP) would you consider significantly different from 1 ? Related theoretical question : what is the distribution of error estimates in NIR calibration ? | ||
| Tony Davies (Td) |
Hello Eric, Tom Fearn discussed "Comparing standard deviations" as one solution to this problem in the Chemometric Space column in NIR news, 7(5), 5 (1996). Hope this helps. Best wishes, Tony | ||
| hlmark |
Eric - the problem with comparing calibrations is that one of the components of the error is the error contribution of the reference laboratory, which is the same for all models. Therefore in all cases, and especially in those cases where the reference laboratory error is the dominant error source, it masks whatever differences might be present in the other error terms, and makes the standard statistical tests not apply. One way to approach this difficulty is to collect data in a way such that each sample's data gives a measure of the other error terms individually, then pool the results from all the samples. An example of how this can be done is given on pages 43-45 of "Principles and Practice of Spectroscopic Calibration", 2nd ed., Wiley (1991). Howard \o/ /_\ | ||
| Richard Kramer |
What test would you recommend to compare the performance of two NIR calibration ? Eric, I would not recommend any test for this purpose. There is no test which is able to determine which of a number of alternative calibrations is likely to work best over time. Fortunately, picking calibrations isn't the same as picking horses at the race track. We aren't forced to bet on a winner before the race starts. We can follow all of the horses, or all of our favorite horses, to see which one eventually comes out ahead. The best test is to track the performance of all of them over time. Inevitably, some candidates will deteriote more rapidly than others at which point the better choices start to become clear. At any given point in time, we would be required to designate a calibration as the authorative calibration. The outputs from all other calibrations can be treated as "supplementary diagnostic values." If ongoing validation shows that the performance of the designated authoritative calibration should be deteriorating, we would have the opportunity to "switch horses" in the middle of the race. Richard | ||
| David Russell (Russell) |
I agree with both Howard and Richard's comments. In the process of doing a calibration you generate several candidate models for evaluation. What I like to do is start with the simplest model that is likely to work and hold the others in reserve. I also like to have some way of looking into the terms of the equation to assure my self that each term being used is significant. In the case of MLR I use the P value of the regression term as a metric. For PLS I like to look at the loadings vector for each factor or the explained variance plot both of which would be plotted vs wavelength. If the plot doesn't appear to have any meaning chemically or just looks like noise, then it's probably best to exclude that factor. Bottom line is that there is no substitute for judgement, experience, and on-line validation. | ||
| Eric LALOUM |
Thank you for your help, The sense of my request was actually to compare two measurement techniques (diffuse reflectance and transmission) and in that respect, I built two calibrations from two data sets corresponding to the same samples. I have then optimized individually the PLS model for each technique and at the end I have two models with associated "best" SECV. One SECV is (of course) smaller than the other one but it seems a little quick to deduce that one technique is better than the other... Maybe one trick would be to keep the statistics of several calibrations made on each data set (for instance with different pretreatments or different selection of samples), and then to compare not only two individual SECV but two series of SECV ? | ||
| hlmark |
Eric - I can see where you have an incentive to come to a decision in advance, one way or the other. Doing a series of comparisons of SECV is at best, marginally better than a single one, because there is no way to really know whether the quantities you are comparing are actually comparable. Unfortunately, having more incentive doesn't make information magically appear out of limited data. Ordinarily, I'd advise you to try Richard's suggestion since that can create more information, but that doesn't really seem practical in this case. You may have to just "bite the bullet" and make a decision without absolute knowledge of whether it's the "right" one. On the other hand, if the performance of the reflection and transmission methods are so close that the decision really is that difficult, then it may not matter a whole lot which one you choose, simply because the accuracies are so nearly the same. Howard \o/ /_\ |