| Author | Message | ||
     Pierre Dardenne (dardenne) Senior Member Username: dardenne Post Number: 48 Registered: 3-2002 |
Dear all, Interesting talks. We are using NIRS predicted values to transfer models (for instance from reflexion to transmission or reverse) only when the standardization (cloning) of spectra is not possible; Always much better to use actual ref values. Be aware the SD of the predicted data are shrunk in the proportion of the coefficient of correlation. (r=SDyest/SDy for any calibration set). Thus reported shrunk ref values for a new model with lead to a new shrinkage. The final model will really underestimate high values and overestimate low values. I set the predicted data to cover the same SD and the original set before recalibrating with NIR predicted �reference values�. In any case as Peter said, the whole set of stats in calibration are not valid and only an independent test set with actual wet values will give the actual performances. The method works fine with very high original R2. Lower is the original R2, higher is the risk to fail. Regards, Pierre | ||
| Gabi Levin (gabiruth) Senior Member Username: gabiruth Post Number: 35 Registered: 5-2009 |
Howard, Excellent point - and it is only due to my assumption that no one would consider relying solely on the NIR for the validation of the second spectrometer that I missed it, it is a number one rule - final validation will require using a sufficient number of samples analyzed by the reference method, HPLC or be it whatever they use, and these should be representative of the entire range over which the calibration is being used. The advantage of using an existing lab calibration is to reduce the amount of work that would be otherwise required if only lab work is to be used. And I pointed out that it has a price, and while many times this price is tolerable, there are circumstances where this price is not acceptable. Practitioners should be practical and understand the pros and cons in doing so. Good common sense is a key asset in such cases. Thanks, Howard, gabi | ||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 326 Registered: 9-2001 |
Gabi and Peter - you're both talking about the reference values as though they are coming from the reference laboratory. In the situation Alisha is describing, however, the "reference" values are the values from the first NIR instrument, which is the only one calibrated against the actual "reference" values. That's where the risk comes. You make a good point that a process instrument may have different systematic errors than the lab instrument that was calibrated, but the risk is still there. The only way to resolve the question for sure is for someone to try it all out and do the experiment: measure process samples on the calibrated lab instrument to provide "reference" values and then on the process instrument to calibrate that. Then to validate it, it will be necesary to measure process samples by the reference lab method; that will be the required validation in any case. \o/ /_| | ||
| Gabi Levin (gabiruth) Senior Member Username: gabiruth Post Number: 34 Registered: 5-2009 |
Hi Alisha, The proof of the pudding is in the eating - this applies here as well, and, it is important that we realize that many times the process spectrometer will differ to a good extent from the bench top - having a flow cell in line is different from having a cuvette or similar arrangement. The main question is - how good are the NIR values you use for reference values in the regression you will run on your process spectrometer spectra. If the SEP is well within your allowed process variation - you stand a good chance of success. If your SEP values from your lab NIR are close to the process limits, you risk that the process readings from the process spectrometer will hit the limits quite often. The reason is that the SEP from your process spectrometer will be the minimum SEP you can ever hope for, and it will surely be larger. This is why it is so important to compare the process allowed limits and the reference data uncertainty. The general rule of thumb is that for a reasonable regression that will hold water in real life, the uncertainty in the reference data will be less than 5% of the range over which the regression is used. In your case it becomes also critical to compare the SEP and the process variations allowed. There is a partial remedy to that - if your process is between 20 to 30, just for illustration and your new SEP based on the NIR instrument as ref values is 1.5, you may try to control you process between 23 to 27, leaving enough "margin" to give you the confidence that your process is not out of line. Since the process analyzer can give you frequent readings, as in our instrument case every 5 to 6 seconds, you can base your process control function on the trending of the results and ignore occasional readings that are outside the control range. This is a pratical solution our customers use very frequently, in fact, even pharmaceutical products that are produced in such control method are acceptable. The price you pay for easing your way into calibration is by smaller allowed control range - but it may be worth it, depending on how good is your process and your ability to control it. If you wish to hear more, send e-mail [email protected] Gabi Levin | ||
| Peter Tillmann (tillmann) Intermediate Member Username: tillmann Post Number: 18 Registered: 11-2001 |
Dear Alisha, being a friend of validation I see no problem to calibrate on NIRS data, if you validate your NIRS calibration properly. These validation samples have to be analysed with the reference method. There are three reasons to my statement: - If the calibration on the first NIRS instrument is a good one (properly validated !!), the NIRS predictions on new samples are good ones as well. - In several occasions you can analyse many more samples with the first NIRS instrument than with the reference method, adding more information to the calibration based on "sample set heterogenity". - It doesn't matter where you get your calibration from, as long as it is validated properly. (You might even steel it from someone else.) BUT: During the calibration phase the named high correlation from target data (NIRS data from the first NIR instrument) and NIRS spectra from the second instrument will cause all cross validation or other statistical rules (choosing number of PLS factors, ...) to fail. So this is not a straight forward approach to develop NIRS calibrations anymore. Adding noise will not help because it can't model the specific information of the "true reference data" in the samples properly. Yours Peter Tillmann | ||
| Alisha (agnosus) New member Username: agnosus Post Number: 4 Registered: 1-2009 |
Thanks for your comments. I appreciate them a lot. But I think what I was trying to get at was not a calibration transfer case (or instrument standarization). Imagine that we have a well established NIR calibration that is working fine in the lab. Now we want to calibrate a different NIR instrument online to analyze the same type of sample in process. What I was trying to understand was that if it is ok to sample from the process (different temperature, pressure, etc.) and then analyse them using the lab NIR and then using the results as reference to calibrate the process NIR. Is this a valid procedure? Best regards, Alisha | ||
| David W. Hopkins (dhopkins) Senior Member Username: dhopkins Post Number: 144 Registered: 10-2002 |
Alisha, Howard and Jerry are valued and respected friends, with much experience in NIR calibrations. However, I expect that they have more experience now, and might not object to that approach so strenuously now. The calibration on the first instrument will have its own fingerprint to the data, especially regarding the geometry of light collection and many other factors. However, if the calibration is well done in the first place, the calibration responds to the chemical information in the spectra to give valid predictions for the set of samples used for the calibration and validation. Now, if you run selected samples on a second instrument, you should be able to see which samples should be submitted to the lab for reference analyses. Then when you obtain good reference values, as Howard points out you must do, you can slope and bias adjust the results to obtain results that should reflect the chemical information in the second instrument and compensate for systematic differences between the second and first instruments. There are many ways to perform such a calibration transfer, that is a big topic. In addition, you will want to run further samples to validate your calibration transfer. At the end, you may find that you have a perfectly serviceable calibration for your needs. You may have used fewer samples, had to spend less money on reference analyses, and obtained your calibration faster than had you started your calibration from scratch on the second instrument. However, in this process you may find that you analyze sufficient samples in the end to support regression and validation. It is highly likely that the specific regression for the second calibration will perform better than the transferred calibration. But that difference may not be critical to your needs, and you will have had the calibration in service much earlier by using the calibration transfer method. I have heard of people who were forced to use the calibration transfer method, due to instrument repair after using a calibration successfully. The repair changed the response enough that the calibration no longer tracked the sample constituent values properly, requiring fast adjustment of the calibration. That is perhaps the largest reason that the techniques have been developed. Best wishes, Dave | ||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 325 Registered: 9-2001 |
Alisha - as one of the authors of that article, I have to say that your proposal is risky. Consider that one of the (computer) experiments that led to our recommendation used exactly the fact that both datasets had random numbers for their "constituent" values. And don't forget that since you would still need some actual reference lab values in order to come up with the calibration in the first place, so then you might as well just read those same samples on the second instrument. All that said, if you want to try implementing your suggestion, I would be extremely interested in your results. Howard \o/ /_\ | ||
| Alisha (agnosus) New member Username: agnosus Post Number: 3 Registered: 1-2009 |
Dear all, I recently read a very good paper: Howard Mark, Jerome Workman; Spectroscopy; Jun 1, 2007 In this article authors emphasize that "an NIR calibration should never, ever be obtained from another NIR instrument" due to the fact that "The coefficients of the models are merely multipliers of the systematic effects present in the underlying spectral data. Because those are the same in both instruments, then of course the predicted values agree between the two instruments and give a very high correlation between the instruments' predicted values" (except for when the calibration model is reflecting noise structure in the spectra [overfit]) Now this is my question: Is it possible to manipulate (e.g. by adding random noise) the predicted values of one NIR system and then use the transformed values to calibrate the second system? Will appreciate your comments. Best regards, Alisha |