| Author | Message | ||
     Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 47 Registered: 9-2001 |
NIRManiac11 - I'd say that one premier reason that your results with the "new" calibration are the same as the results with the "old" calibration is exactly the first two steps in your filtering process: you removed all the samples from the new calibration that were different than the ones from the old calibration. So naturally, if you use only the samples that are operationally the same, you would expect to get essentially the same results. I showed, in "Principles and Practice of Spectroscopic Calibration"; Wiley (1991), that every calibration is a compromise. If you expand the universe over which the calibration is valid, you will inevitably degrade the performance - usually a little, sometimes a lot - but always to some extent unless you are extremely lucky in your application. The comparison has to be taken over the whole data set. So if you include the "different" samples, then you will do better for the extreme samples at the expense of the "normal" ones. Its up to you to decide on the compromise you want to make. I think you should do a couple of test calibrations using all the samples, to see how well (or badly) the old ones, as well as the new ones, perform when you do this. You never know: you might just be in for a pleasant surprise. If you don't like what you see, then you should delete samples selectively, keeping track of the performance of the calibration (again, the overall performance as well as the behavior of the new and the old samples). Then you can decide more intelligently on the compromise you want to make. \o/ /_\ | ||
| Dr.K.Balasubramanian (drkbala) New member Username: drkbala Post Number: 3 Registered: 9-2006 |
Create Reference and SST and carry on.You have got quite a good no of samples and scan at least 5 per sample to have a good average spectra | ||
| ksharghi (nirmaniac11) New member Username: nirmaniac11 Post Number: 5 Registered: 6-2006 |
I've been keeping up with reading the new postings and have found myself in a position similar to the last thread, but in fact I have an excess of real samples I am trying to use in building a new calibration model. Here is my situation: I have over 500 "real" samples that have been analyzed using my existing quantitative calibration model (n = 83). I want to introduce the variability of those 500 spectra, which were collected for over a year, into my current model to make it more robust with respect to changes in response factors such as temperature and humidity. I began by visually looking at each of the 500 new spectra zooming in on my calibration regions and removing spectra which I could label as unrepresentative due to the appearance and location of band peak intensities. After filtering usable spectra from unusable, I inserted the usable spectra into the old model, bringing the calibration population to n = 260. My next step was to remove calibration standards that differed by more than 1.0% from the actual value measured by the primary method. This brought my calibration population to n = 140 (40 original model standards and 100 new standards). What I am seeing when comparing validation samples with the new model and old model is that there is not a large difference in values between the two. Some samples are improving by 1.0% (closer to the actual), but samples that had +/- 4-6% difference from the actual value using the old model are still being predicted in this range with the new model! The questions I pose to the NIR community follow: 1) Is this the right way to improve calibration models? 2) Why is there not a greater difference between the two models in predicting sample values? 2a) How can I achieve better NIR predictions? 3) Is there an "ideal" number of samples to use in a calibration model? How many is too many? |