| Author | Message | ||
     hlmark |
Solomon - the "obvious" answer is that you guessed wrong about the variables to use! In the original 1985 paper there were suggestions for how to automate the variable-selection procedure when computing Mahalanobis Distances, and there is software available that implements those. If you contact me off the discussion group we can discuss this further. Howard \o/ /_\ | ||
| NIR (Solomon) |
Dear experts, I am using NIRS technique for qualitative and quantitative analysis on CH compound mixed with water, in a pure form and slurry form. This CH compound has two types of forms (polymorphs). For qualitative analysis I used Mhalanobise distance to classify the polymorphs. In the first attempt I used the raw spectra. The raw spectra is affected by the slurry hence it has some base line variation, high baseline offset on polymorph 1, low base line shift on polymorph 2. The baseline offset is a function of concentration as well. I clearly see a classification on M-distance between the two polymorphs. On the second attempt I used the 2nd order derivative data of the raw spectral data, I see clear classification but in the reverse order. On the third attempt I chose particular variables which I thought responsible for the classification and used those variables but I could not see a classification of the polymorphs. My question what is the reason of the classification on the pervious two successful attempts? | ||
| hlmark |
Jon - you're ambitious beyond your years, I think, and it's a good thing, too. You're asking some pretty heavy-duty questions, and that's why you're having trouble understandng what's going on. To fully understand the answers you'd have to take courses in advanced math and statistics, although it may not seem like it to you now. Much of your questioning involves what's called "design of experiments": how do you set up an experiment, or a survey scheme to get the most information out of a limited number of samples? For perspective, this is how they conduct polls to "predict" the results of elections, for example. Some designs of experiments involve random selection, and some don't. Both are used, and both have particular characteristics. For example, when you propose to sort your data by constituent value and then select every third one: yes, that's a good way to insure covering the range of the constituent, but it's not a random selection, despite your claim that is it. The key calibration statistics (SEE, SEP) were designed so that the value of the statistic during calibration should be a good estimator of the value you should be able to achieve on prediction. There are several requirements for that, though, the main one being that the two data sets (calibration and prediction) should represent the same population. In the absence of complete prior knowledge about the samples, the best way to assure that is random sampling (and large enough data sets). Yes, you might get a sample with an unusually large error in the prediction set, but if you've done everything right, there's an equal chance that there will be a similar sample in the calibration set. And if your data set is "large enough" the effect of that sample will be "averaged out", anyway. Believe it or not, all this is included in the math behind the simple-seeming equations that we wind up with and use; those definitions were not chosen arbitrarily or lightly. If you want to improve your results by averging lab values, that's also a good thing to do. But then why not do it for the calibration samples too, and have better results to start with? In fact, if you don't do that, your "better results" will be spurious, since they will not be a true representation of what you can expect to get when you put your calibration model into routine use, when you will not be checking them. Then your true error will be the higher value that you'll see when you don't average - - you just won't be seeing that. There's a theorem called the Central Limit Theorem that guarantees that. As for the meaning of SEP, yes 68% of the errors will be within 1 SEP - - as long as the errors are Normally (Gaussian) distributed. But the Central Limit Theorem also guarantees that they will be, in the absence of known sources of non-Normality. In other words, it's usually safe to make the assumption of Normality, unless you have specific reason to think otherwise. Howard \o/ /_\ | ||
| jon |
Hello, my name is Jon. I am doing an honours project developing NIRS models to predict constituents in eucalypt leaves such as condensed tannins, nitrogen and oils. I have made a previous posting(-�anyone using a bruker MPA?�.�-thanks again to those who responded). I�m having difficulty understanding the various methods involved in developing and evaluating the models. Some of the questions I have are � - If the difference between the chemistry duplicates is high (e.g. greater than 5%) is it better to not include them in the model, redo the duplicates or put them in the model anyway and concentrate more on analysing the validation samples more accurately? ie. The lab chemistry error is random so you still end up with the right equation, but then make sure the chemistry for the validation samples is really good(eg done in triplicate) so that when you test the model you don�t get a low SEP simply because the reference values are incorrect? - I am also wondering about the standard error of performance. I�ve seen this written in various ways, but I�m still not sure how it is measured and what the output means( e.g. SEP = 0.07) Is it that 68% of the predicted values in the independent validation set will be 0.07 out. For example if my Nitrogen SEP is 0.07 does that mean that 68% of the time(1 standard deviation) if I have a known chemical value of say 1% nitrogen, the predicted value would be between 0.93 and 1.07%? - Also in the literature I have read there seems to be two ways to test a model independently( in addition to cross-validation). One is to select samples from the same population and build a model using two thirds of these samples to independently test the remaining 2 thirds, so its independent validation but still using the same population (internal independent validation?) And then there�s validation where the samples from one population are used to predict samples from another population (external independent validation?). I know that the model should include representatives from all populations that are to be sampled so the first approach I mentioned seems best, but it also seems that it would be useful to know how robust a model is regarding whether it would predict samples from a different population � are both approaches useful? - I�m also wondering whether internal independent validation should be conducted on a random subset of samples, which many people seem to do, but it seems that if its random the r/square validation is influenced by how much those validation samples happen to cover the range of the calibration model, i.e. if they are randomly selected maybe they won�t test the whole range of the model. I was wondering whether it would be possible to arrange the known chemical values of the population from which the calibration and validation samples will be selected in order, and then select say every third sample (and develop the calibration model from the remaining two thirds) so that you end up with randomly selected validation samples that you know are at least rouphly covering the range of the calibration model. If anyone could help e out on these issues or know of any good articles or journals that could be useful regarding these technical issues, I would be very grateful. Thank you in advance! Jon |