| Author | Message | |||
     Andrew McGlone (mcglone) New member Username: mcglone Post Number: 2 Registered: 2-2001 |
Dune effect? I�d never heard of the name but it resonated metaphorically with a vision of a steeply eroding sand dune; seemed very apt. So I reached for the grey bible to find out more. I found that the term is actually the �Dunne effect�, as in Dunne and Anderson, Can. J. Plant Sci. (1976). I�m a little disappointed not to have the metaphor�. | |||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 21 Registered: 9-2001 |
Pierre - interesting how we come to the same conclusion by different pathways. But after all, truth is truth no matter which way you look at it. I'd forgotten about Phil's description, though, thanks for reminding me of it. \o/ /_\ | |||
| Pierre Dardenne (dardenne) Junior Member Username: dardenne Post Number: 15 Registered: 3-2002 |
Jose, Howard is right. This is because your model has to weak correlation. By definition of the least square R(correlation)=SDyest/SDyref. For every prediction, the lowest values are overestimated and the highest values are underestimated. This was quite well explained as the �dune effect� by Phil Williams in the grey bible �near infrared technology in agriculture and food industries� Pierre | |||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 20 Registered: 9-2001 |
Jose - I note that you plotted the residual versus the lab values, rather then the predicted values. When the errors are small, it doesn't matter, since the two sets of values will be almost identical. When the errors are large, as in your case, then the plot you get will be different depending on which plot you make. This is discussed in Draper and Smith's "Applied Regression Analysis". They had a whole book in which to discuss these sorts of details and is worthwhile reading, but obviously I can't repeat their whole discussion here. However, the root cause of this sort of behavior of the data is one of two things: 1) Large random error in the X-variables used in the calibration 2) Lack of predictive capability of the model. In some senses, these are the same. In either case, however, the model simply tries to predict the mean of the data set, since in the absence of predictive capability of the data, the mean is itself a least-square estimator. If that is the only "predictor" it has, then that is what it uses. Therefore you get what statisticians call "bias toward the mean". Howard \o/ /_\ | |||
| j.u.m. (jose) New member Username: jose Post Number: 2 Registered: 3-2005 |
Hello everybody, I�m trying quantification in routine analysis as a way of controlling the dosage of production prebatches. I have built many equations with 59 production prebatches (not lab mixes) and later I predicted 74 new ones. While doing the latter, I saw that predictions tended to make up for the real value: if the prebatch had a lower-than-normal value, the prediction was above the real value, and if it had a higher-than-normal value, the prediction was below the real value. You can see what I mean in the attached file. This happened with all the equations I had built, and it also happened when I edited the regression methods. Do you know why? My spectrometer is a Foss XDS, prebatches are composed of calcium carbonate, cellulose ether and redispersible powder, and the reference values are taken directly from the weighing device (I know, some amount of the ingredients is lost over the production process. At present I just want to show that NIR quantification works). Thanks, Jose
|
prediction results graph