| Author | Message | ||
     Forrest Stout (forrest) Member Username: forrest Post Number: 15 Registered: 7-2006 |
Dongsheng Bu, I'm not ignoring your comment, I've just been busy. I'll try to look into that data, later. | ||
| Dongsheng Bu (dbu) New member Username: dbu Post Number: 4 Registered: 6-2006 |
Forrest, I just prepared a matlab script for SNV and Detrending. Can you check if the result is the same as yours? I also have two questions regarding SNV usage: 1) Can STD in the equation (x-mean)/std be replace by spectral area in some cases? 2) Is it true that SNV is only suitable for the cases of small concentration changes. % data is part of 30th spectrum from % Harald Martens, et. al. Analytical Chemistry, % Vol. 75, No. 3, February 1, 2003 394-404. % http://www.models.kvl.dk/source/EMSCtoolbox/index.asp data = [2.576,2.585,2.595,2.605,2.615,2.624,2.633,2.641,2.647,2.653,2.658,2.662,2.665,2.668,2.671,2.673,2.676,2.679,2.683,2.688,2.693,2.698,2.703,2.706,2.707,2.707,2.705,2.701,2.696,2.691,2.684,2.677,2.670,2.663,2.655,2.646,2.638,2.630,2.622,2.614]; w = 970:2:1048; % SNV SNV_data = (data - mean(data))/std(data); % Then Detrending % From PLS IQ/Plus Help % A linear least squares regression is used to fit a quadratic polynomial to % the responses in the spectrum. This curve is then subtracted from the % spectrum to give the result. The quadratic curvature component attempts to % correct for the effects of particle size and sample packing. [p,ErrorEst] = polyfit(w,SNV_data,2); % Calculate fit parameters pop_fit = polyval(p,w,ErrorEst); % Evaluate the fit SNV_detrend_data = SNV_data - pop_fit; subplot(211);plot(w,data); legend('Original spectrum'); subplot(212);plot(w,[SNV_data;SNV_detrend_data]); legend('SNV only','SNV detrending'); xlabel('Wavelength,nm'); | ||
| Michel Coene (michel) Senior Member Username: michel Post Number: 37 Registered: 2-2002 |
If I remember correctly, I divided the spectrum by the fitted curve and then did a normal SNV. But as I said, this was a very theoretical thing on ultra-pure components. In real life, "your mileage may vary", as they say in the commercials. | ||
| Forrest Stout (forrest) Member Username: forrest Post Number: 13 Registered: 7-2006 |
Michel, are you refering to "detrending?" If so, did you simply subtract the quadratic fit from the spectrum already pretreated via SNV? I have tried this and seen improvements, but not substantially. Also, I have noticed that in data with less scatter, pathlenghth variation, etc., this detrending and even SNV degrades results. | ||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 45 Registered: 9-2001 |
As Suresh said, the net effect of each on the calibration is essentially identical. I disagree, however, that the SNV is simpler, each requires two parameters, and to implement it for calibration is just a matter of clicking the proper button in the window. On prediction (or validation), there is a difference that is hardly ever mentioned, however. As you hinted at, Forrest, in order to use the calibration model you derive from MSC-transformed data, the computer must save and provide the mean spectrum that was used in developing the MSC correction, so that the future prediction samples can be fitted to the same mean spectrum that the calibration samples were fitted to. This is a detail of implementation that can be (and, I suppose, usually is) transparent to the user. On the other hand, I have never seen it mentioned and sometimes wonder if, in fact, the various implementations of MSC in different software packages take that into account, and save the mean spectrum along with the other parameters for a calibration model, when it is used. Howard \o/ /_\ | ||
| Michel Coene (michel) Senior Member Username: michel Post Number: 36 Registered: 2-2002 |
If you are used to looking at your spectra you might prefer MSC because the resulting spectra will be more similar to what you already know. As you mention yourself, SNV is more easy. I once did a study on different particle sizes of pure sugars and found that a quadratic version takes 99% of the differences away. i.e. you fit a quadratic curve trough your spectrum and then divide the spectrum by the curve. How this would work on real samples (not pure) I don't know. | ||
| Suresh Kumar B.V (suresh_kumar) New member Username: suresh_kumar Post Number: 1 Registered: 8-2006 |
MSC: Multiplicative Scatter Correction (MSC) is a transformation method used to compensate for additive and/or multiplicative effects in spectral data. MSC was originally designed to deal with multiplicative scattering alone. However, a number of similar effects can be successfully treated with MSC, such as: - path length problems, - offset shifts, - interference, etc. The idea behind MSC is that the two effects, amplification (multiplicative) and offset (additive), should be removed from the data table to avoid that they dominate the information (signal) in the data table. The correction is done by two simple transformations. Two correction coefficients, a and b, are calculated and used in these computations.The correction coefficients are computed from a regression of each individual spectrum onto the average spectrum. Coefficient a is the intercept (offset) of the regression line, coefficient b is the slope. Standard Normal Variate (SNV) is a row-oriented transformation which centers and scales individual spectra. Each value in a row of data is transformed according to the formula: New value = (Old value � mean (Old row) ) / Stdev (Old row) Like MSC (see Multiplicative Scatter Correction), the practical result of SNV is that it removes scatter effects from spectral data. An effect of SNV is that on the vertical scale, each spectrum is centered on zero and varies roughly from �2 to +2. Apart from the different scaling, the result is similar to that of MSC. The practical difference is that SNV standardises each spectrum using only the data from that spectrum; it does not use the mean spectrum of any set. The choice between SNV and MSC is a matter of taste. | ||
| Forrest Stout (forrest) Member Username: forrest Post Number: 12 Registered: 7-2006 |
Which of the two do you prefer and why? SNV definitely seems to be more inviting. (It is more simple and does not require a representative spectrum.) |