| Author | Message | ||
     Charles E. Miller (millerce) Intermediate Member Username: millerce Post Number: 19 Registered: 10-2006 |
I see nothing confusing about Barry's comments- If I read correctly, they simply mean that OSC in "model mode" can use measured Y to calculate the correction, whereas OSC in "apply mode" must use some estimate of Y to do the same. This could well explain Leonardo's discrepancies (although I think he just found another root cause�) In manufacturing/GMP environments, I like to use the simplest tool that will do the job- this tends to minimize the chances of algorithm/tabulation discrepancies that raise the "red flag" to Quality folks/regulators, and wreak havoc on any method transfer efforts. Also, I echo Dave's experience that I've never observed cases where OSC has improved prediction performance within statistical significance. I've seen a few cases of prediction improvement with the wavelength-localized version of "Piecewise OSC"- although this is even more complex to implement. IMHO - OPLS is an intriguing and clever attempt to balance interpretability with prediction relevance in a single algorithm. However, as these two objectives are inherently different, I see nothing wrong with using 2 different modeling tools (say, PCA and PLS) to attack these two objectives separately� ..This is especially the case if deploying the final models is simpler algorithmically and conceptually. For interpretability: Yes- it can be annoying when the PLS/PCA factor axes do not line up exactly with known sample properties. However, in such cases, there is a much simpler time-tested tool called factor rotation (first cited in 1947) that simply rotates the original PCs/LVs in user-preferred directions, while preserving both orthogonality and the original model sub-space. (for PLS Toolbox users, one can try the function "MANROTATE.M"�). Regardless of the interpretative modeling tool one chooses to use, there is still "burden" on the user to perform additional "auxiliary" experiments (ex. collect pure component spectra, spectra of materials in different states) to maximize the overall "interpretative experience": Just relying on an algorithm usually won't get it done. Best Regards, Chuck | ||
| Ciaccheri Leonardo (leonardo) Member Username: leonardo Post Number: 11 Registered: 5-2010 |
Dear all, I agree with Johan on a point. It is true that OSC use the knowledge of the Y variable in calibration, but once you have extracted a loading vector from 'osccalc', when you apply that loadings to the same spectra you used for calibration you should obtain the same corrections that 'osccalc' gave you. I made some experiment with 'osccalc' and, maybe, I have found the cause of this discrepancy. In the final steps of the OSC procedure, after 'osccalc' has generated the "orthogonal" vector, it uses PLS to create the final weight vector, w (the one given as output), then uses it to generate the output score vector, t. At this point a last orthogonalization step is made: t = t - y*pinv(y'*y)*(y'*t) (y is the target variable) and THEN the output loading vector, p, is created. This means that w and p are not linked to the same t vector. When you multyply your spectra by w you get the 'old' t, but it is the 'new' t the must be multyplied by p to get the right corrections. I am not able to understand the role of this last orthogonalization. I only know that if I de-activate this instruction (turning it into a comment line) the corrections become reproducibile as I expected they were, but I don't know if removing this step can lead to wrong results and predictions. Best Regards. Leonardo | ||
| Johan Trygg (trygg) Junior Member Username: trygg Post Number: 9 Registered: 1-2006 |
Hi again, Barrys argumentation is remarkably lengthy and confusing. As Leonardo pointed out, you should of course get the same output if you use identical training and test data in a multivariate calibration model. The OSC method (Wold), Fearn and H�skuldsson, and the OPLS method all yield identical results. Why PLS toolbox variant does not is a bit worrying actually, but I leave it at that. When I say that OPLS and PLS performs similar in terms of prediction, I am talking about the algorithmic properties of the model. However, one should not neglect the power of model intepretation to aid in the selection of a good training set, identification of outliers and for deciding the number of components. As proof, I want to mention a recent paper in the prestigious NATURE BIOTECHNOLOGY journal where OPLS did actually give MUCH BETTER predictions compared to PLS. This is not due to overfitting or anything like that, it was coordinated by the FDA and one could only submit ONE candidate model, not several. Reference: Leming Shi et al. , The MicroArray Quality Control (MAQC)-IIII study of common practices for the development and validation of microarray-based predictive models, NATURE BIOTECHNOLOGY, 28: 827-U109 2010 To my knowledge, the underlying reason of this success was that OPLS was able to aid in the model creation process by enhanced model interpretation, rather than relying on statistical measures. Actually, OPLS ranked best of all different types of prediction methods, including SVM, for one of the endpoints. Hence, OPLS DOES IMPROVE model interpretation. More than 300 citations (in Journal of Chemometrics, 4th all time) and exponentially rising every year is not by coincidence. The concept of Orthogonal variation is here to stay. However, in the field of NIR spectroscopy (where model interpretation is more difficult and timeconsuming) it has not yet had such an impact as in metabolomics. I would be happy to have a constructive discussion with you who are experts on NIR spectroscopy to see why it has not been used as much in this field. Or better yet, how we can work together to showcase these types of methods also do work for NIR spectroscopy. Will visit South Africa's famous wine institute in Stellenbosch next week and hopefully this work can get started with their NIR spectroscopy group. Gabi: If you get more than one PLS component for a single-Y PLS model, then Orthogonal variation is present in the PLS model, and you would benefit using OPLS for interpretation of what that variation actually constitutes. Leonardo: I have sent you several OPLS papers by email. best regards, Johan Ume� University, Sweden | ||
| Barry M. Wise (bmw) New member Username: bmw Post Number: 1 Registered: 2-2011 |
As Gabi Levin experienced and Johan Trygg pointed out, on average, OSC DOES NOT improve predictions on separate test sets. The initial results from Ume�/Umetri/Umetrics were overly optimistic on that point, in part because of the (unusual) way they used to do cross-validation. It does make for simpler models, though you then also have to try to interpret the OSC factors. OPLS also DOES NOT improve predictions, though some people prefer it for interpretability. As to why, with OSC, you get slightly different results on the calibration X data and the same test X data: it is the difference between fit and prediction. In calibration mode, the algorithm uses the corresponding Y data to find the orthogonal components in X. In application mode, Y is unknown, so the amount of each component to remove must be estimated. You can tweak the parameters in the that estimation, but in general (unless you overfit it), you won't get exactly the same thing on X in calibration mode (with Y present) as you do on the same X in apply mode (without Y present). If you want to do something similar to O-PLS in PLS_Toolbox/Solo, (I call it faux-PLS, ba ha ha), you can use the "Orthogonalize Model" function. It works with PLS and PLS-DA models. See our wiki page about it at: http://wiki.eigenvector.com/index.php?title=Orthogonalizepls If you want to try something that might actually improve prediction error, I suggest trying some the "Clutter Filter" variants we've made available, including Generalized Least Squares (GLS), External Parameter Orthogonalization (EPO), and Extended Mixture Model/Extended Least Squares (EMM/ELS). You can find out more about using these methods in PLS_Toolbox/Solo at: http://wiki.eigenvector.com/index.php?title=Declutter_Settings_Window There are a number of movies that we've made using these "Declutter" techniques available at: http://eigenvector.com/eigenguide.php I've had particularly good luck using GLS with the clutter defined by the Y-block gradient method. For a movie showing how to use this, see: http://eigenvector.com/eigenguide.php?m=Advanced_Preprocessing_5 If you want to learn more about all these methods, I suggest you attend Eigenvector University, May 15-20 in Seattle: http://eigenvector.com/courses/EigenU.html Let me know if you have any questions. BMW | ||
| David Russell (russell) Senior Member Username: russell Post Number: 53 Registered: 2-2001 |
Thank you Johan for the detailed reply. I have experienced improved visualization and understanding of data sets using OSC however did not experience improvement in prediction performance. Thank you for validating my experience. Leonardo - My PLSToolbox license is for V4. However there have been updates. Perhaps oscapp was released as an update. | ||
| Ciaccheri Leonardo (leonardo) Junior Member Username: leonardo Post Number: 10 Registered: 5-2010 |
Dear All, Thank you for your kind assistance. You guessed right, David, the function 'osccalc' I have is that of Eigenvector Research, but I have no the function 'oscapp' on my PC. I don't know why, bacause the PLStoolbox was already installed when I begun to work with it. Maybe it was not installed correcly or maybe it is an old version (I had still MATLAB 4 when I begin to prcticize with chemometrics, many yesrs ago). Dear Johan, O-PLS looks very interesting, and I would like to receive the pdf of your paper about it. Thank you very much. My e-mail address is <[email protected]>. Best regards. Leonardo | ||
| Gabi Levin (gabiruth) Senior Member Username: gabiruth Post Number: 56 Registered: 5-2009 |
Dear Johan, Many thanks for your comments. This is actaully good help for me - as a practitioner - i.e. one who is primarily involved in providing best predictions to process users - I was always concerned that maybe we do not provide the best tools to my instrumen customers (Brimrose). Since I tried it and did not see any improvement I jsut gave it up. As for interpretation, in a process, there is no room for experimentation - it is a given, so too much interpretation does not yield any advantage to the user, unless there are process problems. In most cases the LW of the Unscrambler are sufficient for my needs. Since you mention that these methods can help to detect non lineariy, drift, base line - how much do you expect a real time dual beam for each wavelength instrument to be influenced from drift and base line? It is my understanding that if you have a dual beam design, for each wavelength, in real time these effects should not play a role, what do you think? Gabi Levin Brimrose | ||
| Johan Trygg (trygg) Junior Member Username: trygg Post Number: 8 Registered: 1-2006 |
Hi Gabi, If you only have an interest in the prediction error and not model interpretation, these methods will not help you. These methods do not improve prediction performance compared to only applying the PLS model. Instead, these methods provide a new concept in terms of model diagnostics, visualization and interpretation compared to the PLS model. This in turn, helps the scientist to produce even better results in the next round of experiments and in this way actually achieve better predictions. Sample workup problems, experimental problems, drift,non-linearities, contaminants, side-reactions, moisture problems are all identified in both size and profile as part of the Orthogonal variation. For instance, in a binary mixture example, we used these methods and identified non-linearity in the collected NIR spectra. With the PLS model and its components, this was not the case. What PLS did was to mix the two types of variations (orthogonal and predictive) together, this created less interpretable model components but with same predictions. In pharma industry, GlaxoSmithKline and AstraZeneca are looking into these methods to understand the relation between Chemistry and Physical process parameters. All part of QbD and process understanding along with FDAs goals. In medicine and biology, model interpretation is much more interesting than prediction. They want to understand why male and female separate, why healthy controls separate from patients diagnosed with a disease, and so on. These methods have helped biologists to understand the molecular phenotypes of not only disease (predictive variation), but also its relation to demography, gender, age and so on. I am happy to send you several references on applications from pharma industry, medicine and biology. I understand that NIR spectral interpretation is difficult and time consuming on a molecular level, but these methods will highlight such things as non-linearity, base-line problems, drift, sample handling and storage, and so on. So I hope my arguments will encourage more people, not less, to try these methods, I prefer of course OPLS. regards, Johan Trygg, Assoc. Prof. Ume� University, Sweden | ||
| Gabi Levin (gabiruth) Senior Member Username: gabiruth Post Number: 55 Registered: 5-2009 |
Hi guys, I have two conflicting pieces of eveidence, but maybe not really conflicting - long time ago, 2000 or so I tried the Umetrics OSC (done by them) on a set of data from a process. The regression really looked better by a nice reduction in SEP. However, when later tried on additional data from the process we were not able to realize most of the improvement. We did not pursue this option any further. About 1 year ago we gave people from Camo a set of data to implement their OSC - but the regression didn't look any better than with the regular 1st derivative. It is not said in order to detter peole from trying it, but to let people to know that it is not necessarily the golden answer to regression problems. Gabi | ||
| Johan Trygg (trygg) Junior Member Username: trygg Post Number: 7 Registered: 1-2006 |
Dear Leonardo, The problem you are experiencing is that the success of the original OSC method (published by Svante Wold et al in 1998) got many researchers and software companies during 1999-2002 to want to create their own unique filtering method all with different results. Unfortunately, many used the same name as the original method which caused, and still causes a great deal of confusion and frustration as in your case. The one you mention in PLS toolbox is one of those unfortunate variants, whose result becomes not only different, but as you also observed, seems to simply not work as it should. The original OSC method has been implemented in SIMCA-P software by Umetrics (www.umetrics.com) where you can download a fully working demo to test. I would also encourage you to have a look at the OPLS method (reference below), which is a combined PLS method and OSC filter that has received much attention the last few years. OPLS calculates two types of components, Orthogonal and Predictive, all in a single PLS-like prediction model. Let me know if you want me to send you the pdf:s for both papers, best regards, Johan Trygg, Ume� University, Sweden Reference, Wold S, Antti H, Lindgren F, O� hman J. Orthogonal signal correction of near- infrared spectra. Chemometrics Intell. Lab. Syst. 1998; 44: 175�185 Trygg J, Wold S. Orthogonal projections to latent structures, O-PLS. J. Chemometrics, 2002; 16: 119-128. | ||
| David Russell (russell) Senior Member Username: russell Post Number: 52 Registered: 2-2001 |
Is the Matlab(R) function originally used the version of osccalc provided as part of PLSToolbox from Eigenvector? Did you use function oscapp to apply the correction to the "new" spectra? If not, please try again using oscapp and see if you get a correct result. Dave Russell | ||
| Howard Mark (hlmark) Senior Member Username: hlmark Post Number: 400 Registered: 9-2001 |
Leonardo: while some chemometricians monitor this NIR discussion forum, there is also a Chemometrics discussion group (sponsored by NAmICS (the North American chapter of the International Chemometrics Society)), where you'll probably find more people familiar with the OSC technique than will be monitoring this one. Howard \o/ /_\ | ||
| Ciaccheri Leonardo (leonardo) Junior Member Username: leonardo Post Number: 9 Registered: 5-2010 |
Dear All, I have recently made some experiment about Orthogonal Signal Correction (OSC) that leave me quite astonished. I made OSC on a set of spectra, using the MATLAB function 'osccalc'. It gave me, beside the matrix of corrected spectra, the weights and loadings for correcting subsequent spectra sets. The formula for spectra correction, as reported in the MATLAB code, was the following: Sc = S - S * whg * inv(ldg' * wgh) * ldg'; where Sc are the corrected spectra,S the original spectra, wgh the weights, ldg the loadings, inv means matrix inversion and ' indicated trasposition. Well, I tried to apply this formula to the SAME SPECTRA I used to evaluate the model (in order to test it) and I was surprised when I DID NOT GET the same corrected spectra matrix that 'osccalc' had given me as output. I was a bit worried about the possibilty of some error in the scaling formula or, maybe, in the MATLAB code, thus I tried the same experiment with Unscrambler X. Ha ha! Unscambler provides both to create and to apply the OCS model, therefore any errors in the projection procedure are ruled out. Or, at least, I believed this. I made a calibration set of 14 spectra and duplicate it to create a test set (I made the same with the target variable). Then I evaluate the OSC model with the former set and applied it to the latter but, again, the two set of corrected spectra WERE NOT EQUAL. Now I am quite worried. Am I making some gross mistake? Or there is something that don't work in the OCS algorithm? May anyone help me? Leonardo |