| Author | Message | ||
     Karl Norris (knnirs) Senior Member Username: knnirs Post Number: 61 Registered: 8-2009 |
Hi Fran, You may be interested to know that your intersts were the ame as two young college graduates 60 years ago. I met Dave Packard and Bill Hewlett in their garage workshop, and they described work they were doing to detect weeds in sugar beet fields. Yes, these are the same gentlemen who formed the Hewlett-Packard Company. Perhaps you can do as well. SEE en.wikipedia.org/wiki/Hewlett-Packard. | ||
| Tony Davies (td) Moderator Username: td Post Number: 272 Registered: 1-2001 |
Hi Fran, I'm not familiar with your spectrometer. What is the reproducibility of your spectra? PCA is not the best tool for discrimination but it is a good starting point. My favourite discrimination method is Canonical Variance Analysis (CVA). You put the PC scores and the sample identity (B or W) into the program and it will try to find a vector (or vectors) which will discriminate B from W. Then you try to validate it on data not previously used. There are lots of similar techniques (some are the same but have different names!). Hope this helps! Best wishes, Tony | ||
| Francisco Garcia (ssisscu) New member Username: ssisscu Post Number: 5 Registered: 11-2011 |
Hi Tony, I go to the field and take samples from different plants, the same species each day (kind of temporal measurements), I do not prepare them in any way because I just want to find a way of "diferenciate" Crop (sugar beet) and weeds in field conditions (natural conditions), it is not a weeder, but it could be in the future... I know that this natural conditions can be a problem because there is kind of mixture of spectra in the sampling spot, the spectrum is not 100% pure since there can be dust and other elements attached to the leaf, but I have no time to control all factors... I use a SVC spectroradiometer GER1500 range from 350 to 1050 nm (http://www.spectravista.com/1500.html). I use the lens with 4�FOV. It measures relative reflectance. Actually it measures radiance from a spectralon and radiance from the target and makes a ratio to calculate the relative target reflectance. Dave, as Francesco says, this beets are used for sugar production and they do not have red color in their leaves, at least not in the early stages, when we do want to find the weeds in order to treat them. I'll be glad to answer as much as I can, and as much as I know... But everything is still in preliminary stages and don't really know how will be developed in the future. As I see from your coments, I have a lot to try and to study! Best, FRAN | ||
| Francesco Davini (franz) Intermediate Member Username: franz Post Number: 18 Registered: 2-2009 |
David W. no, sugar beet is white. See http://en.wikipedia.org/wiki/Sugar_beet And the leaves are green. No traces of red, as far as I know. | ||
| Tony Davies (td) Moderator Username: td Post Number: 271 Registered: 1-2001 |
Hi Fran, And "Welcome!" from me. One of the problems of asking the discussion group is that we always have more questions than you! (And we often start arguing amongst ourselves!). What I have understood (or guessed) is that you are looking at leaves from different plants and different days? Do you have different plants on the same day? How many days between measurements? [Are you hoping to make an automatic weeder?] If you are looking at leaves, do you prepare them in any way and how are you measuring them? [Instrument, wavelength range, transmission or reflectance ...] If you are just trying to separate your two groups of samples, I would suggest you look at the PCA results with and without MSC. As you are using PCA (not doing a calibration) it is quite OK to do separate MSCs on each group. DO look at several PCs. sometimes interesting results turn-up in the later PCs. I would go to 10 PCs (you do not have many samples). Good luck and best wishes Tony | ||
| David W. Hopkins (dhopkins) Senior Member Username: dhopkins Post Number: 207 Registered: 10-2002 |
Hi Fran, I'm not familiar with the appearance of sugar beets. Do they have a red appearance, like beets, due to anthocyanins? In that case, gathering spectra down to 500 nm or so might help in the discrimination. Do you have that option? Best regards, Dave | ||
| Francisco Garcia (ssisscu) New member Username: ssisscu Post Number: 4 Registered: 11-2011 |
Hi Dave, Sorry, I wanted to write species. Sugar Beet and Weeds. They are similar in the spectrum but with some differences due to their leaf morphology. Let's see what we can do! Best, FRAN | ||
| David W. Hopkins (dhopkins) Senior Member Username: dhopkins Post Number: 206 Registered: 10-2002 |
Hi Fran, I'd just like to ask for a clarification, do you mean spices, as in oregano, or species, as in apples vs cherries? I'm curious, which plants in particular are you interested to differentiate? It seems to me that this will be very challenging. I would expect that you might find differences between 2 adjacent apple trees, just because of local differences in growing or shading conditions. Best regards, Dave | ||
| Francisco Garcia (ssisscu) New member Username: ssisscu Post Number: 3 Registered: 11-2011 |
Thanks David, Well, I took 20 samples per day, but in fact only 10 leaves (2 samples per leaf). This are preliminar experiments to see how the data looks like and think about the statistics to be used. In reality I expect to collect more data, and many more days (6-7 during the season). The objective is to differenciate two plant spices from the spectral data. And PCA would be the first step for finding the bands with high variability between spices. Based on some bibliography that's what is commonly used and this would be my first approach... Thanks for your welcome wishes, hopefully I will learn and share a lot in this group. Best, FRAN | ||
| David W. Hopkins (dhopkins) Senior Member Username: dhopkins Post Number: 205 Registered: 10-2002 |
Hi Bilal, There is nothing in the MSC or SNV procedures, that precludes another pre-processing step, performed either prior to or after the MSC or SNV. I have found that a 2Der before the MSC gives a slight improvement. Others have reported that it works better to do the 2Der afterward. Or, a 1Der might be useful. I think one should examine the spectra, and determine what seem to be the major problems in the scans, and how you might solve the problems. Alternatively, it might be best to try several procedures and see what works best. The danger is, making the decision on the basis of insignificant differences in the statistics. Also, after more than one procedure, the validation step becomes part of the calibration process, and another validation step should be considered, with an independent set of samples. This may difficult in Francisco's case. We should also ask Francisco, how many independent samples did you take on days number one and two? Duplicate NIR scans don't count as independent. I would hope for 20 or more samples per day. What properties are you trying to measure? By the way, I neglected to offer a welcome to you to the discussion group, Francisco! Best wishes, Dave | ||
| Francisco Garcia (ssisscu) New member Username: ssisscu Post Number: 2 Registered: 11-2011 |
Dear all, Thanks a lot for your inputs. I'll have to look at all the options you suggest carefully. I kind of wanted to get rid of the incoming radiation variations between samples as well as scattering to focus only in the physiological differences between plants. That's why I was thinking about making two different MSC in the two different data sets (different days). Regarding Bilal's question, the actual pre-processing of my spectral data is based on a smoothing by using Savitzky-golay filter and scattering correction by doing a MSC, prior to performing the statistical analysis itself. As I am just initiating in spectroscopy, I would appreciate recomendations on that of course. Thanks again, FRAN | ||
| David W. Hopkins (dhopkins) Senior Member Username: dhopkins Post Number: 204 Registered: 10-2002 |
Hi Francisco, Sorry, I should have made that read, "the aim of MSC (or you should consider SNV), is to remove the slope, offset and multiplicative or scale differences among the samples". Best regards, Dave | ||
| Bilal Ahmad Malik (elp09bm) Member Username: elp09bm Post Number: 11 Registered: 7-2011 |
Hi All, I would also like to jump in this question. I would like to add Francisco statement " I assume that it might be crucial to do the right pre-processing " what is right pre-processing technique for a data. How could one decide that which pre-processing technique one should use for a certain data from NIR. i.e. Should I use first derivative, second derivative or MSC or any other advanced signal processing filters. Thanks, Bilal | ||
| David W. Hopkins (dhopkins) Senior Member Username: dhopkins Post Number: 203 Registered: 10-2002 |
Hi Francisco, If you have only 2 days worth of day, you have 2 choices, you could use one day for the training set, and the second day for validation, or you can combine them and randomly select Set T and Set V. In either case, the aim of MSC (or you should consider SNV), is to remove the slope and offset differences among the samples, and I would recommend that you combine the 2 days of data for the calculation of the average spectrum to use for the target spectrum for the MSC. It may not make much difference, to use the average of day 1 for the target spectrum. You should try both methods, and see whether the PCA shows less difference between the days using one method or the other. To get around this problem, you should also try SNV, which does not require a separate of an average spectrum. In any case, you should recognize that the calibrations you develop will need to be carefully monitored in the future for accuracy, as 2 days may not represent very much of the future variation. Good Luck! Best wishes, Dave | ||
| Jose Miguel Hernandez Hierro (jmhhierro) Advanced Member Username: jmhhierro Post Number: 23 Registered: 4-2008 |
Hi Francisco, MSC works wiht the average spectrum,If you work with two different data sets you will use two different average spectra for performing this correction. I suggest to use only one data set. Regards | ||
| Francisco Garcia (ssisscu) New member Username: ssisscu Post Number: 1 Registered: 11-2011 |
Hi, I have two sets of spectral data from plants taken in two diffrent days. The day was sunny in both cases, but with slightly different incoming radiation measured by a Pyranometer. Also I assume differences in the fisiology of the plants in different days. I want to do a Multiple Scatter correction (MSC). My question is: Should I do a MSC for the two datasets independently? One MSC for each day? Or merge the two datasets and perform the MSC? After all this corrections I do a PCA, and the results are completely different, and I assume that it might be crucial to do the right pre-processing. Thanks for your help! |