I'm looking for a formula for three f... Log Out | Topics | Search
Moderators | Register | Edit Profile

NIR Discussion Forum » Bruce Campbell's List » I need help » I'm looking for a formula for three filter types used in Perkin Elmer Software « Previous Next »

Author Message
Top of pagePrevious messageNext messageBottom of page Link to this message

hlmark
Posted on Wednesday, October 19, 2005 - 12:48 pm:   

Lingfen - I don't know what the algorithms are, and so I can't tell you, but there is a way to find out what the net shape of a filter is, at least for simpler filters. This may not work for more complicated filters such as the ones using a Fourier Transform or other filters equally complicated. However, for simpler filters such as Savitzky-Golay or other convolution-type filters it should work. With care and a little ingenuity it can also tell you some more about the algorithm, but that is speculative. But if you can do it at all, then you can try different spectra, for example, with different numbers of data points, resolutions, etc., and compare results.

Anyway, in order to do this you need to be able to generate a synthetic spectrum. If your software will accept JCAMP-DX format files, then this may be the easiest way to set the data up, since that can be done completely from scratch using NOTEPAD or some other ACSII text editor.

What you need to do is to make a synthetic spectrum that is zero everywhere except for one data point in the middle, that data point should have a value of one (unity). Then apply the filter to that spectrum. The resulting spectrum will reflect the net values of the filter. With complicated filters such as those using an FT, or ones that depend on estimating the spectral noise level (which would be zero for the synthetic spectrum) this approach may not work, but for many types of filters it should give you a lot of information about the filter that is being applied to the data.

If you need help on this, feel free to contact me off the discussion group; I'd be pleased to work with you.

Howard

\o/
/_\
Top of pagePrevious messageNext messageBottom of page Link to this message

Lingfen Rao
Posted on Wednesday, October 19, 2005 - 10:37 am:   

Dennis,

I have the same interest in this topic. I am a spectroscopic person and is starting to use lots of chemometrics.

My experience with PE filters is that the resolution filter really works for my IR and NIR data.

However, I do not know the algorithms and I am desperately want to know. Have you find out the algorithms? Would you please let me know what you find out?

Lingfen Rao
Top of pagePrevious messageNext messageBottom of page Link to this message

lingfen Rao
Posted on Wednesday, October 19, 2005 - 10:27 am:   

Dennis,

I have the same interest in this topic. I am a spectroscopic person and is starting to use lots of chemometrics.

My experience with PE filters is that the resolution filter really works for my IR and NIR data.

However, I do not know the algorithms and I am desperately want to know. Have you find out the algorithms? Would you please let me know what you find out?

Lingfen Rao
Top of pagePrevious messageNext messageBottom of page Link to this message

Dennis Mooibroek (Dennis)
Posted on Tuesday, February 04, 2003 - 3:52 am:   

Hi all,

Currently I'm working with software provided with a Perkin Elmer NIR (Spectrum IdentiCheck) which enables the user to use different types of filters.

I really would like to know the algorithm used for these filters to have a better understanding when to use them and when not.

Below is the description for each filter as provided by the User's references.

Resolution weighting:
A weighted derivative function emphasizes the influence of features with widths if approximatly 8 to 16 cm-1, corresponding to absorption bands. It discriminates against both broad baseline features and high frequency noise.

Intensity weighting:
A black body filter reduces the influence of data in those regions at the ends of the spectrum where the signal to noise ratio is lower.

Noise weighting:
A black band filter reduces the influence of regions where transmission is low because they have high noise in absorbance.

Hopefully anybody out here can help me.

TIA!
Top of pagePrevious messageNext messageBottom of page Link to this message

hlmark
Posted on Tuesday, February 04, 2003 - 7:17 am:   

Dennis - unless anyone has specific knowledge of how the P-E algorithms work, the best I can do is to give you a answer based on my general knowledge of how these things are done.

Resolution weighting - this is almost certainly done using a Fourier Transform (FT). The basic concept of an FT is that each frequency of the signal is converted into a single data point in the transform space, the higher index points representing the higher frequencies (which , in turn, correspond to sharper bands). Once you've done that, reducing the high-frequency noise and low-frequency drift is simply a matter of multiplying the corresponding points (at the two ends of the transform) in FT space by a small weighting factor, and emphasizing the frequencies (representing bandwidths) of interest by a larger factor. Then doing an inverse FT of the modified data gives you a signal with the desired apsects (bandwidths) empahsized. The only remanining problem then is to know the details of the weighting factors applied to the various points in FT space.

Intensity and noise weighting (2 and 3) are likely done directly to the signal. They are both probably some sort of weighted averaging. The general formula for that is:

WA = sum (WiSi) / sum (Wi)

where:

WA = the weighted average
Wi is the weighting factor applied to the ith wavelength
Si is the spectra value at the ith wavelength

The weighting is probably performed over each section of the spectrum at a time, replacing each wavelength point with the weighted average of the data in its neighborhood. Here again, the devil is in the details: how the weighting factors are selected. It seems likely, though, from the names of the filters used, that the weighting factors are simply proportional to the indicated quantity: the spectral intensity, or the S/N ratio at each wavelength.

Howard

\o/
/_\
Top of pagePrevious messageNext messageBottom of page Link to this message

Dennis Mooibroek (Dennis)
Posted on Tuesday, February 04, 2003 - 7:31 am:   

Howard,

Thanks for your reply. It has helped me a little bit further in my quest to understand these filters.

The reason of my interest in these algorithms is because I would like to transfer the calculations from the P-E software to Matlab. The main reason is the lack of different symbols and colors in the P-E software after using PCA.

On a personal note: I'm not a NIR guy, I'm merely someone with an interest in chemometrics and who does a lot of programming in Matlab and Excel.

If anybody could provide me with more details on these filters it would be more than helpfull.

Dennis
Top of pagePrevious messageNext messageBottom of page Link to this message

Richard Spragg
Posted on Monday, February 10, 2003 - 3:03 am:   

Dennis

If you want to know more about this please contact me at PerkinElmer.
Howard is mistaken in thinking that the company where Savitzky and Golay worked would feel any need to go the FT route to weighted derivatives. Also, our intensity and noise weightings do not use averaging.
I doubt that knowing the algorithms would help most users in deciding when to use them but you have a special need. My practice is to leave them ON permanently as they are very benign.

Richard

Add Your Message Here
Posting is currently disabled in this topic. Contact your discussion moderator for more information.