T compensation: FAT analysis of fish ... Log Out | Topics | Search
Moderators | Register | Edit Profile

NIR Discussion Forum » Bruce Campbell's List » I need help » T compensation: FAT analysis of fish samples « Previous Next »

Author Message
Top of pagePrevious messageNext messageBottom of page Link to this message

Pierre Dardenne (dardenne)
Senior Member
Username: dardenne

Post Number: 50
Registered: 3-2002
Posted on Tuesday, October 05, 2010 - 7:29 am:   

Hi Jody,

I am quite surprised with your results.
May I give some comments;
- The gap between SEC and RMSECV is too wide indicating a too small data set.
- 12 factors for a problem which seems to be a binary mixture (water-fat as protein is quite constant) seems a high number. (Our meat data set (reflection 650 spl; 0.3-35% SECV 0.5 with 4 factors). With your quite narrow range one would expect +/- 0.2% as RMSECV with an error of the reference method around 0.1%
- The RMSECV is going down until factor 19 showing perhaps an inappropriate setup of the CV (duplicates?). A CV error is increasing generally earlier.

Best regards,

Pierre
Top of pagePrevious messageNext messageBottom of page Link to this message

Jody Hoefs (hoefs)
New member
Username: hoefs

Post Number: 2
Registered: 9-2010
Posted on Tuesday, October 05, 2010 - 6:47 am:   

This is the report for the high range, we are using Sensologic software in combination of the polytec diode analyser both are German companies.
Perhaps something you can use:

Polytec PSS-M Polychromator details:
- Transmission design for maximum
sensitivity and extremely low scattered light
(contrast ratio > 1:50,000) - Lower dark current, also in NIR range, due
to new Hamamatsu detector technology - Models covering 5 standard spectral ranges
between 550 nm and 2,500 nm - Pre-aligned and supplied with wavelength
calibration data (�� < 0.5 nm) - Electronics can be directly attached - With integrated shutter for dark signal
measurement


Report High FFA range:

Name : 08july2010
Comment :
Filename : C:\MROPLO~1\EOCOPY.CPF
CWS version : 2.10

-------------------------------------------------------------------------------
SERIES:

Name : FFA for probe (1) extended
Comment :
Created : 05/06/2009
Modified : 09/02/2010
No of spectra : 1200
No of datapoints : 501
Wavelength range : 1100..2100 nm, 501, 2 nm steps
No of properties : 1
Properties : ffa

-------------------------------------------------------------------------------
CALIBRATION:

Name : FFA (high) PLSR P1 extended
Comment :
Created : 05/06/2009
Calibration method : PLSR
Calibration Set : 80 Spectra
889-944 961 964 969 976 987 990 993-994
1024 1027 1032 1039 1050 1053 1056-1057
1087 1090 1095 1102 1113 1116 1119-1120

Selected property : ffa
Property range : 1.775 to 6.395
Selected
wavelength range : 1114..1600 nm, 244, 2 nm steps
Transformation : SNV(abs)
- Absorbance
- SNV
Int. Calibration number : 001B001F

-------------------------------------------------------------------------------
REGRESSION RESULTS PLSR

CROSS VALIDATION

Selected CVS : 5
Number of factors : 12 (Selected Maximum : 20)

Number of factors RMSECV
----------------- --------
0 1.400543
1 1.236108
2 1.106175
3 0.774568
4 0.681779
5 0.667040
6 0.594106
7 0.534346
8 0.459576
9 0.361227
10 0.323735
11 0.306105
12 0.281469*
13 0.267258
14 0.252092
15 0.250368
16 0.242642
17 0.256191
18 0.252295
19 0.232173
20 0.238797

-----------------------------------------------------------------------------------

OUTLIER DETECTION:
Number Predicted Actual Difference 'T' 'H' 'D' 'S'
921 3.91604 3.38100 0.53504 3.28406* 1.28078 1.09024 0.861864
944 5.76820 6.39500 -0.62680 -3.81258* 1.19167 1.34260 0.353435
969 1.99545 2.03000 -0.03455 -0.20436 0.90700 0.00278 6.642235*

-----------------------------------------------------------------------------------
Multiple correlation coefficient : 0.992705
Standard error of estimate : 0.183082
Root mean square error of cross validation : 0.281469
Index of systematic variation : -14.127
Index of random variation : 25.5268



Signature : ____________________
<end>
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 120
Registered: 3-2004
Posted on Monday, October 04, 2010 - 8:58 pm:   

Dear Mark ;
Thanks for your suggestion .However the manufacturer of DAS is OEM company , they could not help in this .We NIRs lovers have work for it.
I have observed that Dark value unaltered .
The reference changes might be due deposit of materials ?
I have observed a micron grew was made to fix window materils , there the oil residual deposits .
I might case the problem.
I am working on it. Your valubale suggestion , I will take note of it.
Top of pagePrevious messageNext messageBottom of page Link to this message

Howard Mark (hlmark)
Senior Member
Username: hlmark

Post Number: 353
Registered: 9-2001
Posted on Monday, October 04, 2010 - 12:16 pm:   

Venky - I recommend that you contact the manufacturer of your instrument for recommendations on how to troubleshoot the varying reference signal problem

\o/
/_\
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 119
Registered: 3-2004
Posted on Monday, October 04, 2010 - 10:35 am:   

Dear Jody Hoefs
Thanks for the message .
My range is 1% to 4 % at one end .
We are getting good result at High FFA.
The main problem here is Reference changes .
I am using Flow cell with saphire window.
I have changed the fiber 1000 micron senstivity increased but Ref,problem persist.
I have to use it for on-line measurment .
If I am correct you have used MEMs based NIRS .
Is it
Can you send more details on detrend and SNV.
I have worked on SNV.But it is not yiled expected change over .
Top of pagePrevious messageNext messageBottom of page Link to this message

Jody Hoefs (hoefs)
New member
Username: hoefs

Post Number: 1
Registered: 9-2010
Posted on Monday, October 04, 2010 - 3:39 am:   

Hi,

For our customer we are using the Polychromatic Diode Array and have seen that we can reach 0.03% in SECV range up to 0.15%FFA and 0.19% range 0.15-2.831 and a high range as well. So three ranges to cover the whole area, I have used SNV detrend and spectral ranges 1114-1600+1850-2080nm. Second derivative had the problem that when you change a lamp or measuring over time that the results were not repeatable. With FTNIR we don't see this problem, only with diode array.
We are using a Hellma probe pathlength 5mm as well and unfortunately I just got the message that the tip has been broken off. Probably something in the stream hit the probe tip.

Hope this helps.
Top of pagePrevious messageNext messageBottom of page Link to this message

Howard Mark (hlmark)
Senior Member
Username: hlmark

Post Number: 352
Registered: 9-2001
Posted on Wednesday, September 29, 2010 - 9:55 am:   

Venky - if random variability is affecting your results, which seems likely from the fact that the dark readings are changing, then there's no data transform or algorithm that will "magically" fix it.

First the problem has to be solved, THEN you can start to consider ways to optimize your calibration.

\o/
/_\
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 117
Registered: 3-2004
Posted on Wednesday, September 29, 2010 - 6:49 am:   

Dear Mark & David ;
Thanks for your sugesstion.
You have not answered can go with PLS with ratio method that is taking part of the spectra dividing it with processed spectral data.
Here the flow is uniform , there in no chance for exteternal elements to play .
I have read papers , all them used MLR with selective wavelength. It is good for experimental but pratical it is difficult.
Top of pagePrevious messageNext messageBottom of page Link to this message

Howard Mark (hlmark)
Senior Member
Username: hlmark

Post Number: 351
Registered: 9-2001
Posted on Wednesday, September 29, 2010 - 4:53 am:   

Venky - I agree with Dave about the spectroscopy, but I would go even one step further back. The fact that the dark reading changes when you're measuring dynamically means that something about the measurement conditions is affecting the instrument.

Presumably the static measurements were made in a laboratory, where environmental conditions (temperature, humidity, electrical noise, vibration, etc.) were well-cotrolled, while the dynamic conditions are out in a plant where the environment is not well-controlled.

Since a "dark" reading by definition does not involve any light, your problem seems to be even more basic than the spectroscopy, and I don't foresee you being able to get any stable results until that is solved.

After the dark readings are stable, then you want to consider factors that would affect the light readings, which is where the spectroscopy comes in.

Another condition you would want to look at, once you're assured of a stable instrument, is whether the sample is homogeneous. The points David made are good ones, but I would also recommend that you verify whether the composition of the sample material you're measuring is constant and uniform.

\o/
/_\
Top of pagePrevious messageNext messageBottom of page Link to this message

David W. Hopkins (dhopkins)
Senior Member
Username: dhopkins

Post Number: 164
Registered: 10-2002
Posted on Wednesday, September 29, 2010 - 3:23 am:   

Hi Venky,

I think it is possible you need to look at what is happening in the cell in the dynamic mode, when the sample is moving through the cell. It sounds like you may have bubbles or schlieren lines due to the uneven flow and mixing in the cell, which would mess up your measurements. You need to solve this problem before proceeding, because I think you should be able to use the static models on the dynamic situation.

I would certainly recommend that you stick with the PLS (or PCR) models, so you can use the full spectral models. Then you have the possibility to use a method to qualify that the spectra are good before giving a result. From what you say, you would be smart to test that the spectra/baseline are high quality, to avoid giving erroneous result.

There is no magic treatment that can substitute for good spectroscopy.

Best regards,
Dave
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 116
Registered: 3-2004
Posted on Wednesday, September 29, 2010 - 1:01 am:   

Dear All

I am working on on-line measurment of FFA value (low value 2% to 10 %) for sunflower oil.
Iam using DAS of range 1000 nm to 2200 nm with 512 elements .
I have done pre-porcessing like (1) Baseline, (2) Smoothing (3 ) Dervative for the captured spactral data .
I have used flow cell with path length 5mm in transmission mode.
I got good linearity and calibaration in static condition.
But in the dynamic , the Dark and reference are subject to change and mislead the calibration .
(a) I need help whether PLS to be applied or MLR?
(b) In the case of MLR which oil band I have to use( oliec group samples )
(c) If I use PLS , can I use one varaible as ref and ratio metric approach is good in case of on-line measurment?
(d) Any simple approach to measure low FFA in on-line .
Top of pagePrevious messageNext messageBottom of page Link to this message

Malcolm Ray Brown (bro609)
Junior Member
Username: bro609

Post Number: 7
Registered: 7-2007
Posted on Thursday, June 17, 2010 - 6:07 pm:   

Thanks Tony. I'll try taking out the water bands as well. It's good to hear some positive feedback on the T stabilisation approach. I met with Christel Solberg at the NIR2009 conference in Nov last year, and we had a good discussion about our respective work on NIRS and scanning salmon. We didn't specifically discuss T effects ( I wasn't aware of this problem then) - but I'm sure I could follow up with her if necessary.
Malcolm
Top of pagePrevious messageNext messageBottom of page Link to this message

Tony Davies (td)
Moderator
Username: td

Post Number: 233
Registered: 1-2001
Posted on Thursday, June 17, 2010 - 5:54 am:   

Hello Malcolm,

The lady that Bruce was thinking of is Prof. Christel Solberg of Bod� University Norway.
I have tried to e-mail her but no reply; she may be on holiday preparing to enjoy the midnight sun!

Anway you seem to have found some good advise, I have used the temperature stabilisation technique successfully.

The other thing you can try, assuming you are using PLS, is to take out the water bands. The calibration might not be so good but it should remove the temperature problem.
Best wishes.

Tony
Top of pagePrevious messageNext messageBottom of page Link to this message

Malcolm Ray Brown (bro609)
Junior Member
Username: bro609

Post Number: 6
Registered: 7-2007
Posted on Thursday, June 17, 2010 - 1:58 am:   

Thanks Dave, Karl, Bruce and Venkatarman for you collective responses.

Bruce � I recently visited the NIRS group at NOFIMA, Norway led by Jen Petter Wold. Their work on salmon suggested that fat calibration can produce systematically different results as a function of T (eg. may be linked to shifting of water peak with lower T) � though they did not specifically correct or account for this in their modelling (maybe they had good T control).
Dave � I have not specifically plotted moisture against fat (have not measured moisture), but there is a general well-recognised inverse relationship between fat and moisture in fish (protein is relatively constant). So, as you suggest, this may be the main factor contributing to the T sensitivity of my model. I have gone through the exercise of repeat-measure predictions of fat on my fillet over a 2h period as the T warms from 0 to 9C � and yes, I see a straight (sloping) line, eg. with predicted fat changing from 11% wet weight to 10%. I can�t be 100% sure that this is not due to metabolic breakdown, but at that T and timescale I suspect not.

The other thing that has been suggested to me by a colleague, is to add into the calibration a �temperature stabilisation set� eg. scan up to a dozen individual samples at several temperatures (upper and lower extremes of scan temperatures and midpoint) � and put this into the calibration. My understanding is that, within the revised calibration the changes in spectral response from T are then interpreted as noise. This approach is also documented in the paper by Segtnan, V.H. et al. (2005). Low-cost approaches to robust temperature compensation in near-infrared calibration and predictions situation. Appplied Spectroscopy, Vol 59 (6), pp. 816-825. So, I�ll also give this a try to see it improved the robustness of the model.
I further take your point Dave of the many other factors that may be associated to model robustness, such as sample presentation and analytical error.

Thanks again. Malcolm
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 104
Registered: 3-2004
Posted on Monday, June 14, 2010 - 11:29 pm:   

Dave ;
You are partilly corrcect .Temp would not effect FAT only certain extend. But when it cross the limit it goes wrong .
I do agree moisture plays vital role in calibration of other produts .
We can provided compenstion for it that was suggested by indirectly .
Top of pagePrevious messageNext messageBottom of page Link to this message

David W. Hopkins (dhopkins)
Senior Member
Username: dhopkins

Post Number: 153
Registered: 10-2002
Posted on Monday, June 14, 2010 - 10:48 pm:   

Hi Malcolm,

My experience with meat measurements showed that fat measurements were not sensitive to temperature, while moisture measurements can be very sensitive. I think that your fat calibration may be sensitive to temperature if there is a correlation between moisture content and fat content in your training set. Have you plotted fat, protein and moisture against each other pairwise, to check for correlations? Or possibly, the scattering in the fish fillets is changing with temperature.

If you plot the predicted fat contents for individual samples at various temperatures, do you see a straight line? I expect a straight line for moisture calibrations, from my experience. If the plot is more random, there may be light scatter problems. In either case, I suspect you need to increase the number of samples in your training set.

What is the level of fat in fish fillets? Does it degrade metabolically, so that it may be critical to make your lab reference measurements on the same day as you make the NIR measurements? Does your lab obtain good reproducible results if you submit blind duplicates? There are many reasons that a fat calibration might not be robust, and you need to examine all possibilities.

Best wishes,
Dave
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 103
Registered: 3-2004
Posted on Monday, June 14, 2010 - 8:52 pm:   

To my knowledge I have not seen up to 0 to 45deg.c the FFA not affecting the spectral pattern .
Top of pagePrevious messageNext messageBottom of page Link to this message

Bruce H. Campbell (campclan)
Moderator
Username: campclan

Post Number: 122
Registered: 4-2001
Posted on Monday, June 14, 2010 - 8:42 pm:   

Malcolm,
The first thing to do is contact the Norway department that uses NIR to study fish. They may have encountered your problem. I don't remember the name of the department but they are located in northern Norway.
Bruce
Top of pagePrevious messageNext messageBottom of page Link to this message

Karl Norris (knnirs)
Senior Member
Username: knnirs

Post Number: 34
Registered: 8-2009
Posted on Monday, June 14, 2010 - 7:57 pm:   

Malcolm,
I am not aware of fat spectra being sensitive to temperature, but the water band changes dramatically for small temperature changes around 0 degrees.
See S.R. Delwiche, R.E. Pitt, and K.H. Norris. Cereal Chem. 69(1):107-109. 1992.
Karl
Top of pagePrevious messageNext messageBottom of page Link to this message

Malcolm Ray Brown (bro609)
New member
Username: bro609

Post Number: 5
Registered: 7-2007
Posted on Monday, June 14, 2010 - 6:49 pm:   

Hi,

I�ve developing models to predict fat in fish fillets, but recently noticed significant day-to-day variation in prediction values. Prior to testing, the fish had been chilled in saltwater/ice overnight. After removing each fish and quickly excising the fillet and scanning, the T of the fillet may be in the range -1 to 4 C. Though this is a relatively narrow range, a separate lab experiment has shown that predicted fat changes significantly over this range (eg. from 14% at -1C, to 11% at 4C in the same sample).

I have done a bit of reading on "T compensation" options within calibration models - still a little confused, but wondering if anybody has advice:
eg.
1) should I be simply trying to build a "global calibration" model, incorporating many samples spread across the range of temperature of scanned
2) measure T of fillets prior to scanning (eg using an infrared thermometer) and make a correction based on the lab experiments I have done on T vs predicted FAT. This assumes all fillets have the same T/predicted FAT relationship.
3) accept that there will be some noise in the data related to T - but minimise as much as possible T sample fluctuations (there are some logistical constraints here associated with sample volume/ throughput).

I appreciate it may be difficult to give a definitive answer, especially without a better knowledge of the process, but any thoughts are appreciated.
Thanks, Malcolm

Add Your Message Here
Posting is currently disabled in this topic. Contact your discussion moderator for more information.