New training examples with sample dat... Log Out | Topics | Search
Moderators | Register | Edit Profile

NIR Discussion Forum » Bruce Campbell's List » Chemometrics » New training examples with sample data to learn chemometrics « Previous Next »

Author Message
Top of pagePrevious messageNext messageBottom of page Link to this message

Dintis Thomas (dintis)
New member
Username: dintis

Post Number: 3
Registered: 11-2007
Posted on Saturday, January 09, 2010 - 2:39 am:   

Hi Venky,
Mailed you one abstract I collected in the past from web.
Regards
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 91
Registered: 3-2004
Posted on Saturday, January 09, 2010 - 1:01 am:   

Dear All;
Can any body mail the pdf of the paper
" Partial Least Square regression : A tutorial"
By paulGeladi and Bruce R.Kowalski Published in the Journal Analytica Chimica Acta ,185 (1986) Page -1-17"
or source for C++ code of PLS

Please .
[email protected]
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 90
Registered: 3-2004
Posted on Saturday, December 19, 2009 - 11:06 pm:   

Deepak;
Multiple scan helps to see the repeatability and reproducibility only.It would not add more sample data. More the samples for the training set better the model .
Top of pagePrevious messageNext messageBottom of page Link to this message

Bruce H. Campbell (campclan)
Moderator
Username: campclan

Post Number: 120
Registered: 4-2001
Posted on Saturday, December 19, 2009 - 2:57 pm:   

Deepak, I would be very careful with the timing of scanning your samples especially since you are interested in the moisture content. You may want to scan each sample before and then after the reference method testing and keep the samples in an environment that would not lead to loss or gain of water from the atmosphere.

Further, you would be taking a very large risk of an inadequate calibration with only ten samples and looking for three components. The probability is very large that one or more of the components doesn't have a full range of values and also are not fairly uniformly spread throughout the range. I would think that the number of samples for a preliminary calibration should be about 40 (although others may suggest a different number).
Bruce
Top of pagePrevious messageNext messageBottom of page Link to this message

Howard Mark (hlmark)
Senior Member
Username: hlmark

Post Number: 302
Registered: 9-2001
Posted on Friday, December 18, 2009 - 10:38 am:   

Deepak - every little bit helps, but multiple scans doesn't substitute for more samples. Ten samples is very far on the skimpy side. The only time so few could be used is for clear liquids when only one or two components is changing.

\o/
/_\
Top of pagePrevious messageNext messageBottom of page Link to this message

Deepak Sharma (deepak)
New member
Username: deepak

Post Number: 5
Registered: 11-2007
Posted on Friday, December 18, 2009 - 9:38 am:   

I just want to add one thing, if we scan the sample twice or thrice so we can capture maximum variation in the sample and also need less batches to build a model.
thanks/cheers
Top of pagePrevious messageNext messageBottom of page Link to this message

venkatarman (venkynir)
Senior Member
Username: venkynir

Post Number: 89
Registered: 3-2004
Posted on Thursday, December 17, 2009 - 9:54 am:   

Dear wDeaaleed Makarem
Can you please down the data of NIR spectra for 10 batches of rifampicin powder with there Conc. , Particle size , Moisture content.
Since we are working same area and we back to you with result and model , with explanation to you soon.
I prefer more batches samples so that we can use for on-line tuning model.
You can mail the data to [email protected]
I have worked moisture and particle size (range based ) the results are encouraging one .
Top of pagePrevious messageNext messageBottom of page Link to this message

Waleed Makarem (pharmad)
Junior Member
Username: pharmad

Post Number: 6
Registered: 11-2009
Posted on Thursday, December 17, 2009 - 6:50 am:   

Dear All ,

Thank you so much for your kind support , I really appreciate this so much .

I will download these sample data . But what is next. are these data represent a problem that I should solve using chemometrics ? .



Also , I have NIR spectra for 10 batches of rifampicin powder with there Conc. , Particle size , Moisture content . How can I build a model to predict these data from unknown sample IR spectra.


Thanks all for your kind support .
Top of pagePrevious messageNext messageBottom of page Link to this message

Waleed Makarem (pharmad)
New member
Username: pharmad

Post Number: 5
Registered: 11-2009
Posted on Thursday, December 17, 2009 - 5:52 am:   

Dear All ,

Thank you so much for your kind support , I really appreciate this so much .

I will download these sample data . But what is next. are these data represent a problem that I should solve using chemometrics ? .



Also , I have NIR spectra for 10 batches of rifampicin powder with there Conc. , Particle size , Moisture content . How can I build a model to predict these data from unknown sample IR spectra.


Thanks all for your kind support .
Top of pagePrevious messageNext messageBottom of page Link to this message

Tony Davies (td)
Moderator
Username: td

Post Number: 219
Registered: 1-2001
Posted on Wednesday, December 09, 2009 - 9:33 am:   

Hello Waleed, Howard,

Yes, you were correct Howard, 2002 is the pharmaceutical data.

Waleed, I suggest you have a go at tackling this data and then we can tell you where to find some published work using this data.

Your will find that there is data for the same samples on two instruments. I suggest you start by just working on the data from the first instrument.

Good luck! and best wishes,

Tony
Top of pagePrevious messageNext messageBottom of page Link to this message

Howard Mark (hlmark)
Senior Member
Username: hlmark

Post Number: 301
Registered: 9-2001
Posted on Wednesday, December 09, 2009 - 9:08 am:   

Waleed - there are several sets of data available on the WWW. One group of them consists of data used at the biannual IDRC, in what's called a "Software Shootout", which is a contest for attendees of the conference to all analyze the same data set. Each conference's data comes from a different source and at least one of them is pharma data. I think that the 2002 data is pharma data, but I wouldn't swear to it. For most years the same data is available in different formats, so you may need to download only the one you want. You can find the sets at the following links:

1992
http://www.idrc-chambersburg.org/shootoutfiles/Ss1992.zip

1994
http://www.idrc-chambersburg.org/shootoutfiles/Ss1994.zip

1996
http://www.idrc-chambersburg.org/shootoutfiles/Ss1996.zip

1998
http://www.idrc-chambersburg.org/shootoutfiles/Ss1998.zip

2000
http://www.idrc-chambersburg.org/shootoutfiles/Ss2000.zip

2002
http://www.idrc-chambersburg.org/shootoutfiles/Shootout2002.zip

2004
http://www.idrc-chambersburg.org/shootoutfiles/SPC.ZIP
http://www.idrc-chambersburg.org/shootoutfiles/Nsas.zip
http://www.idrc-chambersburg.org/shootoutfiles/Jcamp.zip
http://www.idrc-chambersburg.org/shootoutfiles/xfile.zip
http://www.idrc-chambersburg.org/shootoutfiles/ANALYTES.TXT

2006
http://www.idrc-chambersburg.org/shootoutfiles/2006/matlab.zip
http://www.idrc-chambersburg.org/shootoutfiles/2006/unscrmbl.zip
http://www.idrc-chambersburg.org/shootoutfiles/2006/nsas.zip
http://www.idrc-chambersburg.org/shootoutfiles/2006/jcamp.zip
http://www.idrc-chambersburg.org/shootoutfiles/2006/spectrometer-signal-to-noise-ratio.xls
http://www.idrc-chambersburg.org/shootoutfiles/2006/Parameter%20Description.xls
http://www.idrc-chambersburg.org/shootoutfiles/2006/shootout2006validation.xls

2008
http://www.idrc-chambersburg.org/shootout2008_data/Unscramb2008.zip
http://www.idrc-chambersburg.org/shootout2008_data/matlab2008.zip
http://www.idrc-chambersburg.org/shootout2008_data/nsas2008.zip
http://www.idrc-chambersburg.org/shootout2008_data/jcamp2008.zip
http://www.idrc-chambersburg.org/shootout2008_data/ExcelFred2008.xls
http://www.idrc-chambersburg.org/shootout2008_data/ExcelFred99.xls

Howard

\o/
/_\
Top of pagePrevious messageNext messageBottom of page Link to this message

Waleed Makarem (pharmad)
New member
Username: pharmad

Post Number: 4
Registered: 11-2009
Posted on Monday, November 23, 2009 - 8:42 am:   

Dear iyas ,

I need sample data as training . eg , if members have these data and did their testing to such data. This i will go step by step to understand using chemometrics .

Thanks for your support
Top of pagePrevious messageNext messageBottom of page Link to this message

iyas (iyas)
Senior Member
Username: iyas

Post Number: 32
Registered: 7-2007
Posted on Monday, November 23, 2009 - 1:28 am:   

let us start with the homogeneity test
choose one formulation with the concentration more than 1 %w/w and the source of the raw material , is it changing ?
other thing do you have a validated analytical method for the product

i think your device a an ft - nir Michelson mechanism ???
and the formulation of the product will give help us in the design of the calibration and validation (do use the unscramble to make design of the calibration design )

regards
Top of pagePrevious messageNext messageBottom of page Link to this message

Waleed Makarem (pharmad)
New member
Username: pharmad

Post Number: 3
Registered: 11-2009
Posted on Sunday, November 22, 2009 - 2:56 pm:   

Dear Iyas ,

Thanks so much for your instant support , I really appreciat this so much

Regarding yoru inquiries , the Instruemnt is Perkin |Elmer Spectrum One NTs with AssureID software . I do not use assureID because it is non proffessional software , so , i use Unscrambler v.9.7

I am working in a pharmaceutical company . so , we produce so many products. , I plan to use in both finished products and IPC samples .. eg granules. but I think that finished products analysis will be more difficult to start learning with . so , i think that start analyis of IPC samples eg granules will be much easier .

Thanks again for your support.
Top of pagePrevious messageNext messageBottom of page Link to this message

iyas (iyas)
Senior Member
Username: iyas

Post Number: 31
Registered: 7-2007
Posted on Sunday, November 22, 2009 - 2:34 pm:   

dear waleed can i know the device you are using
and what are the main products which you are intending to measure
is it in process control IPC , or finished products
and what are the products which you like to measure and their concentrations
regards
Top of pagePrevious messageNext messageBottom of page Link to this message

Waleed Makarem (pharmad)
New member
Username: pharmad

Post Number: 1
Registered: 11-2009
Posted on Sunday, November 22, 2009 - 1:53 pm:   

Dear All ,

I work in pharmaceutical company in quality control , I am so interested of Chemometircs and its application for NIR, manufacturing process optimization and Quality By Design QBD . I am using Unscrambler . I want a training examples with samplesets (not pure chemometrics books) to learn the applications of Chemometrics .
Can I can your support , please

Thank you in advance for your support

Add Your Message Here
Posting is currently disabled in this topic. Contact your discussion moderator for more information.