3. To sample condition or not?

I am sending this, combined with all the responses to date. I am also repeating the first and second sendings, because we have had some additional people joining and they should see all the discussion. I have marked the ends of each "session," so if you have read the previous ones, just skip to the appropriate line of dashes.

A question that has implications ranging from theoretical to applied: Is it better to condition the sample before taking a spectrum or to build a calibration to take into account changing environmental conditions? I know of at least one instrument supplier that advocates sample conditioning; more that one paper also advocates conditioning. Therefore, what is better? Sample conditioning or not? Better in this case covers all aspects, such as precision, accuracy, ease of use after calibration, difficulty in calibration (either to condition the sample, or the calculations, etc).

Another aspect to this question applies to the theoretical side. If, in extracting information from a set of data, one or more of the parameters is listed as known, but actually is not quite correct, the error associated with this assumption carries over into the calculation of the constituent "concentration." Two papers by Bruce Campbell have shown this for titrimetry and polarography. Thus, by fixing the conditions for obtaining spectra, such as temperature, one may find the results from a calibration have an undesired error component of unknown size (if very small, no trouble).

Comments?

----------------------------------------------------------

From: Howard Mark

Bruce -

Re the question of conditioning samples versus including the variations in the data: the way you put the question makes it very difficult to answer, since you are asking to compare numerical values (i.e., calibration and prediction results) with non-numerical considerations (ease of use, etc.)

As usual, the answer is going to vary in individual cases, and will depend not only on the actual improvements achievable by both means, but also on the user's PERCEPTION by how much the sample preparation has become more difficult, and whether the improvement in results is worth the extra effort..

I have addressed this topic a long time ago in a theoretical sense, and have published the results. First of all, you have to recognize that playing with data to remove the effects of errors in the data is never as good as eliminating the errors at the source. If you've minimized the errors to start with, you can still play with the data afterward. The question then becomes whether the results are "better" in the sense that the improvement is worth the extra work, since you then have to do it ever after, on all samples that you run in the future. This is not unheard of, in fact it is routine in most other areas of analytical chemistry, but it does fly in the face of one of the advantages of NIR analysis, which is the capability of fast, easy analysis through avoidance of the need for sample preparation.

Having gone this far, you then have to recognize that there are four possible cases, since you can decide independently to condition the samples during calibration and/or during prediction, and I have analyzed the effects; the results have been published in Anal. Chem., 58, p.1454 (1986). This is probably the most accessible publication of this work, but unfortunately a copy editor compressed a key part of the derivation to the point where it is almost incomprehensible, so I also included it in my book "Principles and Practice of Spectroscopic Calibration" Wiley, pages 41-42 and 47-53 (1991). In that case I considered the effect of repack variation on the total error and on the precision; that gave me two sets of numerical results that could be compared, as well as allowing a determination of how much the repack variation affects to total error. The "conditioning" in this study was the averaging together of multiple individual readings of each sample.

The results were as follows:

A) treating both sample and prediction data the same is better then treating them differently - either way (no surprise there) B) Not averaging (or, by analogy, conditioning) during calibration and averaging during prediction is better than averaging during calibration and not averaging during prediction.

Thus the four cases stack up as follows, best to worst:

1 - Conditioning during both calibration and prediction.

2 - Not conditioning during calibration but conditioning during prediction

3 - Not conditioning during both calibration and prediction

4 - Conditioning during calibration but not during prediction

Your second question lies at the heart of all calibration methodologies, and indicates conformance to one of the fundamental assumptions of regression and all least squares types of calibration modelling. As expressed by all statisticians (Draper and Smith's "Applied Regression Analysis", Wiley, is one of the best expositions I've seen), there is no (fundamental theoretical) problem in having error in the constituent concentration values (the statistical terminology for this is the Y, or independent, variable). Quite to the contrary - the near infrared and spectroscopic communities in general have approached the problem backwards (for understandable reasons, but it is still wrong). Regression theory tells us that it is OK for error to exist in the Y variable, but it is not OK for it to exist in the X variable (these are the spectroscopic readings). Fortunately the accuracy and precision of modern spectrometers is sufficiently good that this assumption is pretty well met (i.e., the errors are small enough to approximate zero): the total calibration error we see is ALWAYS the combination of the two contributions. This has always been the difficulty in comparing different calibration methodologies, etc: the differences due to different algorithms, models, etc, is a small variation superimposed on a (much larger) error due to the reference lab error; any real systematic differences in the effect of the algorithm is invariably swamped out by the larger random differences in the (chi-squared distributed) random errors of data at different wavelengths. This effect of this has not been appreciated; the relatively recent recognition that the reference lab error is this limiting error of any analytical method based on spectroscopy is only a small step in a proper understanding of the different roles that these different error sources play.

Howard

From: "W. F. McClure"

Sample condition, as our writer has stated, can really cause problems. The largest one I have found is, with agricultural sample, in lab calibrations do not work if your environmental conditions change, such as, going out in the field. Even if you go from the lab to on-line, tweaking of the calibration must be done to make it useful. However, all analytical methods have to be tweaked for best results. Bottom line - in my lab we always run "unknowns" in the same lab under the same conditions, prepared the same way the training set was prepared.

BTW, Tony Davies has shown that within the same lab there can be some very strange things happening. So, even with careful sample conditioning, errors still remain. My gut feeling is to "treat" the unknown samples the same way you treat the training samples.

Mac

--

W. Fred "Mac" McClure, Professor

North Carolina State University

PO Box 7625

Raleigh, NC 27695-7625

From: "Emil W. Ciurczak"

My personal preference would be to control the conditions and eliminate all variables. Unfortunately, life is seldom linear. The future of spectroscopic measurements (money-wise) is in a production setting. In a process stream, conditions are seldom controlled (as compared with a laboratory setting.)

While a calibration based on variable temperatures and flow rates may not be as pretty as one from the lab setting, it reflects reality. I have often pointed out to clients that there are two types of calibrations: one for publication and one for QC.

By that I mean that there are many published reports of calibrations based on a handful of samples, run in the lab under ideal conditions. If any attempt at running samples is made, they are often from the same pool of materials. This is good for a model (and is about the extent of the free feasibility study performed by instrument companies) or quick check of whether NIR is capable of performing a particular analysis. However, when push comes to shove (a real-life product on the line), a far more robust equation must be generated. This working equation will be based on far more samples, often at different temperatures and, if performed in a process stream, flow rates.

Unfortunately, because of proprietary reasons, these are the very equations you will NOT see at PittCon, EAS, or FACSS. I cannot fault a company (lawyer) for wishing to protect intellectual property, but the new scientist gets a skewed picture of what is needed to produce a working NIR analytical method. The literature is dominated by the feasibility-study type of work and is lacking the actual equations used in the plant.

To make an excruciatingly long comment short: I believe in using uncontrolled samples when I can't force a process into behaving, but prefer nice, controlled cuvettes whenever I can get away with them.

Emil W. Ciurczak

----------------------------------------------------------

Second round remarks on sample preconditioning.

From: Howard Mark

We're all saying the same thing, each in our own way. No surprise that treating samples the same during calibration and prediction gives best results. Also no surprise that removing extraneous error improves results. These two truths haven't changed from day 1, as far as I can remember, and I don't expect them to in the future. The only point that we all mentioned in round one that you don't normally find in the standard discussions, is the question of how feasible sample conditioning is, in any particular case, and this is what, in practice, will become controlling.

Howard

More comments from Bruce

There seems to be a number of points, each pulling its own way in n-dimensional space. Firstly, there is the repeated observation that preconditioning the sample leads to more robust calibrations (witness what I call "classical" analytical - one where all but the analyte is stripped away from the sample, such as in chromatography). This is in opposition to the growing practice of using other than linear, monovariate analyses. But then, the particular problem or calibration each of us faces calls for a robust calibration. Further, preconditioning the sample may mean extra work at the time of analysis, and also has equipment with extra parts, sometimes moving parts. This in itself decreases the robustness of the overall analysis.

Secondly, using sample preconditioning means less effort as there are fewer spectra to test when calibrating and one less variant, whether modeled or not. This savings in effort is not total, because one must should test the calibration over the range of the variant used for sample conditioning. This latter validation is often skipped; for example, temperature is usually held constant at 25 degrees for laboratory usage.

Thirdly, if the value of the variant supposedly being held constant should change for calibrations where sample preconditioning is used, the analysis can give serious, incorrect predictions.

` Fourthly, sample preconditioning means a need for maintenance. This may not be done with the regularity required, especially in a production setting where the plant has to decrease expenditures (I have seen more than one company cut maintenance at plants as one of the first places to "save" money).

Fifthly, if a sample preconditioning step is used, recalibration may become necessary if there is a change in the sample, such as in pH. A calibration that had encompassed a range of pH's would not need recalibration.

Sixthly, who will be the user? With NIR, usually, precision is very adequate, so ease of use is more important. In a place where many samples need to be analyzed, the easier and faster the analysis, the better. Sample preconditioning may slow down the throughput. Witness the move to not grinding grain but working with unground kernels (or at least that is what I see in literature).

Seventhly, and perhaps the most critical in many cases, is how much effort, money, and/or time will the new calibration save? If sample preconditioning saves more, than that would appear to be the route of choice. If not, no preconditioning is preferred.

It appears to me to come down to a question of what is required and the amount of effort that can be budgeted to develop a calibration, as well as the preference of the one doing the calibration.

It looks like I have been rather "long-winded" on this topic.

Finally, it would be interesting to have each of us reply as to whether or not sample preconditioning is a first choice or not. So please send just a short note and I will compile them before sending the results to everybody.

-------------------------------------------------------

From: Gallaher, Ken

I do not know if Applied Automation - the process analyzer company I work for as Manager Technology Development - is the vendor being referred to as preferring conditioned samples, but we certainly do.

To put this in context many of the applications we do involve very complex samples such as gasoline where to properly give results on perhaps three to six grades, over three seasonal specifications, each made up of perhaps 6 components each of which has variability in itself we may have to have hundreds of samples to really calibrate correctly.

As a result the last thing we need is any additional sources of variability. We control hydrocarbon sample temperatures to +/- 1 degree C or better, we filter the sample, we eliminate water vapor from the optical path, we do periodic automated referencing through the cell, produce all spectrometers so as to be spectroscopically identical forever - all to reduce variability. As a result we do NOT have to tweak models and we do not have to model in any of these forms of variation. In 5 plus years of working with AAI and Bomem FTNIR's I have never tweaked a model because of these types of noise - or analyzer changes. Whether users notice the advantage of this or realize it's importance I am not sure. But I am convinced that this approach has saved us and our customers large amounts of work. It is hard to tabulate service calls that never came.... We also meet lots of ex-NIR users frustrated with high maintenance.

As is commented vendors do not well document means of removal of these types of noise or results, but my wager is that they do not work very well. To begin with all of these variables do add noise to the data, at some point this limits the ultimate performance possible from the analyzer. Further, most of these kinds of noise are by no means random and they are also not small compared with what one is trying to measure.

I often hear in conferences the comment from vendors and others that process samples cannot be conditioned. I have come to learn that this really is an art - but we do it all the time and it is essential. I believe that many of the vendor assertions that sample systems are not needed is due to their inability to provide them as well as the cost and complexity of a fiber optic-based system if one needs a sample system at each sample point.

I am going to differ with Howard on the quality of current NIR analyzers. I would agree that for short periods of time that most NIR analyzers are very reproducible. Feasibility studies are often done under such conditions and thus give misleadingly optimistic results. However, are they reproducible after a source change? Are they after a detector change? are they after a grating motor change? These changes can cause biases on the order of the errors of the laboratory methods. This is where the need for bias corrections and complex calibration transfer schemes come from. Even FTNIR systems are not reproducible without proper design and quality control.

Ken Gallaher

From: Howard Mark

My third round comments:

1) I'd say you've summed up the situation pretty well, even if in what you call your "long-winded" way: you can get better results by putting more work into the calibration process (and that is true whether or not you do it by conditioning the sample!)

2) re Ken Gallaher's comments: I believe that when most vendors say "you can't condition the sample" they are referring to an on-line measurement, where the sample is in it's process pipe and "conditioning" the sample means disrupting the process (because the sample IS the process).

Re Ken's comments about changing sources, detectors, etc: yes and no. The terminology is relative: a bias correction is considered by most of the NIR community not to be a "change of the calibration" because of the history of the field: I guess not everybody remembers the old days, when in some cases changing a source would require a completely new calibration! We've advanced a long way from then and I for one am certainly pleased to hear that you can make these significant changes without the need for even a bias correction.

On the other hand, control of the instrument is one thing, but the question posed involved variability of the sample, which is a whole different can of worms. I agree with Ken as I stated above, that minimizing that variability will always improve the results, since you can still do data-related manipulations afterward. Don't forget that in some cases, the previous history of the sample variation affects the results, no matter how well the sample is controlled at the time of measurement, and this may require manipulations of the data to deal with anyway.

So it seems to boil down to two issues:

1) Whether "conditioning" is possible in the particular case (e.g., the on-line situation where it is probably not)

2) Whether the improvement attained is worth the effort needed - this one can only be decided on a case-by-case basis, by the people involved.

Howard

From David Hagen:

What full Uncertainty Analyses for NIR measurements have been done for process flow conditions versus laboratory conditions with multiple components such as Applied Automation's Ken Gallaher's comments on NIR in petroleum refining? ie including all bias as well as random errors. See NIST Technical Note 1297, 1994 Edition "Guidelines for Evaluating and Expressing the Uncertainty" http://physics.nist.gov/Pubs/guidelines/outline.html

ps I don't remember any discussion on uncertainty analysis with NIR. If you are interested in it, why don't you pose the question to the group?

David L. Hagen, PhD

AcrossTech

3107 Ulysses St NE

Minneapolis MN 55418-2244

Anyone want to comment on David's question?