Food Quality and Preference 13 (2002) 341–353

www.elsevier.com/locate/foodqual

A comparison of sensory methods in quality control

E. Costell*
Instituto de Agroquı´mica y Tecnologı´a de Alimentos (CSIC) PO Box 73, 46100 Burjassot, Valencia, Spain

Accepted 8 February 2002

Abstract
Many different types of sensorial methods have been proposed and used to evaluate and control the sensory quality of foods.
However, not all of them are suitable for incorporation in to quality control programmes. To simplify comparison a distinction is
proposed between methods that can be used to define sensory specifications or to select a product quality standard and those that
can be used to check if a product complies with stated requirements. With this approach, the appropriateness and limitations of
different methods and their practical applicability, according to their use with or without a previously selected or developed stan-
dard (product, mental or written), are discussed. # 2002 Elsevier Science Ltd. All rights reserved.
Keywords: Sensory quality; Sensory analysis; Quality standards; Quality specifications

1. Introduction by market requirements and degree of commercial

competition for the particular food product.
The term ‘‘quality’’ has been used so much and in so Notwithstanding the lack of conceptual definition of
many contexts that its meaning is frequently unclear. A quality, food quality control and assurance is evidently
number of definitions have been proposed, always with a top subject both in industry and in public and private
reference to the situation or problem to be solved in each control institutions (Gould & Gould, 1988; Herschdor-
case. They vary widely between simple expressions such as fer, 1984; Juran, 1974; Kramer & Twigg, 1970; Stauffer,
‘‘Fitness for use’’ (Juran, 1974) to more detailed ones like 1988) and continues to be a matter of discussion in both
that proposed by Molnar (1995): ‘‘The quality of food academic and industrial forums. Some of the most rele-
products, in conformity with consumers’ requirements vant items, advances and problems related to the defini-
and acceptance, is determined by their sensory attri- tion and measurement of quality of foods have been
butes, chemical composition, physical properties, level discussed in a Special Issue of Food Quality and Preference
of microbiological and toxicological contaminants, published in 1995.
shelf-life, packaging and labelling’’. Any of these or Basically the utilisation of any type of method in food
many other definitions could be useful in a certain con- quality control follows a common approach: first, defi-
text but none of them is always satisfactory. Quoting nition of specifications or quality standards and second,
Fisken (1990), ‘‘quality is a fuzzy and relative term and development and testing of methods to evaluate, in a
it is in a constant motion’’. Due to this lack of con- reliable manner, whether or not a product complies with
ceptual definition, any specification, method or group of the requirements of the quality standards. Two ques-
methods designed to control the quality of a certain tions arise with this approach: (1) Which food char-
product may be applicable in a particular situation but acteristics or properties should be included in the
they are subject to a constant evolution. The changes standard? (2) Which methods are to be used for their
come, on the one hand in function of methodological analysis or evaluation? The answer to the first question
advances in each area (chemical analysis, microbiology, is usually conditioned by the need to establish a com-
toxicology, etc.) and on the other, of changes undergone promise between two extreme alternatives: either con-
sider a large number of the food samples or a long list of
food properties, leading to a rather complete specifica-
* Tel.: +34-96-3900022; fax: +34-96-3636301. tion but difficult to apply in practice, or else, select only
E-mail address: [email protected] (E. Costell). those characteristics of higher incidence on quality that
0950-3293/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved.
PII: S0950-3293(02)00020-4
342 E. Costell / Food Quality and Preference 13 (2002) 341–353

make it possible to decide if the food fulfils the require- reference to a standard, one should consider not only
ments of a certain quality grade in a simpler manner. their direct effect on acceptability but also to what
On answering the second question a similar situation extent such a variability may affect consumer confidence
holds: not always the most precise and costly methods in the product. It is evident that a constantly changing
are most suitable but, in general, the selection is based product is certain to affect consumer confidence. For
on the capacity of the method to measure variations in example, in a recent study (Costell, unpublished data),
each of the characteristics that influence product quality to determine the influence of some sensory attributes on
with sufficient precision. the acceptance of commercial chocolate milk beverages,
The implementation of food quality control and no relationship was found between perceptible colour
assurance systems, in the areas of chemical composition, differences and consumer acceptance. Products of
microbiological and toxicological safety, and nutritional clearly different colours were equally acceptable. How-
characteristics, brings up problems related to the selec- ever, it is evident that manufacturers should define their
tion of properties or characteristics to be measured and colour standard and control variability between lots to
to the methods to be used. These problems are much avoid the negative effect that perceptible differences will
more numerous when the system is designed to control have on the confidence of regular consumers.
what is known as ‘‘sensory quality’’. Sensory quality is The recentness and slow development of the discipline
even more difficult to define because it is linked not only of sensory analysis is perhaps the cause of the lack of
to food properties or characteristics but to the result of immediate responses to the real need for methods to
an interaction between the food and the consumer. measure and control sensory quality both in industry
Besides, sensory evaluation is a rather recent discipline, and in control organizations. Consequently, many
as compared with others such as chemical or micro- methods or systems, with variable scientific base, have
biological analysis. It was born and slowly developed its been proposed for sensory quality control. Some of
methodology during the second half of the twentieth them are being used at the present time simply by inertia
century (Costell & Durán, 1981; Costell, 2000; Lar- or habit. Every quality control technician or group has
mond, 1994; Moskowitz, 1993). As a consequence, not tried to solve problems by themselves in the best possi-
all methods developed and used by different research ble way, as, although some books on this matter are
teams at different times can today be considered ade- available (ASTM, 1996; Pangborn & Roessler, 1965;
quate to evaluate and control the sensory aspect of Lawless & Heymann, 1998; Moskowitz, 1983; Stone &
quality. Sidel, 1993), it was not until 1992 that Muñoz, Civille &
The concept of sensory quality has changed with time Carr published the book Sensory Evaluation in Quality
since it was defined by Kramer in 1959 as ‘‘the compo- Control, the first one exclusively dedicated to this specific
site of those characteristics that differentiate among topic.
individual units of a product and have significance in For these reasons, when analysing the different alter-
determining the degree of acceptability of that unit by natives proposed and used by control organizations and
the user’’. Some authors centre their attention on the by different industries, the first impression is that there
first part of this definition. For them sensory quality is is a great diversity of approaches, requirements, levels
product oriented. Others emphasise the second part and of strictness and practical applicability and still today it
consider that sensory quality is consumer oriented. In is generally considered that the correct application of
the first case, quality is considered as a convention sensory methods requires a lot of time to carry out and
developed by experts and it may therefore be considered to analyse data and that the number of qualified asses-
as constant over a limited period only (Molnar, 1995). sors is not always available. To simplify comparison
With the second approach, quality is mainly related to between methods, the distinction between the sensory
consumer acceptance and is context dependent (Car- methodology to be used in the development of stan-
dello, 1995). The product oriented approach may, in dards and specifications from that to be used to check if
some cases, render results of doubtful practical validity a product complies with their requirements is impor-
since it is assumed that the opinion of a group of experts tant. On one hand, when establishing standards and
is representative of the reaction of the potential con- specifications, collecting data sets from different tests
sumers of the product in question. But the second and relating them allows for powerful modelling of the
approach is not totally satisfactory either because if a relationships between physical process and ingredient
specification or standard has to be established in order variables and the perceived attributes from descriptive
to define the sensory quality of a certain food product, it profiling, and ultimately, consumer appeal, present no
is not sufficient to collect acceptability data that merely problems (Muñoz & Chambers, 1993). In this case it is
give statistically significant results (Booth, 1995). In neither necessary nor convenient to use fast methods or
relation to the latter point it should be considered that, to take quick decisions. On the other hand, in practical
according to Stone and Sidel (1993), when fixing accep- quality control, fast methods using a few assessors are
table deviations of the magnitude of an attribute with needed in order to take quick decisions at a given
 E. Costell / Food Quality and Preference 13 (2002) 341–353 343

moment. For these reasons, the author proposes to dif- also be established. An interesting point to consider
ferentiate between methods that can be used to define when a real product is used as a quality standard is the
specifications or to select a product quality standard and establishment of a clear methodology to substitute it
those destined to decide whether a particular food item when necessary. This need arises when the standard
fulfils the requirements of the appropriate standard. product is running out of stock or when the end of its
shelf-life is getting near. The new standard must be sen-
sorially identical to the previous one. This similarity
2. Preliminary steps should be ascertained by means of sensory discriminatory
or difference tests, such as the triangle test. An impor-
2.1. Selection or establishment of sensory quality tant consideration here is that the objective of the sen-
standards or specifications sory test is not to detect differences between samples but
rather to establish that they are sensorially equivalent.
2.1.1. Sensory quality standards In this case the analyst must determine what constitutes
The establishment or definition of the quality stan- a meaningful difference by selecting the proportion of
dards is the critical point in the implementation of a distinguishers and then select a small value of b-risk to
quality control programme. In practice, each company ensure that there is only a small chance of missing that
or institution must define the quality level to be con- difference if it really exists (Ennis, 1993; Meilgaard,
trolled in a certain product and then develop a standard Civille & Carr, 1999; Schlich, 1993).
that fits their objectives. When dealing with foods and
with sensory quality it is difficult, and often practically 2.1.1.2. Mental standard. One of the most controversial
impossible, to obtain a product or a series of products strategies in sensory quality control is to assign a quality
showing the same unaltered sensory characteristics dur- level to a product with reference to a mental standard
ing enough time to permit their use as reference items in developed by one or a group of experts or panellists.
subsequent comparisons. Fortunately, for some attri- Criticisms have been based mainly on two aspects. The
butes, such as colour or appearance, quality standards lack of concordance between experts as to the mental
(photographs or reproductions of the food in materials standard applicable to a certain product and the fallacy
like plastics or ceramic) have been used successfully of assuming that the opinion of the experts represents
when the product itself cannot be used, generally for that of the consumers. Mental standards should only be
reasons of sensory variation or alterations. In the criticized if each expert or panellist operates under dif-
majority of cases even this is not possible. This problem ferent criteria or mental standards, and if panellists do
has traditionally been solved in two ways: either relying not evaluate products uniformly. Therefore, when a
on the mental standard created by one or several experts, mental standard is to be used, panellists need to be
or developing a written standard, in which a description trained on the criteria and product attributes that are to
of the main attributes is commonly included. form the mental standard. This training provides valid-
ity and reliability to the use of mental standards and
2.1.1.1. Product standard. As indicated above, the use thus contributes to a sound evaluation method. In
of the same product as a standard in the evaluation of addition, criteria and/or products that form the mental
the quality of raw or processed foods is almost always standard should periodically be reviewed to strengthen
difficult or impossible. However it is more frequently the principles learned and the reliability of this practice.
used in quality control of ingredients or of some raw Muñoz et al. (1992) describe the procedures to form and
materials. According to Muñoz et al. (1992) a control teach sound mental standards to panellists for some of
standard selected for quality purposes is referred to as a the QC/sensory methods that use these standards (e.g.
product that is used as a representation of certain char- ‘‘in/out’’ method)
acteristics (not necessarily the ‘‘optimal’’) and a product Consumer opinion is affected by the context in which
that can easily be obtained, maintained or reproduced. the food is examined and by the expectations that some
The criteria for choosing a product as a control stan- external factors, such as brand or price, will exert (Car-
dard can be arbitrary or deliberate. In any case, before dello, 1995; Lawless, 1995). In principle, it is understood
its selection, information must be obtained on the pro- that experts are those individuals who posses a highly
duct variability and on its incidence on the sensory developed ability to recognize and evaluate sensory
quality of the final food item. This implies the identifi- properties and detailed technical information about
cation and quantification of the sensory attributes of the their companies’ products. For the most part, these
studied ingredient or raw material by using sensory individuals have been successful and their activities have
descriptive techniques (profiles, QDA, Spectrum) and constituted one of the earliest organised efforts in sensory
the determination of those attributes that influence the evaluation (Stone & Sidel, 1993).
final product quality assessment by consumers. Accep- One of the main problems posed by the use of experts
table variation limits for each of the attributes should in the evaluation of product sensory quality is that the
344 E. Costell / Food Quality and Preference 13 (2002) 341–353

qualification of ‘‘expert’’ is not well defined. As a result, considered a specialist in the product and/or the process
sometimes some people are considered ‘‘experts’’ when in and/or marketing but they will also have a contrasted
fact they are not and their personal opinion on a product physiological sensitivity and a wide knowledge of sen-
is erroneously identified as a valid mental standard for sory evaluation techniques. Having these characteristics,
assessing the quality of same. The period of education and the experts are expected to use more homogeneous
training is not clearly specified and the criteria used to quality criteria and to communicate their qualifications
select experts may vary from case to case and from time to in more precise terms. However these advantages do not
time. Consequently the concept of individual quality, solve the other problem, that of the lack of concordance
defined by the mental standard, usually varies from one between the experts opinion and that of consumers.
expert to another. Recently Guinard, Yip, Cubero, and However, in practice, the significance of this lack of
Mazzucchelli (1999) have confirmed this fact by analys- concordance depends on the type of product and the level
ing the quality scores of different types of beer given by of quality that is to be assessed. To rely on a mental stan-
several ‘‘experts’’ and observing that they clearly dif- dard to decide the quality level (acceptable or not, good
fered in their concept of quality. Perhaps one of the or better) of a widely consumed food product continues
reasons for these results lies in the lack of coincidence in to be risky. Yet, when it is a matter of differentiating
the training process of what the authors call ‘‘brewing between good quality and exceptional quality in certain
experts to various degrees’’. As with other terms used in products (wine, coffee, etc.), the assessment of quality by
sensory analysis, the fight against ambiguity should start real experts, in accordance with their mental standards, is
by clearly defining what ‘‘expert’’ means and establish- still considered to be a valid alternative.
ing the minimum requirements in education and train-
ing that a person must possess to be qualified as such. It 2.1.1.3. Written standard. The elaboration of written
is recommendable to first consult the ISO Standard sensory standards to be used as references when deter-
(1994) on this subject. This Standard defines what a mining a product quality must include definitions for
selected assessor, an expert assessor and a specialised both critical attributes, the perceptible variations of
expert assessor are (Table 1) and describes criteria for which depend on the raw materials and on the manu-
choosing people with particular sensory skills. This facturing process and the attributes that drive con-
Standard offers principles and procedures for expanding sumer acceptance. The type of standard will depend
their knowledge and abilities to the levels required for on the quality level to be controlled, on the objective
expert and for specialized expert assessors. The main of the control and on the type of product. The quality
contribution of this Standard is that the experts, as level is important: it is not the same to develop a
defined in it, will not only have the experience to be standard designed to distinguish between acceptable

Table 1
Definition and characteristics of selected assesors, expert assessors and specialized expert assessorsa

Type of assessor Definition Characteristics

Selected assessor Assessor chosen for his/her

ability to perform a sensory test

Expert assessor Selected assessor with a high Good consistency of judgements,

degree of sensory sensitivity and both within a session and from one
experience of sensory methodology, session to another
who is able to make consistent and Good long-term sensory memory
repeatable sensory assesssments of
various products

Specialized expert Expert assessor who has additional Extensive experience in the relevant
assessor experience as a specialist in the specialist field
product and/or the process and/or Highly developed ability to recognize
marketing, and who is able to and evaluate sensory properties
perform sensory analysis of the Mental retention of reference standards
product and to evaluate or predict Recognition of key attributes
effects of variations relating to raw Deductive skills which may be applied to
materials, recipes, processing, problem solving
storage, ageing, etc. Good ability to describe and communicate
conclusions or to take appropriate action
a
Based on ISO 8586–2 (1994).
 E. Costell / Food Quality and Preference 13 (2002) 341–353 345

and unacceptable products as it is to do so in order to of this product. The Quality Index Method (QIM) is
differentiate between two acceptable products (which based on objective evaluation of the key sensory attri-
one is of higher quality) or even to set a standard butes of each fish species using a points scoring system
applicable to differentiate between high quality products (from 0 to 3). The lower the total score, the fresher the
and those of optimal or exceptional quality. Evidently, fish. QIM procedures for 12 fish species have now been
the difficulties increase as the quality level increases developed. As an example, a QIM scheme for cod is
because the number of critical attributes grows and their shown in Fig. 1. It is expected that the QIM will become
selection becomes more complex as quality levels are the leading reference method for the quality assessment
higher. Frequently it is difficult to locate and describe of fresh fish within the European Community (www.qi-
the difference between a good quality product and a m.eurofish.com). Another option is that chosen by the
better quality one mainly because of the small differ- International Olive Oil Council (COI) to define the
ences found (Powers, 1981). This problem has not yet quality standard for virgin olive oil. This organization
been satisfactorily solved but it is undoubtedly of the has recently proposed a revised method for the orga-
highest interest. According to Cardello (1997), ‘‘estab- noleptic assessment of virgin olive oil (COI, 1996) with
lishing the relationship between sensory responses and the purpose of determining the criteria needed to assess
the pleasure associated with food is one of the most the flavour characteristics of this product and develop-
important and practical contributions that sensory ing the methodology for its classification according to
science can make to the study of food’’ the intensity of the perceptible defects. In this case, a
In the development of a quality standard the specific profile sheet was defined including negative (defects)
objective of the quality control programme should be and positive attributes and unstructured continuous
taken into account. It is not the same when the objective is scales for measuring their intensity were incorporated
to design a sensory quality control within either a public (Fig. 2).
or private control organisation or to control the quality of When the objective is to control the sensory quality of
products to be included in a specific Designation of Origin products of a Designation of Origin (EEC Council,
or to control the quality of an industrial food product to 1992), the approach is different. According to Bertozzi
compete with other products in the market. (1995), the denomination of a product marked with the
In the first case the aim of the programme is to ensure geographic name of the zone in which it is produced
that no inadequate products reach the consumer. Here includes information on the manufacturing process and
the term ‘‘quality’’ is equivalent to ‘‘absence of defects’’. on product characteristics. In this context, it is necessary
Regulatory quality should reflect the minimum accep- to furnish objective methods to certify the typicity of
table quality and form a base from which individual every production in such a way that it can be differ-
companies can develop their standards. The appropriate entiated in comparison with imitations. For these reasons,
standard must then include a description of the most in this case, apart from considering the possible pre-
common defects in the product, comprising those sence of common or sporadic defects in the product, the
defects due to inadequate characteristics of the raw standard must include not only the attributes defining
materials used or to the process conditions, those its sensory profile and those affecting acceptability but
resulting from incorrect or prolonged storage or even also the attributes which can establish differences with
those derived sporadically from accidental causes. The other similar products from other designations of origin.
description of the standards used by the Inspection The latter additional attributes may not be necessary
Branch of the Canadian Department of Fisheries and because frequently the differences between designations
Oceans to determine the sensory quality of fish, as lie in differences in intensity of the same attributes
reported by York (1995), constitutes a good example of rather than in different attributes. The development of
criteria to be applied in the development of this type of this type of standard involves a lot of time consuming
standard. In this case, the regulatory definition of quality work including the collection and initial screening of a
should ensure food safety and should be a reflection of great number of different samples, representative of the
consumer expectation of minimum acceptable quality. variability of the products belonging to the designation
This regulatory sensory quality is reduced to three spe- and also the generation and selection of the attribute
cific measurable characteristics: taint, decomposition descriptors. From the results of the required descriptive
and unwholesomeness. As stated by York, ‘‘Consumers analysis a sensory profile is finally defined which serves
have other concepts of quality such as product form, as the specific standard for the designation. An example
species and processing conditions (e.g. fresh vs. frozen) of this type of standard is the Guide to the sensory eva-
which are outside the mandate of regulatory quality’’. A luation of the texture of hard and semi-hard ewes’ milk
different approach has been used by various European cheeses (EUR, 1999). This guide includes the attributes
Fisheries Research Institutes to develop an accurate and to be evaluated, their physical and sensory definitions,
objective method for the determination of fish freshness their evaluation techniques and seven point intensity
considering that freshness is a critical quality parameter scales, of which three points are fixed by a standard
346 E. Costell / Food Quality and Preference 13 (2002) 341–353

Fig. 1. Quality Index Method (QIM) scheme for cod.

reference product. Definitions, evaluation technique and 2.1.2. Sensory specifications

scale for friability evaluation can be seen in Fig. 3. Broadly speaking, a sensory specification is designed
Finally, the development of a quality standard for to determine the acceptable or tolerable variation in a
commercial food products can follow a scheme almost product with reference to a previously selected product
similar to the above mentioned type of standard. The dif- or an established written standard. In the latter case the
ference lies in the fact that in this case defining the quality specification defines the range of intensities accepted or
standard requires the consideration of several points such tolerated for each of the attributes or the range of
as marketing objectives, production variability, attributes defects included in the written standard. Specifications
that vary, attributes that drive consumer acceptance, can be set based on management’s criteria alone and/or
manufacturing conditions and available resources. on consumer response. The second option provides
 E. Costell / Food Quality and Preference 13 (2002) 341–353 347

Fig. 2. Profile sheet for the organoleptic assessment and classification of virgin olive oil.

more realistic specifications because here the influence 4. Analysis of the relationship between the varia-
of the variability of the product or of each considered bility of the attributes or the product and the
attribute on consumer acceptance is taken into account. variability in consumer acceptability.
A product test for establishing a sensory specification
includes:
The main information thus obtained will show for
1. Selection of a group of samples showing different which attributes, their variability influences consumer
sensory properties and representing the actual acceptance. It must be accepted that variability in the
variability in the marketplace. In some cases it is intensity of some attributes may not affect acceptability.
convenient to add samples showing an especially Furthermore the extent of the variability in an attribute
important defect. is not necessarily related to the magnitude of its effect
2. Evaluation of the perceptible difference/s on acceptability. With this information and the particular
between each sample and the standard either by criteria used by the institution or company a definite sen-
direct comparison or by means of descriptive sory specification can be established. This specification
analyses in which the magnitude of the defects includes not only those attributes affecting acceptability
and/or attributes are evaluated. but also all those proposed by the responsible organisa-
3. Evaluation of the acceptability of samples by a tion according to its particular understanding of quality
large consumer panel. for the product studied.
348 E. Costell / Food Quality and Preference 13 (2002) 341–353

Fig. 3. Definitions, evaluation technique, scale and standard reference products for friability evaluation of hard and semi-hard ewes’ milk cheeses.

In any case, the development of standards and speci- ference or affective tests are not appropriate for the
fications is neither an easy nor a quick task. On many routine evaluation of products quality. Difference tests
occasions the results obtained in the first study are not are too sensitive to relatively small differences and do
satisfactory and the initially proposed standard or spe- not determine the extent of the difference and the
cification must be modified. On the other hand, varia- determination of the preferences of a small group of
tions produced in the market because of changes in assessors does not represent the consumer population.
consumer preferences or habits, degree of exigency, new In practice, the selection of the method to be used will
trends, or even changes produced in the market struc- depend on the objectives set and the characteristics of
ture when new products are introduced, must be fol- the products to be evaluated. For example, as Muñoz et
lowed. The validity of standards or specifications may al. (1992) comment, if the variable attributes are limited
vary with time and must then be periodically updated to to five to ten key attributes, the comprehensive descrip-
adapt them to market variations. tive method is feasible but when the product variability
is not easily defined by specific sensory attributes, but
2.2. Selection of methods can be more readily reflected in the broad sensory
parameters (appearance, flavor, texture) the quality rat-
Following the described procedure, the application of ing method is a likely method of choice. In those cases,
sensory methods to the development of standards and when variation cannot be specifically defined by sensory
the establishment of sensory quality specifications pre- attributes or when examples of unacceptable product
sent no special problems. The objectives, the experi- cover a multitude of sensory conditions, the in/out
mental designs, the testing conditions, the number of method is recommended.
assessors and their level of training, the criteria for the To sum up, in each particular case, the choice of sensor-
selection of consumer panels and the statistical analysis ial method should be made taking the following criteria
of data are well defined in many recent texts (ASTM, into account:
1996, 1997; MacFie & Thomson, 1994; Moskowitz,
1994; Lawless & Heymann, 1998; Meilgaard et al.,
1999). The problem arises when, once the quality stan- 1. The objective of the quality control programme
dard has been established and the specification of a 2. The type of standard previously established
product defined, it is necessary to use sensorial methods 3. Whether or not the perceptible variability of a pro-
in order to decide if the product meets the requirements duct can be defined by specific sensory attributes
set or not. In principle, the most suitable sensorial and, if so, the number of parameters or sensorial
methods are those which make it possible to measure attributes necessary to do so.
the magnitude of variability between a product and a 4. The magnitude of perceptible variability that
previously defined standard (intensity scales, quality must be detected
rating or difference from control method) while the dif- 5. The level or levels of quality to be assessed.
 E. Costell / Food Quality and Preference 13 (2002) 341–353 349

Applying these criteria it is possible to select the the final decision be taken by the person responsible for
most suitable sensorial method to obtain the sensorial quality control, according to the scores given by the
information needed in a timely and cost-effective way. judges. The judges should centre their attention on the
magnitude of the perceptible differences. The responsi-
bility to decide may have a psychological influence on
3. Sensory methods in quality control the evaluation of the differences. Another source of
influence may be the knowledge of the specification by
As has been commented above, various publications the judges. To compensate for the latter it is common
have proposed different types of sensorial methods practice to introduce a blind sample of the standard
which can be used in sensory quality control of food product to be compared with the declared standard.
products. According to Muñoz et al. (1992), they can be This a useful method in public or government organi-
classified into eight groups: Overall difference tests; sations, where the objective is to separate samples of
Difference from control; Attribute or descriptive tests; low quality. In industrial quality control, this method
In/out of specifications; Preference and other consumers lacks the capability of giving information on the nature
tests; Typical measurements; Qualitative description of of the difference, necessary to identify the cause and
typical production and Quality grading. The problems correct the difference (Aust, Gacula, Bearm, &
associated with using some of these methods have Washam, 1985; Muñoz et al., 1992).
already been mentioned, such as the difference or con- A more informative method consists in selecting the
sumer tests but other test types also present serious most important attributes in the product sensory quality
limitations in their approach as is the case, for example, and evaluating the differences from the standard for each
with methods which have the objective of classifying a attribute (Fig. 4b). A difference higher than the specified
product as typical or atypical giving no information as tolerance in any attribute will mean that the product is
to the reason for the classification given. But apart from out of specification. With this information, corrections
the particular limitations of each of the methods, in can be introduced when necessary.
many cases, the lack of validity of results may not be Still, this method detects the magnitude of the differ-
attributable to the method itself but to a defective rea- ences in attributes but not their direction. A possible
lisation of the test and/or to an incorrect analysis of the alternative, successfully used in some cases (Costell,
information obtained. The same sensorial method used unpublished data) is to design a scale similar to the
with a correctly established and well defined standard or ‘‘just-right scale’’, in which the central point corre-
applied directly, relying on the individual criteria sponds to the standard product (Fig. 4c). With this scale
regarding product quality held by a small group of information is obtained not only on the magnitude but
company employees or a group of selected and trained also on the direction of the differences from the stan-
assessors may render results of different validity. dard in the different attributes. This procedure may be
of interest when the objective is to evaluate the effect of
3.1. Methods involving a comparison to a standard a change in the formulation of a certain product on its
sensory quality and the direction of the possible change
The objective of these methods is to evaluate the dif- in any quality attribute is not predictable. Besides the
ferences between the product and the corresponding type of scale used, the quality of the obtained informa-
standard. This involves the clear definition of terms tion depends on the degree of training of the judges and
used and of the experimental test conditions, the design their knowledge of the product, on the realisation con-
of a score card, the selection and training of a panel and ditions and on the correct analysis of data as a function
the selection of the method to be used to analyse data of the type of scale (ordinal or interval scale) used. A
obtained. large trained panel (30–40) is appropriate when only the
degree of difference from the control is to be evaluated.
3.1.1. Difference from a standard or control product When additional information about differences on spe-
There are several ways to examine the differences from cific attributes is required a smaller and more highly
a standard product. The simplest one is to evaluate the trained panel is recommended.
overall degree of difference using a single scale (rating,
category or unstructured) with the extremes labelled 3.1.2. Difference from a mental standard
‘‘no difference’’ and ‘‘extreme difference’’ (Fig. 4a). It is As commented above, the use of a mental standard by
an easy and fast method, useful when the analysed pro- one or several experts to define the quality of a food
duct does not have complex sensory characteristics. Its product presents two serious problems, derived from the
objective is to distinguish between the samples showing a possible difference between the mental standards used by
tolerable difference from the standard and those for which the experts and from the fact that their opinions are not
the difference is greater than the tolerance established in representative of consumers’ opinion. In principle, based
the corresponding specification. It is recommended that on these considerations, it would not be recommendable
350 E. Costell / Food Quality and Preference 13 (2002) 341–353

Fig. 4. Different types of scales for: (a) overall difference from standard product ratings; (b) difference from control for selected attributes and (c)
directional diference from control for selected attributes.

to use this method to evaluate the quality of certain market price. These differences can hardly be detected
products. But this position must be reconsidered. In by naive consumers.
cases when an expert or a group of experts, showing
recognised ability to evaluate the magnitudes of the 3.1.2.1. In/Out method. This is the simplest method to
perceptible differences between products and a pro- compare product quality with a mental standard by
found knowledge of the product and its manufacturing experts. It is mainly used to identify products that show
process is available there are situations in which their clear deviations (presence of off-notes or other defects)
performance is not only admissible but recommendable. from the normal production. It can be recommended
One of these situations is when the characteristics of the for the evaluation of raw materials or relatively simple
product will not be directly evaluated by the consumer, finished products. Its advantages are the simplicity and
such as, when dealing with raw materials or ingredients the direct use of results obtained. The main disadvantage
or when only previous information is sought on the is its inability to provide descriptive information and
effects of formulation, process or storage conditions on therefore its lack of direction and actionability to fix
the product quality. Another situation where quality problems (Muñoz et al, 1992). The validity of the
evaluation by experts is appropriate is when the objec- information provided depends on whether the ‘‘experts’’
tive is to evaluate differences between quality grades of are indeed genuine experts. If this method is used by one
products of exceptional sensorial characteristics, such as or a small group of people in a company who do not
wine, coffee or olive oil, in which small differences possess the necessary expertise, each of them makes
between high quality levels may be decisive in their decisions based on his individual experience and on his
 E. Costell / Food Quality and Preference 13 (2002) 341–353 351

product knowledge. This situation leads to highly variable information gathered during the previous development
and subjective information. of the standard and the establishment of the specifica-
tion is collected in the scorecard, according to different
3.1.2.2. Overall quality rating method. This method criteria and in different ways. Several alternative proce-
consists in assessing the quality of a product according dures have been used but, practically, only one of them
to an established quality criteria. Samples are rated is in use at present: the quality grading test.
using a single quality scale anchored ‘‘very poor’’ and
‘‘excellent’’. A product is rejected when the quality rat- 3.1.3.1. Quality grading method. This method has been
ings are low. Initially it was considered that a quality one of the most popular sensory tests used in quality
rating test represents a combination of affective and control and consists in developing a scorecard that
descriptive tasks (Sidel, Stone, & Bloomquist, 1981). includes a scoring system with points assigned for each
Besides this duality, other problems arise in data treat- grade and a description of sensory characteristics defin-
ment because the quality scores are not clearly based on ing quality for each grade. The scorecard is composed
psychophysical measurements (Lawless, 1994). Another of ordinal scales using discrete numbers and contains
approach consists in considering the product quality as the description of the characteristics. The scale ampli-
an integrated impression like acceptability or pleasure tude may be 3, 6 or 9 points. The upper third of the
experienced when consuming a food or drink. Using scale includes a detailed description of the intensity of
this criterion, the evaluation of a product quality with a each attribute corresponding to a high level of quality,
unidimensional scale may appear logical. This implies the medium third the description corresponding to an
accepting that quality and acceptability are not concepts acceptable quality and the lower third that correspond-
of an exclusively sensorial character (Costell, 2001). As ing to rejectable quality. Frequently a scale is designed
stated by Cardello (1997): ‘‘In psychological terms, for each basic sensory attribute, e.g. appearance, colour,
pleasure and displeasure, liking and disliking, are not flavour and texture. The judges give scores to each
sensory phenomena, although they accompany most attribute and when a product is assigned a score in the
sensory stimuli. Rather, pleasure and displeasure are lower third of the scale, it should be rejected (ISO,
emotional experiences. They are conscious cognitions 1987). This test allows for a rapid qualification of the
that accompany the somatic effects of emotions’’. Based product and for the detection of the possible causes of
on this approach it can be accepted that a group of rejection. However this test requires a group of very well
experts, sharing a common mental standard, may suc- trained judges that can correctly interpret the descrip-
cessfully judge a product quality grade. From a quality tions corresponding to each of the quality grades for
control standpoint, this method has the disadvantage of each of the selected attributes. An important problem
producing an integrated judgement, that may not be here is that the judges are obliged to carry out an ana-
actionable and useful for product documentation or lytical job simultaneous to a qualification which may
guidance. produce deviations in the results, as commented above.
In an effort to overcome the above mentioned prob- Finally it should be taken into account that the data
lem, other methods have been proposed, in which a from this test are of ordinal nature and this fact leads
scale to evaluate overall quality and other scales to to the use of non-parametric statistical methods for
evaluate the attributes’ quality or their intensities are analysing the data obtained.
included in the same scorecard (Muñoz et al., 1992).
This scheme is apparently similar to that described 3.2. Methods of evaluation without a standard
above to evaluate the perceptible differences from a
control product but the situation is not the same. Even 3.2.1. Descriptive method
assuming that the experts possess a solid mental stan- This method consists in having a well-trained sensory
dard of the product quality it is hard to make them pay panel that provides data on a set of the product’s sen-
attention to the overall quality and to the quality or sory attributes. During the initial development of the
intensity of attributes in the same session. In this way sensory standard a number of attributes are selected.
they are obliged to perform functions that require dif- Some of them have been selected because their varia-
ferent mental attitudes which can produce erroneous tions affect product acceptability by consumers and
results. It should not be recommended that experts use some others may be introduced by the industrial com-
this type of scorecard to evaluate a product quality. pany on the basis of their interest in connection with the
identity and/or desired image of the product in the
3.1.3. Difference from a written standard market. In the establishment of the corresponding spe-
These types of tests are among the most frequently used cification their tolerable variability is fixed. Evaluation
in quality control. Essentially they consist in the evalua- of quality with this descriptive method consists basically
tion of the intensity of different attributes and/or defects in the evaluation of the intensity of each attribute by a
or the evaluation of quality grade using a scorecard. The trained panel using descriptive profiling (conventional
352 E. Costell / Food Quality and Preference 13 (2002) 341–353

profile, QDA, Spectrum). The person responsible for the same doubtful value as their opinion on product
quality control then studies the results obtained from acceptance.
the statistical analysis of experimental data and makes Finally something must be said about the quality
the final decision based on the sensory specification evaluation methods based on what is known as ‘‘com-
previously established. In this case, the specifications are plete scorecard’’. These scorecards include evaluations
represented by the range of intensities tolerated for each for different sensory categories such as appearance, fla-
attribute. Products whose intensity on any given attri- vour, texture, etc. as well as for some specific attributes
bute fall outside specifications are considered unac- like sourness or viscosity and a variable number of
ceptable. For example, a company wants to assess the ‘‘quality points’’ is assigned to each one of them. The
quality of the virgin olive oil it produces, in accordance sum of points obtained determines the product quality.
with the standard proposed by the COI (Fig. 2). The Another, alternative, method consists in assigning
first step should be the selection and training of the scores to the intensity of different attributes, multiplying
panel. The training comprises the definition, evaluation them by different factors according to their importance
procedures and magnitude scoring of each of the attri- and adding them up to get a product qualification. One
butes and of the defects included in the scorecard (COI, of the better known methods of this type is the U.C.
1996). Once it has been ensured that the panel works Davis 20-point wine scoring system described in 1981 by
well, the comparison between the panel results and the Amerine and Roessler (Lawless, 1995). These methods
specifications set for each of the product attributes can were once very popular and were adopted by some
be used to make decisions regarding the product qual- industrial firms and control organisations. They would
ity. As stated by Muñoz et al, (1992), the two main appear to make it possible to express the quality of a
advantages of this approach are the absence of any product with a single number. But in practice they pre-
subjectivity in the evaluation and the quality of the data sent several problems. They have been criticised on
obtained. The main disadvantages are the time and cost many occasions because the weight of each attribute has
necessary to train and calibrate the panel and the time been arbitrarily assigned and the product quality is
necessary to perform the test and to analyse the data. taken as the sum of the qualifications given to a limited
This test and the corresponding data analysis can be number of attributes. On the other hand, the scales used
simplified by using the software available (Punter, to evaluate the intensity of the different attributes do
1994). The method described is not suitable for solving not always have sensorially equivalent magnitudes. This
some particular problems that require an immediate also means that the validity of the information obtained
decision. In this case one possibility may be to perform when each score is multiplied according to a previously
a reduced version of the profile. Once the panel is established weighting factor is questionable, even when
trained on the whole profile (10 to 15 attributes), a small this factor has not been established in an arbitrary
group of judges may be selected to evaluate the most manner. For all of these reasons this type of test is not
important attributes (4–5). This simplification may considered recommendable for assessing the quality of a
allow its use in daily quality control. product.

3.2.2. Other methods

Many of the merited criticisms of sensory methods 4. Concluding remarks
used in quality control originate from the lack of a pre-
viously developed standard or an established specifica- In accordance with what has been stated above, we
tion. It should also be considered that the standards and can conclude that not all methods proposed for evalu-
specifications are developed for a specific situation ating the sensorial quality of food products are suitable
(industry, public or private organisation, etc). The use for incorporation in quality control programmes. Dif-
of some methods (In/Out, Quality Rating, etc.) without ference or preference tests, typical measurement or
a previous standard or specification afford results of those methods based on ‘‘complete scorecards’’ are the
doubtful validity. It is especially important to note that less appropriate while difference from control methods
the development of a quality grading system without a and descriptive methods, are the most sound sensory
previous study of the relations between the variations in tests for quality control purposes. Others methods such
attributes and product acceptance may lead to the con- as In/Out, Quality rating and Quality grading methods
struction of scorecards without any practical value. It is may be used in particular situations. The characteristics
also important to point out that the use of these meth- of each product, the degree or level of quality that it is
ods in food research to compare products or to study wished to control and the resources available condition
the effects of processing conditions, must be avoided. the choice of method to be used. On the other hand, it
The evaluation of the sensory quality of any product should be borne in mind that designing an effective
by a group of 10 to 20 more or less trained panellists in programme for testing the sensorial quality of a product
a laboratory without a standard or specification has is based on the following points: (a) The selection of the
 E. Costell / Food Quality and Preference 13 (2002) 341–353 353

sensory quality standard; (b) The establishment of the Quality ratings by experts, and relation with descriptive analysis
sensory specification; (c) The selection of a method to ratings: a case study with beer. Food Quality and Preference, 10, 59–
67.
evaluate differences between the product and the corre-
Herschdoerfer, S. M. (1986). Quality control in the food industry. New
sponding standard and; (d) The selection, training and York: Academic Press.
maintenance of the panel. The practical value of the ISO. (1987). Sensory analysis. Methodology. Evaluation of food pro-
information obtained will be determined by the correct ducts by methods using scales. International Standard no. 4121.
fulfilment of these requirements. Switzerland: International Organization for Standardization.
ISO. (1994). Sensory analysis. General guidance for selection, training
and monitoring of assessors. Part 2: Experts. International Standard
no. 8586–2.. Switzerland: International Organization for Standardi-
Acknowledgements zation.
Juran, J. M. (1974). Quality Control Handbook (3rd ed.). New York:
To Ministerio de Ciencia y Tecnologı́a of Spain (Pro- McGraw Book Co.
ject AGL 2000–1590). The author acknowledges Dr. Kramer, A. (1959). Glossary of some terms used in the sensory (panel)
evaluation of foods and beverages. Food Technology, 13, 733–738.
Luis Durán for revision of the manuscript and helpful Kramer, A., & Twigg, B. A. (1970). Quality control for the food indus-
observations and Alejandra Muñoz for constructive try (3rd ed.). Westport. Connecticut: The AVI Publishing Co.
comments. Lawless, H. T. (1994). Getting results you can trust from sensory eva-
luation. Cereal Foods World, 39, 809–814.
Lawless, H. T. (1995). Dimensions of sensory quality: a critique. Food
Quality and Preference, 6, 191–199.
References Lawless, H. T., & Heymann, H. (1998). Sensory evaluation of food.
New York: Chapman & Hall. International Thomson Publishing.
ASTM. (1996). Sensory testing methods: Second Edition. MNL 26. Lardmond, E. (1994). Is Sensory Evaluation a Science? Cereal Foods
Philadelphia: American Society for Testing and Materials. World, 39, 804–808.
ASTM. (1997). Relating consumer, descriptive and laboratory data. MacFie, H. J. H., & Thomson, D. M. H. (1994). Measurement of food
MNL 30. Philadelphia: American Society for Testing and Materials. preferences. London: Blackie Academic & Professional.
Aust, L. B., Gacula, M. C., Beard, S. A., & Washam II, R. W. (1985). Meilgaard, M., Civille, G. V., & Carr, B. T. (1999). Sensory evaluation
Degree of difference test method in sensory evaluation of hetero- techniques (3rd ed.). Boca Raton. Florida: CRC Press.
geneous product types. Journal Food Science, 50, 511–513. Molnar, P. J. (1995). A model for overall description of food quality.
Bertozzi, L. (1995). Designation of origin: quality and specification. Food Quality and Preference, 6, 185–190.
Food Quality and Preference, 6, 143–147. Moskowitz, H. R. (1983). Product testing and sensory evaluation of
Booth, D. A. (1995). The cognitive basis of quality. Food Quality and foods. Westport, Connecticut: Food & Nutrition Press.
Preference, 6, 201–205. Moskowitz, H. R. (1993). Sensory analysis procedures and viewpoints:
Cardello, A. V. (1995). Food Quality: conceptual and sensory aspects. Intellectual history, current debates, future outlooks. Journal of
Food Quality and Preference, 6, 163–168. Sensory Studies, 8, 241–256.
Cardello, A. V. (1997). Pleasure from food: its nature and Role in Moskowitz, H. R. (1994). Food concepts and products. Just-in time
Sensory Science. Cereal Foods World, 42, 550–552. development. Trumbull, Connecticut: Food & Nutrition Press, Inc.
COI (1996). Organoleptic assessment of virgin olive oil. COI/T.20/ Muñoz, A. M., & Chambres IV, E. (1993). Relating sensory measure-
Doc.no 15/Rev.1. International Olive Oil Council. ments to consumer acceptance of meat products. Food Technology,
Costell, E., & Durán, L. (1981). El análisis sensorial en el control de 47, 128–131, 134.
calidad de los alimentos. Revista de Agroquı´mica y Tecnologı´a de Muñoz, A. M., Civille, G. V., & Carr, B. T. (1992). Sensory evaluation
Alimentos, 21, 1–10. in quality control. New York: Van Nostrand Reinhold.
Costell, E. (2000). Análisis sensorial: evolución, situación actual y Powers, J. J. (1981). Multivariate Procedures in Sensory Research:
perspectivas. Industria y Alimentos Internacional, 2, 34–39. Scope and limitations. MBAA Technical Quarterly, 18, 11–21.
Costell, E. (2001). La aceptabilidad de los alimentos. Nutrición y pla- Punter, P. H. (1994). Software for data collection and processing. In
cer. Arbor, 661, 65–85. J. R. Piggott, & A. Paterson (Eds.), Understanding natural flavors
EEC Council (1992). Council Regulation 2081/92. 14 July 1992. Offi- (pp. 97–111). London: Blakie Academic &Professional.
cial Journal of the European Community. Luxemburg. Schlich, P. (1993). Risk tables for discrimination tests. Food Quality
Ennis, D. M. (1993). The power of sensory discrimination methods. and Preference, 4, 141–151.
Journal Sensory Studies, 8, 353–370. Stauffer, J. E. (1988). Quality Assurance of Foods. Westport, Con-
EUR (1999). A guide to the sensory evaluation of the texture of hard necticut: Food & Nutrition Press, Inc.
and semi-hard ewes’ milk cheeses. No 18829. Official Publications of Sidel, J. L., Stone, H., & Bloomquist, J. (1981). Use and misuse of
the European Communities. DG XII, Brussels. sensory evaluation in research and quality control. Journal Dairy
Fisken, D. (1990). Sensory quality and the consumer: viewpoints and Science, 64, 2296–2302.
directions. Journal Sensory Studies, 5, 203–209. Stone, H., & Sidel, J. L. (1993). Sensory Evaluation Practices ((2nd
Gould, W. A., & Gould, R. W. (1988). Total quality assurance for the edition.)). New York: Academic Press, Inc.
food industry. Baltimore: CTI Publications Inc. York, R. K. (1995). Quality assessment in a regulatory environment.
Guinard, J. X., Yip, D., Cubero, E., & Mazzucchelli, R. (1999). Food Quality and Preference, 6, 137–141.