Solving real school timetabling problems with meta-heuristics

F. MELÍCIO1,3, P. CALDEIRA2,3, A. ROSA3

1
ISEL, R. Conselheiro Emídio Navarro, 1900 Lisboa, Portugal
2
EST-IPS, R. Vale de Chaves-Estefanilha, 2810 Setúbal, Portugal
3
LaSEEB-ISR-IST Av. Rovisco Pais, 1, TN 6.21, 1049-100 Lisboa, Portugal

Abstract: - School timetabling is a classical combinatorial optimization problem, which consists in assigning
lessons to time slots, satisfying a set of constraints of various kinds. Due mostly to the constraints this problem
falls in the category of NP-Complete problems. In this paper we try to show an implementation of a decision
support system that solves real timetabling problems from various schools in Portugal. This implementation is
based on the Simulated Annealing meta-heuristic. The constraints we use were obtained after inquiries made
to several schools in Portugal. We show the results on three schools from different levels of teaching.

Key-Words: - Timetabling, simulated annealing, meta-heuristics, combinatorial optimization problems, local

search methods.

1 Introduction overlaps of lessons of courses having common

Educational timetabling problems are known to be students.
difficult real world problems that have been studied 3. Examination timetabling. The scheduling for
in some detail over the years. This problem is NP- the exams of a set of courses, avoiding
Complete mainly due to the associated constraints overlapping exams of courses having common
[7], [12]. In order to implement a system that would students, and spreading the exams for the
fit the majority of the Portuguese education system, students as much as possible.
we define a set of general constraints. In this paper In this work we are mainly concerned with the
we are going to explain a decision support system classification class/teacher, because in Portugal
that it is used by more than 100 schools with almost every school even in universities students are
significant success in their timetabling elaboration. firstly joined together in groups with common
Meta-heuristics algorithms, like Simulated subjects. Nevertheless, we tried to formulate the
Annealing, Tabu Search, etc., have been applied timetabling problem in a general way in order to
with significant success to different combinatorial take into account all the requirements of every
optimization problems. school in Portugal. Thus, we have the following data
In this paper we explain one specific sets:
implementation of a Simulated Annealing algorithm • A set T = {t1 , , tm } of teachers.
adapted to the timetabling problem.
• A set C = {c1 , , cn } of classes. A class is a
group of students that attend the same subjects.
2 Problem Formulation • A set S = {s1 , , ss } of subjects.
The timetabling problem [9] consists of assigning a
set of lessons to time slots within a time period • A set R = {r1 , , rr } of rooms. Rooms are first
(typically a week), satisfying a set of constraints of grouped in subsets of the same kind, i.e., with
various kinds. It is widely accepted that the the same resources. Each subject has associated
timetabling problem can be divided in three main at least one type of room.
categories [4], [21]:
1. Class/Teacher timetabling. The weekly
• {
A set H = h1 ,… , hp } of time slots. The
scheduling of all classes, avoiding teachers number of time slots is equal to the number of
meeting two classes in the same time and vice- days, times the number of daily periods. Each
versa. period has the same duration and there can be
2. Course timetabling. The weekly scheduling for two types of periods. Periods with or without
all lessons of a set of courses, minimizing the teaching activities.
• A set A = {a1 ,… , al } of lessons. A lesson is the
 teaching unit. It is characterized by the • Certain kind of subjects should not be in
following tuple * * *
T , C , S . Where T is a * contiguous time slots.
For a complete description of the set of constraints
subset of the teachers set, C * is a subset of the we have used, see Table 1.
classes set and S * is a subset of the subjects set.
Each lesson has a duration expressed in time Constraint Description
slots.
There are two types of lessons: C0 Number of time slots of lessons that aren’t yet scheduled
Number of time slots of overlapped lessons
1. Simple lesson. Where C * = 1 and S * = 1 . C1, 2 (1-classes; 2-teachers)
Number of time slots exceeding the maximum allowed
C 3, 4 per day (3-classes; 4-teachers)
2. Compound lesson. Where C * ≥ 1 and/or Number of time slots exceeding the maximum
C 5, 6 consecutive time slots allowed (5-classes; 6-teachers).
S ≥1.
*
C 7 ,8, 9 Number of preferable time slots filled
(7-classes; 8-teachers; 9-subjects).

In general, a compound lesson means that we C10,11 Number of idle time slots (10-classes; 11-teachers)
have several classes joined together to attend a C12 Number of time slots of lessons without a room assigned.
certain subject or it means that a class can be
subdivided into subgroups to attend special subjects, C13,14,15 Number of time slots that are forbidden and are filled
with lessons (13-classes; 14-teachers; 15-subjects)
like laboratories, etc.
C16 Total number of teaching days for teachers
It is associated with each subject the kind of
Number of repetitions of lessons of the same subject in
room it must have, i.e., the resources that there must C17 the same class per day
exist in the room for a lesson of that subject should Number of time slots that doesn’t satisfy the predefined
happen. C18 space between lessons.
Table 1 Constraint set.
2.1 Constraints We also have introduced the concept of flexible
As it was stated in the beginning of this section, a constraint. Which means that a user may choose to
set of constraints must be satisfied in order to have a which category each constraint belongs.
valid timetable. The number and the kind of
constraints vary from school to school, even within Therefore the main objective of any Decision
the same school system. Nevertheless there are only Support System for this kind of problem should be
two categories of constraints: solving the hard constraints and minimizing the soft
• Hard constraints are constraints that physically constraints. Even if it is impossible to find any
cannot be violated. There are also other feasible solution, it is better to give an approximate
constraints in spite of not being any physical solution than none at all.
constraint they fall into this category because of
several reasons, for instance, because they are
governmental ruled. 3 Combinatorial Optimization
• Soft constraints are in general preferences and Problem (COP)
they do not represent a physical conflict. Any timetabling problem belongs to the class of
By hard constraints, we mean the following: combinatorial optimization problem. In general a
• A teacher cannot teach different lessons at the combinatorial optimization problem has a discrete
same time. finite search space S, and a function f, that measures
• A class cannot have different lessons at the the quality of each solution in S.
same time.
• Different classes cannot be held in the same
f :S → (1)
room at the same time. The problem is to find
• Class unavailabilities.
• Teacher unavailabilities. s ∗ = arg min f ( s ) (2)
s∈S
• Etc.
As soft constraints are mainly preferences they vary Where s is a vector of decision variables and f is
a lot among schools some examples are: the cost function. The vector s * is a global
• Teachers may prefer specific time slots. optimum. The neighbourhood N ( s ) of a solution s
• Teachers may prefer specific rooms. in S is defined as the set of solutions which can be
obtained from s by a move. Each solution
Feasible solutions
s ' ∈ N ( s) is called a neighbour of s .
For each s the set N ( s ) doesn’t need to be
listed explicitly, in general it is implicitly defined by
referring to a set of possible moves. Moves are
usually defined as local modifications of some part
of s . The “locality” of moves (under a
correspondingly appropriate definition of distance
between solutions) is one of the key ingredients of
local search. Nevertheless, from the definition above
Search space for all solutions
there is no implication that there exist “closeness” in
some sense among neighbours, and actually Figure 1 Search space of all solutions for
complex neighbourhood definitions can be used as the timetabling problem.
well. This operator can be quite complicated it might
even be a meta-heuristic. In our problem we did more or less the same
thing with the main difference that we didn’t relax
the hard constraints (user defined). Instead, we will
allow partial solutions to belong to the search space.
3.1 Search Space
We have a partial solution when there is at least one
When working with discrete domains it is possible
lesson that is not scheduled. Mathematically this can
to define the search space in terms of the possible
values that each variable can have [13]. For this be represented by augmenting each set H i with one
{
problem we have the set H = h1 ,… , hp } as the set more time slot, h0 . From a technical point of view,
of possible values that each lesson can have. we will assume that the search space (with partial
solutions) satisfies the following properties:
Definition Search space: The complete set of 1. The empty solution is in the search space
solutions that belongs to the search space is defined
∅∈S
by S = H1 × … × H l . If all H i are equal then
2. There is a path from any partial solution
S = H l and S = H = p l .
l
leading to other partial solution along which
the lessons are scheduled one after the other.
This value is an extreme case. For instance, there
are 1010 possible solutions if there are 10 lessons 3. All complete solutions in the search space
satisfy the hard constraints.
and 10 time slots. Even if we restrict each lesson to
a different time slot there will be 10! = 3, 628,800 In an attempt to limit the search space it is possible
to define at the beginning regions of the search
possible assignments. As it can easily be verified the
space that are forbidden, black holes. This can be
search space for this kind of problem is very large.
accomplished by defining a bipartite graph
However not all solutions are feasible, i.e., a feasible
solution has to have its lessons all scheduled and G = (V1 ,V2 , E ) , where every lesson belongs to V1
satisfying a certain number of constraints (hard and every time slot belongs to the other vertex set
constraints). A possible search space for this kind of V2 of this bipartite graph. The edge (i, j ) ∈ E
problem could be similar to the one shown in Figure means that lesson i can be given in time slot j .
1.
For certain problems it is very difficult to know This graph only takes into account the static
if there exists at least one feasible solution before constraints, i.e., class unavailabilities, teacher
starting any search algorithm. Thus any search unavailabilities, subject unavailabilities, etc. It is
algorithm should be able to walk across the search then possible to define the set H i for each lesson.
space even inside infeasible regions. One of the The search space thus formed could be like the one
most common ways to do that is to penalize shown in Figure 2.
constraints that are not satisfied and mixing them
together in a cost function.
 Feasible solutions
1. Select randomly two lessons i≠ j

“Black Holes” (forbidden solutions)

2. Exchange the time slots of each lesson
3. If any of the two lessons isn’t scheduled choose
the “best” time slot for the other

Figure 3 Double move adapted to the

timetabling problem.

Search space for all solutions

The size of this neighborhood is given by,
l (l − 1)
Figure 2 Search space with forbidden N ( s) = (3)
regions. 2
Nevertheless, the number of possible solutions is Where l is the number of lessons. Usually the
still a huge value, so any attempt to check them all number of time slots is much smaller than the
would be impossible for most real problems number of lessons.
instances. Nevertheless for real problems the size of this
neighbourhood is quite large, for instance, if there
are 1000 lessons, there will be 499500 neighbours
3.2 Local search methods for each solution. However it is known that only a
Any local search algorithm starts off with an initial fragment of this number of neighbours can actually
solution and then continually tries to find better improve the quality of a solution. Normally, the
solutions by searching the neighbourhood of the exchange of two lessons of two different classes
current solution. A local process can be viewed as a deteriorates the quality of the solution.
walk in a graph G = ( S , E ) where the vertex set is So, if we limit the exchanges of lessons
belonging to the same class the number of
the set of solutions S and there is an edge ( s, s ') in
neighbours would be,
E if and only if s and s ' are neighbours s ' ∈ N ( s ) .
The efficiency of any local search method depends
C
lc ( lc − 1)
Nc (s) = ∑ (4)
on the modelling [14]. A fine tuning of parameters c =1 2
will never balance a bad definition of the solution
set, of the neighbourhood or the cost function. In Where lc is the number of lessons belonging to
general the following rules apply for any local class c. In the above example, if we have 50 classes
search methods: and each class have 20 lessons, which makes a total
1. It should be easy to generate solutions in S . of 1000 lessons (same number as above). The size of
2. For each solution s ∈ S , there should be a path this reduced neighbourhood would be of 9500
linking to an optimal solution. neighbours.
3. The solutions in the neighbourhood of s Without considering compound lessons the total
should be in some sense close to s (strongly number of lessons is given by,
correlated to s ). C
It is important to define neighbourhoods in which it l = ∑ lc (5)
is possible to determine the best solution within a c =1
reasonable small amount of time.
And if each class has the same number of lessons
3.2.1 Double move expression (5) will become l = C × lc . The ratio
In our system we implemented this kind of move, between these two neighbourhoods will be equal to,
also called pairwise interchange. It is identical to
some other implementations of a neighbourhood N ( s) ( C ⋅ lc − 1) ⎛ 1⎞
operator [8]. = ≈ C ⎜1 + ⎟ ≈ C (6)
Nc (s) ( lc − 1) ⎝ lc ⎠
Where N ( s ) is the neighbourhood size associated
with the double move and N c ( s) is the
neighbourhood size related with the double move constraint is the only one that the user can’t modify
intraclasses. As it can be seen by equation (6) the and it is computed as follows:
ratio between the sizes of these two neighbourhoods K
is approximately proportional to the number of w0 = ∑ pk wk (8)
classes. Hence, for bigger problems better results k =1
one would expect to obtain with this reduced
neighbourhood. The neighbourhood has a major Where K is the number of constraints defined to the
However this move has a major drawback. As it specific problem and pk is the maximum value that
is explained in [2], any kind of move should attempt one can violate constraint k if a lesson is scheduled
to visit all possible time slots and in this case this in a given time slot. For the move operator defined
would never happen after an initial solution. So in in the last section the cost function is computed
our implementation we made a the following change incrementally, i.e., it is only computed the change in
from the described technique, cost between the new solution and the old one.
1 Select randomly two lessons i ≠ j from the
same class
5 Implementation
2 di ← day of lesson i The implemented system is based on a modular
3 d j ← day of lesson j implementation [19].
4 Unschedule lessons i and j
Graphical User
5 Find the best time slot in d j for lesson i Interface
6 Find the best time slot in di for lesson j
Figure 4 Double move with heuristic Automatic
improvement scheduler

4 Cost Function Database

The cost function plays a key role in any
optimization problem. It is through its calculation
that one can measure the quality of any solution. Figure 5 Block diagram of the implemented
system.
Hence its correct definition is essential for the
behaviour of any search algorithm. Our cost It was developed in C++ using an object oriented
function is given by the following expression: technique. It runs on Microsoft Windows ® and the
database is implemented in Microsoft Access®.
f ( s) = ∑ wk Ck (7)
k

Where s ∈ S is a solution in the search space (can

be a partial solution) and the values Ck represent
each of the objectives that we are trying to optimize,
weighted by a factor wk . This weight translates the
relative importance of the related constraint. The
objective of any search algorithm will be to find an
optimum solution s ∗ that minimize f ( s ) .
In our problem there is one objective that is
clearly much more important than the rest which is
the scheduling of all lessons. As we have stated
before partial solutions make part of the search
space. In order to do that and following the same
Figure 6 Graphical user interface.
idea expressed in [6] it is defined a constraint C0
that represents the sum in time slots of all The Automatic scheduler contains two different
unscheduled lessons. The weight that affects this and complementary algorithms.
 1. An iterative algorithm based on Simulated Where α is designated the cooling factor.
Annealing Typically, the value of α is chosen in the range
2. A heuristic constructive algorithm. 0.90 to 0.99.
The iterative algorithm is based on the original In the beginning of the algorithm’s execution, as
Simulated Annealing algorithm [18]. the temperature is high all new possible solutions
Simulated Annealing has been applied with have a big probability of being accepted
significant success to different combinatorial ( p > e −∆c / t ), and we can travel around the search
optimization problems. One of the reasons why SA
space without getting much information from it. It
has been widely used is mostly because it is an
seems in this phase much like a random algorithm.
algorithm very easy to implement and it doesn’t
In order to avoid the time spent in this initial
depend on the problem structure or any solution
stage, we choose to start with a lower temperature
strategy. The basic algorithm is described in Figure
but in a good region of the search space.
7.
The problem is then how to choose the
temperature value to start with the annealing
Generate initial solution i algorithm.
Compute cost(i) We implement an adaptive technique [17] which
Compute initial temperature t0 depends on the quality of the starting solution.
t= t 0
while stop criteria is not satisfied
σ ∞2
{ tk (i ) ≈ (10)
while equilibrium is not reached E∞ − f (i ) − γ ∞σ ∞
{
Generate new solution j Where E∞ and σ ∞ are respectively the expected
∆c = cost(i)-cost(j)
if ((∆c ≤ 0) || (random() < e
− ∆c / t
)) cost and standard deviation of cost over the entire
{ search space, f (i ) the cost of the initial heuristic
i= j solution and γ k represents the number of standard
}
} deviation units the expected cost is greater than the
Update temperature t minimum cost at temperature tk ,
}

Figure 7 Simulated Annealing Algorithm Ek = f (ik min ) + γ kσ k (11)

The design of a good annealing algorithm is Assuming a normal distribution of the solutions
nontrivial [10], it generally comprises four parts: for almost all temperatures [1], we expect the offset,
1. Search space. γ k , to remain approximately constant over all
2. Neighbourhood set.
temperatures except near the temperature
3. Cost function.
corresponding to the cost function optimal value
4. Annealing schedule.
where it converges rapidly towards zero.
We have explained the first three items in the Equation (10) tell us that the starting temperature
previous sections we are going now to explain with for the annealing phase depends on how good the
a little more detail the fourth item. initial solution is, regarding the cost function.
It is widely accepted that this algorithm is very To get an initial solution we developed a
time consuming, i.e., it needs a great amount of time heuristic algorithm based on the following ideas,
to get to a near optimum solution. 1. Sort all lessons unscheduled based on its
Therefore our efforts were made in order to urgency.
speed up the algorithm without loss of quality. 2. Schedule each unscheduled lesson in its best
As it can be seen in Figure 7 the algorithm starts free time slot.
with a parameter called temperature with a high
This notion of urgency is similar to the one used
value, which is then lowered during the annealing
by several authors [15], [24], is given by,
process generally by the following expression [3],
[11], [22], 1
u (i ) = (12)
tk = α ⋅ tk −1 (9) Hi
 Where u (i ) is the urgency coefficient for lesson timetables are made in just a day or two with one or
two persons attached to it.
i and H i is the number of free time slots for
lesson i taking into account all the hard constraints
(section 4). 0 < u (i ) ≤ 1 for all lessons. When 7 Conclusion
H i = 0 means that lesson i is impossible to As it is well recognized people in general are not
interested in solving their optimization problems to
schedule and it is withdrawn from the lessons set.
optimality or even close to optimality. They are
The results obtained from this hybrid algorithm
more often interested in “good enough – soon
were compared with other techniques and were very
enough – cheap enough” solutions to their problems
good [20].
[5].
We also think that good choices of specific parts
of each problem are fundamental for the success of
6 RESULTS any search algorithm. As [23] showed there are no
This system is now used in several schools from the algorithms either deterministic or stochastic
Portuguese education system. We show the results behaving the same on the total set of search and
in three typical schools. The data is shown in Table optimization problems defined on a finite and
2. discrete domain.
In this paper we illustrate the implementation of
ISEL EST Escola Sec. customized simulated annealing algorithm for the
Data (DEEA) Fernando Namora
C1 C2 C3
timetabling problem. It is worthwhile to say that by
now more than 100 schools from the Portuguese
Classes 21 124 54 education system uses this system.
Teachers 79 208 112 Our main conclusion from this work is that we
Subjects 250 1345 492
can solve a very difficult scheduling problem with
simulated annealing and be time competitive, but we
Rooms 21 90 50 must be very careful in the way we choose to
#Lessons 359 1908 1357 implement specific parts of the problem.
(#Time slots) (1135) (3175) (1660)

Table 2 Three Portuguese schools.

References:
Every test was made with real data, this means
[1] Aarts, E. H. L., Van Laarhoven, P. J. M., Korst,
that, every constraint and the correspondingly
J. H. M., “Simulated annealing”, Local Search
weight was defined by each school.
in Combinatorial Optmization, E. H. L. Aarts
We used the real timetable for comparison and
and J. K. Lenstra (eds.), John Wiley & Sons,
the results are summarized in Table 3.
1997.
[2] Abramson, D. “Constructing school timetables
C1 C2 C3 using simulated annealing: sequential and
Real Timetable 479,1 2138,2 481,6 parallel algorithms”. Management Science, v.
Best result from this algorithm 141,0 1998,1 277,0
37, pp. 98-113, 1991.
[3] Abramson, D., Dang, H., Krishnamoorthy, M.,
Gain in percentage 70 % 7% 57 % “An Emprirical Study of Simulated Annealing
Table 3 Results from the 3 schools. Cooling Schedules”, Griffith Univ. report,
The difference in case C2 is due to the fact that Nathan, Qld, Aus. 1994.
the real timetable was made with a previous version [4] Bardadym, V.A. “Computer-Aided School and
of this system while the other two were hand made. University Timetabling: The New Wave”. In
It is also important to mention that the tests were Burke, E.K., Ross, P. (eds), Practice and
made in a Pentium IV, 1,6MHz and could last from Theory of Automated Timetabling, v. 1153,
20 minutes for C1 until 2 hours for C2. Lecture Notes in Computer Science, pp. 22-45.
Nevertheless, the gain obtained in using such a Springer-Verlag, Berlin, 1996.
system is much greater than just the one expressed
in Table 3. Before using this system each school
needed several weeks and a group of people to do it.
Now, after the introduction of such a system real
[5] Burke, E., Hart, E., Kendall, G., Newall, J., [16] Huang, M. D., Romeo, F., Sangiovanni-
Ross, P., Schulenburg, S., “Hyper-Heuristics: Vincetelli, A., “An Efficient General Cooling
An Emerging Direction in Modern Search Schedule for Simulated-Annealing”, Proc.
Technology”. In Handbook of Meta-Heuristics, IEEE-ICCAD, pp. 381-384, 1986.
Glover, F., Kochenberger, G., (eds.), pp. 457- [17] Varanelli, J. M., Cohoon, J. P., “A Fast Method
474, Kluwer, 2003. for Generalized Starting Temperature
[6] Catoni, O., “Solving Scheduling Problems by Determination in Monotonically Cooling Two-
Simulated Annealing”, SIAM Journal of Stage Simulated Annealing Systems”, Report
Control Optimization, v. 36, No. 5, pp. 1539- CS-9508, Dep. Computer Science, University of
1575, 1998. Virginia, 1995.
[7] Cooper, T.B., Kingston, J.H. “The Complexity [18] Kirkpatrick, S., Gellati, C. D. , Vecchi, M. ,
of Timetable Construction Problems”. In Burke, “Optimization by Simulated Annealing”,
E.K., Ross, P. (eds), Practice and Theory of Science, vol. 220, pp. 671-680, 1983.
Automated Timetabling, v. 1153, Lecture Notes [19] Melício, F., “THOR: Uma ferramenta para
in Computer Science, pp. 283-295. Springer- elaboração de horários duma escola”, Proc. 3º
Verlag, Berlin, 1996. Meeting OE, pp. 77-82, Porto, Jun 1997.
[8] Costa, D. “A tabu search algorithm for [20] Melício, F., Caldeira, P., Rosa, A., “Solving
computing an operational timetable”. European Timetabling Problem with Simulated
Journal of Operational Research Society, v. 76, Annealing”. Filipe, J. (ed.), Kluwer Academic
pp. 98-110, 1994. Press, pp.171-178, 2000.
[9] de Werra, D. “An introduction to timetabling”. [21] Schaefer, A. “A survey of automated
European Journal of Operational Research timetabling”. Artificial Intelligence Review, v.
Society, v. 19, pp. 151-162, 1985. 13, pp. 87-127, 1999.
[10] Dowsland, K.A., “Off-the-peg or made-to [22] Thompson, J., Dowsland, K. A., “General
measure? Timetabling and Scheduling with SA Cooling Schedules for a Simulated Annealing
and TS”. In Proceedings of the Second Based Timetabling System”, Proc. 1st Intl.
International Conference on the Practice and Conf. on the Pratice and Theory of Automated
Theory of Automated Timetabling, pp. 7-26, Timetabling, pp. 345-363, 1995.
1997 [23] Wolpert, D.H., Macready, W.G., “No Free
[11] Elmohamed, S., Coddington, P., Fox, G. “A Lunch Theorems for Optimization”, IEEE
Comparison of Annealing Techniques for Transactions on Evolutionary Computation,
Academic Course Scheduling”. Proc. 2nd Intl. vol. 1, no. 1, pp. 67-82, 1997.
Conf. On the Pratice and Theory of Automated [24] Wright, M. “School Timetabling Using
Timetabling, pp. 146-166, 1997. Heuristic Search”. Journal of the Operational
[12] Even, S., Itai, A., Shamir, A. “On the Research Society, v. 47, pp. 347-357, 1996.
complexity of timetabling and multicommodity
flow problems”. SIAM Journal of Computation,
v. 5, pp. 691-703, 1976.
[13] Hemert, J., Back, T., “Measuring the Searched
Space to Guide Efficiency: The Principle and
Evidence on Constraint Satisfaction”,
Proceedings of the Seventh International
Conference on PPSN, Lecture Notes in
Computer Science 2439, Springer-Verlag,
pp.23-43, 2002.
[14] Hertz, A., Widmer, M., “Guidelines for the use
of meta-heuristics in combinatorial
optimization”, European Journal of
Operational, vol. 151, pp. 247-252, 2003.
[15] Hilbert H., “High School Timetabling in
Germany-Can it be done with MIP?”. In
Proceedings of the Second International
Conference on the Practice and Theory of
Automated Timetabling, pp. 325-327, 1997.