21

CHAPTER 2

LITERATURE SURVEY

A survey of the existing types of software agents’ classification,

development infrastructure tools and the problems that exist with these tools
are presented. The survey methodology is presented in Section 2.1. A detail of
the classification is provided that is used to review the software agents in
Section 2.2. In Section 2.3 a survey (2.3.1) of existing development tools is
provided which is used for agents’ development and the (2.3.2) problems exist
with that. Also, 2.3.3 presents the IBM aglets and its features. In Section 2.4
the specific problems that are taken for investigation are presented. Finally,
the chapter is summarized in Section 2.5.

2.1 SURVEY METHODOLOGY

In this thesis are presented various problems and issues that exist with
the digital libraries and how the applications of agents solve a few issues are
exhibited. The detailed classification of intelligent agents based on various
criteria is reviewed first and then the tools used for the development of
intelligent agents are discussed. This survey on intelligent agents enables one
to identify the various characteristics of agents’ based systems that are
developed for solving different application problems. Survey has also been
used to identify the existing suitable tools for development of new
applications.
 22

2.2 SURVEY OF INTELLIGENT AGENTS CLASSIFICATION

Agent technology is a new area of research in the rapidly growing

software development field. They are of profound use in the Internet to assist
any user to obtain the requested information from the overwhelming
information available in it. Their utility has been very much realized and
various types of agents have been developed and are available on the Internet.
It is found that the classification of agents is still incomplete (Stuart Russell
2003, Walter Brenner 1998, Dimitris N. Chorafas 1998, Hyacinth S. Nwana
1996b, Stan Franklin 1996, Borking.J.J 1999, Hiking and Climbing 1998,
Hyacinth S. Nwana 1996, Stuart J. Tuck 1998, Katia Sycara 1996, Michael R.
Genesereth 1994). In this survey the various types of classification available in
the literature, their application and drawbacks are reviewed. In this chapter an
attempt has been made to provide a new method of classification, which can
be adopted as a standard method for agent classification. The method
suggested is based on the fact that the agents can be classified applying six
criteria, namely, level of intelligence, architecture, characteristics,
functionality, user interactiveness and a number-based approach. Since the
agents have a broad spectrum of functionality even the classification suggests
of an overlap; that is one agent may satisfy two or more criteria. The
continuing research and development in the field of agent technology may
lead to the identification of new criteria, which will avoid any ambiguity.

An agent is an active run time entity that is used to perform various

tasks in any complex system environment in accordance with the needs of user
or other system elements. The important difference, between an agent and a
traditional software program is the capability of an agent to follow its goals
autonomously. An agent does not need to have each of its steps approved by
its user or other agents; rather it is capable of acting alone. The degree of an
agent's autonomy will differ in various types of software agents.
 23

Every agent must have a certain minimum degree of intelligence in

order to be designated as being an agent, because it acts as a virtual person to
the user. The intelligence of an agent is formed from three main components.

i. Its internal Knowledge base.

ii. Reasoning Capability based on Knowledge base.
iii. Adaptive behavior (Learning capability).

Agents are referred by different names (Stan Franklin 1996, Katia

Sycara 2001) such as Knowbot, Softbot, Taskbot, Userbot, Personal Assistant,
Transport Agent, Mobile Agent, Search Agent, Report Agent, Presentation
Agent, Navigation Agent, Role Agent, Management Agent, Search and
Retrieval Agent, Domain-specific Agent, Packaging Agent, Artifacts,
Webbots based on their characteristics and functionality (As presented in
Table 2.1).

The classification of agents aims at categorizing the existing agents

according to certain criteria. This is to clearly outline the boundaries of the
different types of agents. However, in the case of the agents the boundaries
may overlap as a particular agent may satisfy more than one classification
criteria.
 24

Table 2.1 Agent Classification

S.No Criteria Sub- Purpose/Usage

Categories
1. Level of Weak Not environment specific and
Intelligence operate autonomously with
minimal intervention and able
to sensing and perceiving the
environment.
Strong Contracted using a cognitive
approach and environment
specific.
2. Architecture Reactive No Explicit Knowledge
Representation
Respond to the changes in the
environment through sensors
and actors.
Do not possess an internal
symbolic model of the
environment
Deliberative Have the components like
planner, scheduler, executor
Work in a dynamic and highly
complex environment
Explicit symbolic model of the
environment.
Hybrid Combination of both
3. Functionality Interface Increases the sophistication to
System interact with system
May add speech and Natural
Language Components
Advisory User in help or diagnostics
Filtering Remove unwanted Information
Retrieval Retrieve information/
documents from servers
 25

S.No Criteria Sub- Purpose/Usage

Categories
Classification Classify
information/documents from
servers
Navigation Navigate to external or internal
networks
Monitoring Monitor the events in the
system and inform to the user
Recommender Recommend a system as per
the user requirement
Profiling Profile the user requirement
and maintain a one to one
relationship with customer
Information Managing, manipulate and
collect the information.
4. Characteristics Autonomy Capable to follow the goals
independently
Reactiveness React to the changes in the
environment
Proactiveness Able to understand the
environment and change the
action accordingly
Goal- Able to define the objective
orientation and work for its improvement
Stationary/ Fix itself in a single system
Mobility called stationary agent
Able to move to different
environment to solve the task
is the mobile agent.
Learning Able to understand the input
and add more value
Adaptability Ability to adapt the changes in
the environment
 26

S.No Criteria Sub- Purpose/Usage

Categories
Communication Able to communicate with its
neighbor through ACL
Co-operation De-centralized automatic
ability of the agent to solve the
problem by associating one
with its neighbor
Co-ordination Centralized controlled ability
of the agent to solve the
problem with its neighbor
Self- Able to share the knowledge
proclamation voluntarily with its neighbor
Personal Trustworthiness
Reliability
Security
5. User- Intractable/ Interact with User
Interaction Instruct able
Non- Is not user interactive
intractable/
Non-
instruct able
6. Number Single Just one Agent to do required
(UserBot) task to functionality
Multi-Agent More then one agent involved
(TaskBot) in solving the problem.
May be classified as
Homogeneous or
Heterogeneous

2.3 SURVEY ON INTELLIGENT AGENT INFRASTRUCTURE

In this section, first the survey of the various tools available and their
capabilities for the intelligent-agent based system developments and the
problems exist with these tools are presented (Jeff Nelson 1999, Danny Lange
 27

1998, Joseph P. Bigus 2001, Jeff Heatson 2002). The Section 2.3.1 is provided
with the summary of survey of the intelligent agent tools and infrastructure. In
Section 2.3.2 the various problems and capabilities of these existing tools are
discussed. Finally Section 2.3.3 presents the features of IBM Aglet and its
advantages.

2.3.1 Survey on Various Tools and Features for Intelligent Agents based
System Development

An agent language is a programming or development environment that

can be used for practical realization of intelligent software agents. The choice
of a suitable language has a large effect on the agent architecture that is to be
produced. The aim of this section is to define a number of basic requirements
that agent systems make on a programming language and also to investigate
the suitability and practicality of existing languages. Many such criteria like
object orientation, platform independence, communication capability, security,
code manipulation, persistence data storage, extensibility etc, are taken in to
account for the selection of a programming language or its development
environment. The Java programming language is most suitable for the agent
development, as it basically supports two concepts, such as networking and
platform-independence. There are also many other tools and frameworks
developed to support the agent construction. A few of these are taken for our
investigation and their detailed summary is presented in the following Table
2.2.
 28

Table 2.2 Agent Development Tools and its Nature

S.No. Name of the Tools Features

1. IBM Agent Building and Java based framework and facility
Learning Environment to link the java applications
(ABLE) It provides a set of reusable java
beans.
It provides the facility for neural
net based classification, clustering,
prediction
Able Agent GUI
2. Agent Builder (Reticular It is a tool for constructing java
Systems Inc.) based agents gives a facility to
specify the behaviors.
It follows the Belief-desire-
intention (BDI) architecture as
well as has a facility to integrate
the KQML based communication.
A special agent interpreter
monitors incoming messages and
processes the agent’s behavioral
rules.
Reticular Agent Definition
Language (RADL) is an extension
of Shoham’s Agent-0 model and
uses behavior rules to define the
agent’s mental model state changes
and actions when given a set of
conditions.
 29

S.No. Name of the Tools Features

3. FIPA-OS FIPA Open Source originally
developed at Nortel Networks
Uses the FIPA agent
communication language
standards.
A directory facilitator or yellow
pages service, and an agent
communication channel for FIPA-
compliant messaging and
interaction protocols.
FIPA-OS includes several agent
shells that are base classes that
make use of the platform services
and that can be extended to add
customized behaviors.
A visualization tools gives a
graphical interface to the FIPA-OS
platform.
4. Gossip Gossip developed at Tryllian Inc.’s
It is mobile agent software.
It uses Java to implement a secure,
distributed Multiagent
environment.
It uses a sense-reason-act loop to
control the interaction between the
agent and the outside world.
 30

S.No. Name of the Tools Features

Secure Sockets Layer (SSL)
connections are used to transmit
the agent code and data.
Specially designed tool kit used to
develop the agents.
The core agent (body) can be
configured by defining the
knowledge (data) and behavior
(actions).
5. Java (JADE) Development Developed at CSELT S.p.A in
Framework Torino. Italy.
It is a FIPA-compliant toolkit for
creating Multiagent systems
applications
It provides a set of tools for
debugging and deploying
distributed agents.
It provides a set of agent services
including an agent-naming service,
yellow pages service, transport
protocols, and interaction protocols
that are FIPA compliant.
It supports JVM, CORBA, RMI
6. Java Agent Template Lite Developed at Stanford University
(JATLite) that can be used to build
Multiagent systems.
It supports the TCP/IP and KQML
protocol communication.
 31

S.No. Name of the Tools Features

The framework provides additional
security that checks the agent name
and password for more secure
connection.
The framework supports both
synchronous and asynchronous
message passing.
7. Voyager Developed at Object Space Inc.
It uses an agent-enhanced Object
Request Broker (ORB) that is
written entirely in Java.
Voyager agent has mobility and
autonomy that is provided in the
base class.
Voyager uses serialization to
stream the agent’s state as the
agent moves from location to
location.
Voyager also includes a security
manager that can be used to restrict
the operations that an agent can
perform.
8. ZEUS Developed by British Telecom
It supports the autonomy, co-
operation, portability and multi-
threading.
It supports KQML communication
and BDI architecture.
 32

2.3.2 Problems and Capabilities of Existing Agent System Development

Tools

Agent development is one of the major complex tasks because of the

multi-fold requirements. It also increases the software complexity and the
development needs of many technology independent abstractions. Generally
there are two major viewpoints which exist in the development of agents. The
artificial viewpoint (Danny Lange 1998) specifies that an agent needs to be
proactive, intelligent and must condensate. The other software engineering
viewpoint (Danny Lange 1998) specifies that the multi-agent system be made
up of multiple independent and encapsulated program entities interacting with
each other in a specific application context.

An agent-oriented system development needs good tools. It involves

various issues such as architecture, functionality, behaviour and
implementation. The problem with the existing tools or languages is that in
most cases it does not enforce any standard for these issues. On the other hand
sometimes the requirement satisfaction does put questions on certain aspects
of these issues. Most of the industrial developers focus only on the
functionality and implementation aspects. Very few tools and language
developers take the responsibility of providing different facilities and others
leave it as the programmer’s choice to develop the required facilities.

Generally there are certain criteria taken into account while selecting
the proper software for the agent development in compliance with basic
abstract agent characteristics. These characteristics are platform
independence, secure execution, event handling, dynamic class loading,
multithread programming, object sterilization, agent communication, easy
integration, resource control, object ownership control, platform independent
object mobility, preservation and resumption of the execution state, etc.
 33

Java basically provides various facilities to support such development;

the features include platform independence, secure execution, dynamic class
loading, multi-threading, object serialization. But, it does not provide facilities
for resource control, protection of references, object ownership of references,
preservation and resumption of the execution state and agent communication.
But, all these facilities are provided through the IBM Aglet and its full
technical details are discussed in the following section.

2.3.3 IBM Aglets Technical Features and Advantages

Aglets are Java-based autonomous agents (Danny Lange 1998)

developed by IBM. In this current research we use these aglets for different
systems development. They provide the basic capabilities that are required for
mobility. Each aglet has a globally unique name. A travel itinerary specifies
the destinations to which the agent must travel and what actions it must take at
each location. In order for an aglet to run on a particular system, that system
must be running an aglet host application. This application provides a
platform-neutral execution environment for the aglet. The aglet workbench
includes a configurable Java security manager that can restrict the activity of
an aglet on the system in the same way that the default security manager
restricts the activities of an applet.

Aglets can communicate by using a whiteboard/blackboard that enables

agents to collaborate and share information asynchronously. Synchronous and
asynchronous message passing is also supported for aglet communication.
Aglets are streamed using standard Java serialization or externalization. A
network agent class loader is also supplied, which enables an aglet’s byte code
stream and state to travel across a network.
 34

This model was designed to benefit from the agent characteristics of

Java while overcoming some of the deficiencies in the language system. In
the aglet object model, a mobile agent is a mobile object that has its own
thread of control, is event-driven, and communicates by message passing.

Aglet API Model

This model defines a set of abstractions and the behavior needed to

leverage mobile agent technology in Internet-like, open wide-area networks.
The key abstractions are aglet, proxy, context, and identifier.

Aglet: An aglet is a mobile Java object that visits aglet-enabled hosts in

a computer network. It is autonomous, because it runs in its own thread
of execution after arriving at a host, and reactive because it can respond
to incoming messages.
Proxy: A proxy is a representative of an aglet. It serves as a shield that
protects the aglet from direct access to its public methods as shown in
the Figure 2.1. The proxy also provides location transparency for the
aglet; that is, it can hide the aglet’s real location. This means that an
aglet and its proxies can be separated so that a local proxy hides the
remoteness of the aglet.
Context: A context is an aglet’s workplace. It closely corresponds to
the place concept. It is a stationary object that provides a means for
maintaining and managing, running aglets in a uniform execution
environment where the host system is secured against malicious aglets.
One node in a computer network may run multiple server processes
(engines), each server may host multiple contexts. Contexts are named
and thus can be located by the combination of their server’s address and
their name.
 35

Identifier: An identifier is bound to each aglet. This identifier is

globally unique and immutable throughout the lifetime of the aglet.

Proxy Aglet
Clients Interaction

Figure 2.1 Relationship between Aglet and Proxy

Behavior supported by the aglet object model is based on a careful

analysis of the life and death of mobile agents. There are basically only two
ways to bring an aglet to life: either it is instantiated from scratch (creation) or
it is copied from an existing aglet (cloning). To control the population of
aglets you can of course destroy them (disposal). Aglets are mobile in two
different ways – actively and passively. The active approach is characterized
by an aglet pushing itself from its current host to a remote host (dispatching).
A remote host pulling on aglet away from its current host (retracting)
characterizes the passive type of aglet mobility. When aglets are running, they
take up resources. To reduce their resource consumption, aglets can go to
sleep temporarily, releasing their resources (deactivation), and later can
brought back into running mode (activation). Finally, multiple aglets can
exchange information to accomplish a given task (messaging).

This is the minimum set of operations required to create and manage a

distributed mobile agent environment. When the Aglet API is created, one of
the goals is to create a lightweight API that would be both easy to learn to use
and sufficiently comprehensive and robust for real applications. The
 36

fundamental operations of an aglet are creation, cloning, dispatching,

retraction, deactivation, activation and disposal. The agent life-cycle model is
shown in the Figure 2.2.

Creation: The creation of an aglet takes place in a context. The new

aglet is assigned an identifier, inserted into the context, and initialized. The
aglet starts executing as soon as it has been successfully initialized.

Cloning: The cloning of an aglet produces an almost identical copy of

the original aglet in the same context. The only differences are the assigned
identifier and the fact that the execution restarts in the new aglet. Note that
execution threads are not cloned.

Dispatching: Dispatching an aglet from one context to another will

remove it from its current context and insert it into the destination context,
where it will restart execution. We say that the aglet has been “pushed” to its
new context.

Retraction: The retraction of an aglet will pull (remove) it from its

current context and insert it into the context from which the retraction was
requested.

Activation and Deactivation: The deactivation of an aglet is the

ability to temporarily halt its execution and store its state in secondary storage.
Activation of an aglet will restore it in the same context.

Disposal: The disposal of an aglet will halt its current execution and
remove it from its current context.
 37

Context A Context B

Retract
Dispatch
Aglet Aglet
Dispose

Clone Create Deactivate Activate

Class Disk
File Storage

Figure 2.2 Aglet Life-Cycle Model

The Aglet Event Model

The aglet-programming model is event-based. The model allows the

programmer to “plug in” customized listeners into an aglet. Listeners catch
particular events in the life cycle of an aglet and allow the programmer to take
action, for example when the aglet is being dispatched, three kinds of listeners
exist:

Clone Listener: Listens for cloning events. One can customize this
listener to take specific actions when an aglet is about to be cloned, when the
clone is actually created, and after the cloning has taken place.

Mobility Listener: Listens for mobility events. One can use this
listener to take action when an aglet is about to be dispatched to another
context, when it is about to be retracted from a context, and when it actually
arrives in a new context.
 38

Persistence Listener: Listens for persistent events. This listener

allows the programmer to take action when an aglet is about to be deactivated
and after it has been activated.

The Aglet Communication Model

Aglets communicate by message passing. The messaging facility

allows aglets to create and exchange messages in flexible ways. By default,
aglet messaging does not assume concurrent message handling. This means
that all messages are handled one by one. In addition to Aglet and Aglet
Proxy, the following are key components of the aglet communication model.

Message: A message is an object exchanged between aglets. It allows

for synchronous as well as asynchronous message passing between aglets.
Message passing can be used by aglets to collaborate and exchange
information in a loosely coupled fashion.

Future Reply: A future reply is used in asynchronous message sending

as a handle that allows the message sender to receive a reply asynchronously.

Reply Set: A reply set can contain multiple future reply objects and is
used to get results as they become available. With this object, the sender can
also choose to get the first result and ignore subsequent replies.

In case of this aglet, it does not support any inherent intelligent

implementation; they do not contain an inference engine, neural networks, or
any other artificial intelligence technology. It is the programmer’s
responsibility to implement things explicitly.
 39

2.4 SURVEY OF SPECIFIC PROBLEMS OF DIGITAL LIBRARIES

There are many issues which exist in Digital Libraries and these are
explained in Section 1.4. Most of these issues are dynamic and knowledge
sensitive. Keeping in vision to improving retrieval effectiveness, four such
problems are taken for the present investigation. These problems are optimal
content allocation in federated digital libraries, intelligent domain specific
proxy design to improve caching, distributed text classification and user
adaptive retrieval.

2.4.1 Optimal Content Allocation

The web content storage management is one among the important

issues, which decides fast and economical retrieval of information in the
federated or distributed digital libraries (Frew J. 2000, Leon Zhao J. 1999).
The system design and development necessitates the need for intelligent
solution in order to address the above requirements.

The delivery of large files to individual users such as Web sites or

blogs or application programs (Frew J. 2000, Cezary Mazurek 1999,
Cardellini V. 1999, Israel Cidon 2002) to the envisioned federated web
enabled video digital library network is one of the important tasks to be
envisaged. As the members of the libraries are scattered across the different
countries contributing to the unpredictable user requests from various regions
simultaneously (Mary Y.Y 2002, Azer Bestavros 1998, Teodoro. G 2003,
Cardellini V. 1999) various approaches such as replicating the information
across mirrored server architecture (Junehwa Song 2002, Cardellini V. 1999)
and increasing the bandwidth capacity (Mary Y.Y 2002) to provide fast and
dense server capacity are implemented.
 40

The huge rate of increase in the data necessitates the need for massive
scale storage architecture at a faster rate than the growth of processor and
disks. In addition, reliance on hardware improvements alone is not sufficient
to keep pace with the demand for data storage.

There is another approach to reduce the bandwidth, which consists in

program caching (Nussbaumer .J.P 1995, Schafa .F 1995). This approach is
more reasonable than the server replication. In this method, a number of
caches are distributed throughout the network to cater to the multiple requests
from its caches rather than going for individual requests. The expected
competition among the service providers may benefit those that can provide
these services at the expected quality level of service at the lowest price. For
example the web content servers, the service providers pay for the storage of
contents copies in the servers and also for using communication links for
transferring information on the network to the content consumers. The content
providers have to manage the storage of content copies within the servers such
that the overall cost is minimized. The question is how many copies should be
allocated for each content and at which server? As a consequence the
researchers are motivated to focus on optimizing not only the delivery network
but also electronic content allocation.

European Union Council, on the 28 th of February 2005, reached a

political agreement on the eContentplus Programme followed by a vote by the
European parliament. This agreement supports the development of multi-
lingual content (http://cordis.europa.eu/econtent, Coralie Jonvel 2006) for
innovative, on-line services across the EU. The Programme addresses specific
market areas like geographic content (as a key constituent of public sector
content), educational content, cultural, scientific and scholarly content where
development has been slow. The Programme also supports EU-wide co-
 41

ordination of collections in libraries, museums and archives and the

preservation of digital collections so as to ensure availability of cultural,
scholarly and scientific assets for future use.

The 4-year programme (2005-08), proposed by the European

Commission in February 2004, will have a budget of € 149 million to tackle
the fragmentation of the European digital content market and improve the
accessibility and usability of geographical information, cultural content and
educational material.

There are 25 group member states that exist in the European

Union. These members are Austria, Belgium, Cyprus, Czech
Republic, Denmark, Estonia, Finland, France, Germany, Greece,
Republic of Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta,
Netherlands, Poland, Portugal, Slovakia, Slovenia, Spain, Sweden,
United Kingdom.
The people of these states use different languages for
communication such as Czech, Danish, Dutch, English, Estonian,
Finnish, French, German, Greek, Hungarian, Italian, Latvian,
Lithuanian, Maltese, Polish, Portuguese, Slovak, Slovene and
Spanish.
Most of the people living in these regions use one or more
languages for communication. Also, transmigration of the people
is taking place in a normal way.

It is easy to see that content is generated in different places and

maintained in many different servers. Also these contents are of different
languages. Normally it is impossible to store all the contents in all the servers.
So the basic question here is where to store which content is a effective way.
 42

Nusssbaumer (Nussbaumer .J.P 1995, Schafa .F 1995) and Damianos Gavalas

(Damianos Gavalas 2002) introduced in 1994 and later improved by Israel
Ciden et. al. in 2001 (Israel Cidon 2002), involving a hierarchical architecture
which takes into account the trade-off between bandwidth and storage
requirements and computed the best level of hierarchy for the server location
which minimizes the combined cost of communication and storage. Since the
requests are unpredictable as in a real network (Internet), the problem can be
perceived as the Partially Observable Markov Decision Problem (POMDP) for
determining the optimal policy. S.H. Gray Chan had proposed another
hierarchical storage system (Gray Chean .S.H 2003), by classifying video-
objects as “so-popular” and “not-so-popular” objects according to the
frequency of video object access. Here the frequently accessed “so-popular”
objects are moved to secondary levels whereas the less frequently accessed
objects are stored at the root level itself.

All the above methods have not taken into account to use content
access semantics. In this present work, a multi-agent based user access pattern
oriented optimal content allocation method for federated digital libraries is
proposed which takes into account the content semantics and frequency of
content access in a particular region, in addition to the storage and bandwidth.
Normally no guarantee can be given for frequency of content access. While
considering the group of user accesses, the semantics of content access pattern
can be learned using a multi-agent system. By taking the apriori information
of the particular pattern for the particular period, the content is moved to the
particular region. The content access will vary according to the regional,
subject, cultural, research interest and so on. The cost effective method for
content allocation in a dynamic environment is possible by considering the
content semantics of user communities. As the number of requests for an
object is unknown, it is assumed to be zero and will be stored in the root
 43

server. If it is accessed from the root then the total traffic in the system will
increase unpredictably, where the number of requests increase The user
access pattern learning aids in identifying the (i+1)th object pattern for
allocation in the same locations, if the (i+1)th object belongs to the ith object
pattern.

Frances Braziner (Frances Brazier 1998) has designed a dynamic

electrical grid load management system for cost effective usage of electrical
production. Such a system is able to do user community classification based
on the negotiations between utility companies and their customers. This kind
of system design needs the user content access pattern learning for adaptation
because this access pattern will change dynamically over time. Whereas the
electrical power produced has to be consumed immediately, the electronic
content once made available can be used at any point of time and has content
semantics.

This work considers the semantics of various content types, user

communities, access patterns, locations and the relation between the user
groups and key contents. Also in order to use the bandwidth effectively, the
user must move the content to the frequently accessed area.

2.4.2 Domain Specific Intelligent Agent Cache Proxy

Content caching is one among the important issues which decide fast
and economical retrieval of information in the web digital libraries (Frew J.
2000, Leon Zhao J. 1999). Caching proxies (Abrams .M 1995, Almeida .J
1998, Elangelos P. Markatos 2002, Dilley .J 1999, Anawat Chankhunthod
1995, Brooks .S. M. C. 1995) become a critical component to handle web
traffic and reduce both network and client latency. These caching proxies
 44

effectively migrate copies of popular contents from servers to proxy servers.

Web client users see shorter delays when requesting the contents. Network
managers see less traffic as the caching proxies reduce the server request rates.

These caching proxies adopt various policies (Elangelos P. Markatos

2002, Dilley .J 1999), such as LFU (Least Frequently Used), LFU with
Dynamic ageing, GDSF (Greedy Dual Size Frequency), LRU (Least Recently
Used), etc. Such policies yield a hit rate ranging from 30-50 percent.

Web enabled proxy posing problems to the growing volume of

applications calls for dynamic scalability. The unparalleled growth of
resources drives one to develop hierarchical proxy-cache (Anawat
Chankhunthod 1995). It yields 50% hit ratio and reduces server load
dynamically. Harvest cache has shown this result experimentally. It is
necessary to observe that the replacement of small size content has been done
in all the methods.

System proxies are designed for general purpose caching. Such proxies
neither consider user access patterns nor content semantics. Also, it is hard to
do the user pattern and content semantic learning. All are installed with static
policies and so one cannot expect a very high degree of performance. Usually
the application proxies are developed to do many application-oriented tasks
(Brooks .S. M. C., 1995). By considering all these issues, in this present work,
an intelligent agent proxy has been developed for web digital libraries. It
considers the content semantics and user access pattern of learning.

2.4.3 Distributed Text Classification

Automated Text Document Classification is one of the long-standing

challenges (Liao .Y 2002, Guowei Zu 2003, Leah S 1996, Douglas Baker .L
 45

1998, David D.Lewis 1995, Ellen Riloff 1993, Kamal Nigam 1998, Andrew
McCallum 1999, HisnChun Chen 2002, Dimitris Meretakis 2000, David
Camacho 2001, Chandra Chekuri 1997, George Forman 2003, Chintan Patel
2003, Anne Kao 2003, Dou Shen 2004, Giuseppe Attardi 1999, Madhusudhan
Kongovi 2002, Thorsten Joachims 2001, Gerard Salton 1989, Neelamagam .A
2002a, Neelamagam .A 2002b, Neelamagam .A 2002c, Shian-Hua Lin 2002,
Ricardo Baeza-Yates 2004, Darrell Laham 2003, Roel Popping 2000, Guha.
R, 2004 , Stefan Decker 2000, William B. Frakes 1992, Klaus Krippendorff
2004, Behnak Yalaghian 2002, http://wordnet.princeton.edu, Y.Li,C.Zhang,
2003, Y.Li,C.Zhang, 1999) to the information and cognitive scientists. The
aim of this classification is to build an internal semantic hierarchy that allows
the user to search relevant documents either by browsing a topic hierarchy or
directly retrieving documents. This document classification is done using
different types of document inputs. There are varieties of such text
documents/contents, which are taken for classification such as title, abstract,
labeled documents, unlabeled documents etc. For example the title, abstract
and keywords of a research literature are taken for our present experiment.

There are two general basic principles for creating categories: cognitive
economy and perceived world structure (Thorsten Joachims 2001). The
principle of cognitive economy means that the function of categories is to
provide maximum information with the least cognitive effort. The principles
of perceived world structure means that the perceived world is not an
unstructured set of arbitrary or unpredictable attributes. The attributes that an
individual will perceive, and thus use for categorization, are determined by the
needs of an individual. These need to change as the physical and social
environment change. Psychologists agree that similarity plays a central role in
combining different items into a single category.
 46

Different methods are available for text document classification. These

methods are broadly classified into eight categories viz., Classic Information
Models (Ricardo Baeza-Yates 2004), Set Theoretic Models (Ricardo Baeza-
Yates 2004), Algebraic Models (Ricardo Baeza-Yates 2004), Probabilistic
Models (Ricardo Baeza-Yates 2004), Natural Language Processing Models
(Ricardo Baeza-Yates 2004), Link Analysis Models (Ricardo Baeza-Yates
2004), Statistical Models (Darrell Laham 2003) and Cognitive Models (Roel
Popping 2000, Guha. R, 2004, Stefan Decker 2000). The Classic Information
Models involve various methods such as Boolean model, vector space model
and probabilistic model. The Boolean model consists exactly two classes that
point out whether a document is relevant to the particular category or not. It is
also sometimes called Boolean set theoretic model. The vector model uses
non-binary weights to find the relativity of the document coming under a
certain category. This classification normally involves more than two classes.
That is the class label y which can assume l different values. These different
values give the closeness of the document with respect to the certain degree
put under a specific category. The error-rate applies to this vector model. The
Vector model ranks the documents based on their degree of similarity. Most
of the cases use term weighting scheme to measure the similarity. The third
model in this category is the probabilistic model. In this model the given
document di is decided as how close it is to the category ci or not based on the
probabilistic estimation. It is empirically proved in many experiments that the
Euclidean distance (K-NN) method gives the best result (Liao .Y 2002).

The Set Theoretic models (Ricardo Baeza-Yates 2004) are fuzzy set
model (David Camacho 2001, Ricardo Baeza-Yates 2004) and extended
Boolean model (Ricardo Baeza-Yates 2004). Fuzzy set model representing
documents with a set of keywords yields descriptions, which are only partially
related to the real semantic contents of the respective documents. This can be
 47

modeled by considering that each term defines a fuzzy set and that each
document has a degree of membership in this set. The key idea is to associate
a membership function with the elements of the class. Extended Boolean
model is the one form of vector with the functionality of partial matching and
term weighting. In algebraic models there are three categories. These are the
generalized vector space model (Liao .Y 2002, Chandra Chekuri 1997,
Thorsten Joachims 2001), the latent semantic indexing model (Darrell Laham
2003) and the neural network model (Andrew McCallum 1999, Ricardo
Baeza-Yates 2004). In case of vector space models, a vector associates an
index for every term. Independence of index terms in the vector model
implies that the sub-set of vectors is linearly independent and forms a basis for
the subspace of interest. The main idea in the latent semantic indexing model
is to map each test document and trained index into a lower dimensional
space, which is associated with the concepts. Instead of doing index
matching, the system does a concept matching. The neural network is used as
the tool for the best pattern matching and learning in most of the discussed
conventional information retrieval models.

The probabilistic models (Kamal Nigam 1998, Andrew McCallum

1999, HisnChun Chen 2002, Dimitris Meretakis 2000, Ricardo Baeza-Yates
2004) involve two categories, such as Bayesian Network and Belief Network
models, also called as the decision theoretic models. The Bayesian network
(Guowei Zu 2003, Kamal Nigam 1998, Andrew McCallum 1999, Dimitris
Meretakis 2000) provides a clear formalism for combining distinct sources of
evidence and that can be used to improve the classification process. Normally
it is represented as the directed acyclic graph (DAG) in which the nodes
represent random variables; the arcs portray causal relationship between
variables. The strength of these causal influences are expressed by conditional
probabilities. In case of belief network model, the user document is modeled
 48

as a network node to which it is associated with a binary random variable.

This binary random variable is set to 1 whenever a complete concept is
covered by the category space. Then the coverage of the concept space with
respect to the category space is computed. This is deemed to be the degree of
the belief associated with the concept space.

The natural language processing models (Ricardo Baeza-Yates 2004,

Roel Popping 2000) aim to lead the system to understand the concepts and
situations that exist around the document and then to classify the documents
under the specific hierarchy. This will also help the system to disambiguate
situations, which arise out of analysis and exactly identify the document with
respect to the specific category. If such a requirement arises it will also enable
the system to create the new category hierarchy. The process of NLP involves
morphological analysis, token identification, syntax analysis (parsing), phrase
identification, semantic identification, situation identification and
disambiguation. Humanly constructed dictionaries (Anne Kao 2003),
knowledge bases (Anne Kao 2003), semantic networks, etc for these processes
are used. WordNet (http://wordnet.princeton.edu) is an online lexical reference
system whose design is inspired by current psycholinguistic theories of human
lexical memory. English nouns, verbs, adjectives and adverbs are organized
into synonym sets, representing the underlying lexical concept. Different
relations link the synonym sets. But WordNet lacks relations between related
concepts. Another Use-case based (Ellen Riloff 1993) approach fed with
thousands of sentences used in documents gives a high relevant classification
of text. These types of approaches are also sometimes referred as Knowledge
Management or Knowledge Engineering approaches (Anne Kao 2003).

The link analysis models (Giuseppe Attardi 1999, Behnak Yalaghian

2002) are based on the assumption that a web page refers to a document must
 49

contain enough hints about its content to induce someone to read it. Such hints
are also sufficient to classify the document referred to. This kind of link
analysis is also used to classify the documents. But, the user navigation style
can not be always guaranteed. The concept of Semantic web is the long-
standing dream of the cognitive and information scientists (Chintan Patel
2003, Guha. R, 2004, Stefan Decker 2000). Cognitive modeling attempts are
made to design the best semantic network that gives the best inference. This
kind of network may give the best classification, if the whole document is
converted into such a network. One such an attempt was made at the Stanford
University through the TAP project (Guha. R, 2004). Such systems in the
internal concepts relationships are mapped through a completely connected
mesh network, creating a coherent semantic web from desperate chunks. But
the links are unmanageable. The practical implementation of such a system is
basically questionable. The Carnegie Millen University has made another
attempt to design such a semantic web. This is the well-known RDF data
model (Stefan Decker 2000). This method provides an aggregation at the data
model level. Higher-level differences between data sources sometimes make it
inappropriate to directly merge data from them. Also, a mechanism that
allows explicitly representing and reasoning with these assumptions and
differences has to be developed. Classification of document with such
ontology is highly difficult.

All the above-explained models are normally used along with the
conventional text information processing steps (Neelamagam .A 2002c, Shian-
Hua Lin 2002, Ricardo Baeza-Yates 2004, William B. Frakes 1992, Klaus
Krippendorff 2004). These text information-processing steps involve
tokenizing, stop word elimination, stemming, lemmatization, indexing,
ranking etc. After completing these steps the user will identify the terms and
then any one of the above said models will be applied to categorize the
 50

documents. In most of the cases, a combination of above models (Leah S

1996) is used in order to achieve the highly effective classification. Also,
optimizing one effective (David D.Lewis 1995) measure will not in general
optimize others. So, there is no single best system for any text classification
task. The choice among plausible alternatives should be made with a clear
goal in mind and one has to make this compromise to make the best design
decision. It has been empirically proved that the phrase reflects the concepts
as well as using the phrases for text processing increases the factor of
relevancy (Madhusudhan Kongovi 2002). Xinghua Fan et. al. (Xinghua Fan,
2005) is taken two steps Chinese Text classification using Bayesian approach
tries to fix the fuzzy area between the two categories. This approach makes
use of the distributional characteristics of misclassified documents which are
near to the separation line between two categories.

In IISc (Indian Institute of Science) various departments proposed to

develop distributed information servers to provide on-line research literature.
These servers are expected to add more and more literature over a period of
time. Some of these servers provide only a specific set of literature based on a
particular subject. Distributed Information Classification is the most important
problem to be solved in this distributed environment. The category of new
documents in a department server is unknown. Also, one cannot give any
guarantee for the type of the new document that certainly belongs to different
categories. Normally, it is necessary to replicate all sets of knowledge in all
servers. It increases the volume transfer of document across different servers.
Or else, in centralized server approach the traffic will increase if the number of
unknown documents arrival increases in different servers.

Shahram Rahimi and Norman F. Carver (Shahram Rahimi 2003) have

identified a suitable domain-specific multi-agent architecture to the distributed
 51

information integration. Similar to the way the current information sources are
independently constructed, information agents can be developed and
maintained separately. Every information agent contains ontology of its
domain and its expertise. Each concept matrix together with the ACM
classification represents the ontology. The ontology consists of descriptions of
objects and their relationship (noun, verb phrases). The model provides a
semantic description of the domain, which is used extensively for processing
query.

In order to solve the above said distribution information classification,

in this thesis a new hierarchical concept-matrix patterned multi-agent based
automated text classification method has been designed and developed for
Digital Libraries. It is a combination of knowledge engineering and statistical
approaches. The ACM Computing Review classification method is adapted for
this purpose and this ACM CR pre-specified category hierarchy defines a four
level hierarchy of nodes. It is able to learn and formulate the new concept-
matrix hierarchy over a period of time. The system uses the extracted technical
phrases for the formation of concept matrix. The ACM concept hierarchy
pattern also gives the semantic meaning at various levels. Every individual
domain specific agent in the multi-agent system is able to decide the relativity
of the document with respect to its own hierarchy. Presently the authors
submitting papers to ACM conferences and journals are responsible for
classifying documents and are doing it manually. Even though the present
experimentation is limited to ACM CR categories, later it may be adapted to
Science /Engineering or any other kind of literature using other (DDC)
classification schemes. The use of multi-agent system supports this scalability.
The Latent Semantic Analysis along with statistical correlation enables the
system to identify the interrelationship among the concept matrix.
 52

2.4.4 User-Adaptive Text Retrieval

E learning is the modern information technology innovation, which

plays a vital role in recent education and research development activities.
These e-learning methods involve various categories (Frontend.com 2001,
Scott Grissom 1998, http://www.imarkgroup.org/2005), which include
labware, course work, presentation, exposition, on-line course, assignment and
on-line research literature presentation etc. The formal high-quality on-line
literature presentation is the most important one to aid researchers of various
communities all over the world to collect various research literatures. Digital
library is the modern social (Scott Grissom 1998,
http://www.imarkgroup.org/2005, Lillian N. Cassel 2003) virtual institution
for information collection, presentation and dissemination and is instrumental
in providing various categories of e-learning services. The design and
development of digital library requires many issues to be addressed (Daniel
Andresen 1996, Dieter W. Fellner 2003, Scott Grissom 1998,
http://www.imarkgroup.org/2005, Lillian N. Cassel 2003) and it must be able
to perform intelligent human oriented tasks in order to make it highly
sophisticated and effective.

The on-line research literature collection is an e-learning activity in all

the research institutions. These are available in distributed locations. In the
Indian Institute of Science (IISc), different departments have proposed to
develop a research literature repository to provide on-line research literature to
various science/engineering researchers. This repository stores all the research
literatures viz. research papers, theses and technical reports submitted by
students and scholars. These literatures are classified and stored in different
department servers. In order to collect the literature in the specific topic or
area, say for example of computing literature the user has to explicitly go to
 53

the related department server and browse the information that is needed.
Instead the user expects a system that has to automatically search and
present/recommend the specific set of literature on the desktop from different
servers. This is called as an information integration problem (Shahram Rahimi
2003).

In this case the relevant literature collection from all these portals
together is the most important problem to be solved. Most of the retrieval
systems just retrieve the passive results (retrieve a set of articles) at the time of
searching. It is not able to retrieve the literature if some new literature has
been added on later. The main advantage of a recommendation system is that
it is able to actively recommend a set of newly added literature even after the
searching is over.

Some of the serious usability problems in e-learning and other web

digital libraries are (Frontend.com 2001) failure to relate the real – world
experience of the user, poor presentation of key information and lack of
accessibility even in the most basic sense. The concept oriented relevant text
information retrieval / recommendation is the long standing dream of the
information and cognitive scientist. The primary expectation is that if the user
presents any phrase or keyword, then the system must be able to identify the
related concepts also. That is the system must be able to find semantically
relevant documents even if it does not contain all the keywords in the queries.
The e-learning interface must be in a meaningful user-centric way in order to
improve the quality and to solve the above said problems. There are two
critical things very much important for retrieval / recommendation system.
The first one is based on the retrieval algorithm or methodology and the
second one is the user-interface design. In this present approach a LSA based
algorithm is used to support the relevant retrieval/recommendation. Secondly
 54

the user personalization (Monica Bonett 2001) is one way for the user-centric
computing and one of the important issues that assist fast, relevant and
economical retrieval of information in the federated or distributed digital
libraries. This user personalization involves a process of gathering user-
information during interaction with the user, which is then used to deliver
appropriate content services, tailored to meet the user’s needs. This
experience is used better to serve the customer by anticipating needs, making
the interaction efficient and satisfying both parties and to build a relationship
that encourage the customer to come again for subsequent operations. The
main difference between customization and personalization is that
customization occurs when the user can configure an interface and creates a
profile manually, adding and removing elements in the profile. In the process
of customization the user profile mapping is explicit and user driven. In other
words, in personalization the user is seen as being passive, or at least
somewhat less in control. It is the system’s responsibility to monitor, analyze
and react to behavior; for example, content offered can be based on tracking
and surfing decisions. A personalized service need not always be based on the
individual user behavior or user input.

The user personalization can be realized in many user profile models

(Gururaj .R 2004), and the user-personal agent (Daniela Godoy 2000, Michael
Fleming 1999) is an instrument in processing such user personal modeling.
The user profile model determines the human and computer process in
processing the information. These user profile models can be categorized as
Boolean-based models, Similarity-based models, Vector Space models,
Keyword Based models; Structure based models, X-path models, XML-QL,
X-Query, Rule-based models, Attribute-value etc. Interface agents (Daniela
Godoy 2000, Daniela Godoy 2000, Daniela Godoy 2003, Marcelo Armentano
2004), also known as personal agents are autonomous software entities that
 55

provide assistance to users. These agents act as human assistants,

collaborating with the user in the same work environment and becoming more
efficient as they learn about user interests, habits and preferences. Instead of
user-initiated interaction via commands and/or direct manipulation, the user is
engaged in a cooperative process in which both human and software agents
initiate communication, monitor events and perform tasks. Also, there are
many such agents (Daniela Godoy 2000 , Daniela Godoy 2000, Silvia
Schiaffino 2003, Michael Fleming 1999, Daniela Godoy 2003, Marcelo
Armentano 2004, Silvia Schiaffino 2004) developed in different environments.
An attempt has been made by Daniela Godoy and Analia Amandi to design
(Daniela Godoy 2000, Daniela Godoy 2003, Marcelo Armentano 2004) a
personal searcher using intelligent agents. Their extension of work involves a
user profiling architecture (Daniela Godoy 2003) for textual based agents. In
another attempt they developed a user association rule to learn user assistance
requirements. D. Cordero and his team members have developed an Intelligent
Agent for generating personal newspapers (Daniela Godoy 2000).

The main user-interface agent interaction personalization issues (Silvia

Schiaffino 2004) are (i) discovering the type of assistance each user wants, (ii)
learning the particular assistance requirements, (iii) acknowledging the fact
that the users have different contexts of analyzing users’ tolerance to agents’
errors, (iv) discovering when to interrupt the user, (v) discovering how much
control the user wants to delegate to the agent and proving the means to
provide simple explicit user feedback, (vi) providing the means to capture as
much implicit feedback, (vii) providing the means to control and inspect agent
behavior. Total personalization is not just to interact with the user to get some
feedback but to understand the user completely and accordingly establish
various sophisticated actions such as warnings or suggestions or actions
without the interference of the user.
 56

Shahram Rahimi and Norman F. Carver (Shahram Rahimi 2003) have

identified a suitable domain-specific multi-agent architecture to distributed
information integration. Similar to the way current information sources are
independent, information agents are developed and maintained separately. The
goal of each information agent is to provide information and expertise on a
specific topic by drawing relevant abstracted information from other
information agents or from servers. To reduce the level of information
processing, each agent is designed to provide expertise on a specific-topic by
drawing on relevant information from other information agents in the related
knowledge domains. Every information agent contains ontology of its domain
of its expertise, its domains model and its information source models. Each
concept matrix together with the ACM classification represents the ontology.
The ontology consists of descriptions of objects and relationships (noun, verb
phrases). The model provides a semantic description of the domain, which is
used extensively for processing query.

In this present work, an attempt is made to design and develop a

domain specific (subject specialist) multi-agent system to support the
retrieval/recommendation. This system is specially designed to aid the
researchers to collect the on-line articles from distributed web servers hosted
at different departments in the IISc. The domain specific agent is able to self-
proclaim the related research literature that the user is seeking about in the
specific area. Subsequently the same information is recommended to the user.
Also, the user interface agent is designed to understand the user in the best
way and is able to initiate different actions under various circumstances.

2.5 CHAPTER SUMMARY

In this chapter, the survey methodology is presented first, and then the
agent system classification is reviewed based on different criteria. Taking
 57

these criteria as the bases, new criteria have been proposed for agent
classification. An agent of a particular functionality coming under a specific
architecture may possess a set of characteristics. That is, an information agent
that is a functional agent may be designed using reactive architecture,
possessing characteristics such as mobility, communication and cooperation.
So, An agent may satisfy one or more of the above said classification criteria,
combined or interchangeably. Agent technology is an emerging field so that a
new technique based on functionality, characteristics or architecture will
emerge that may lead to a new method of classification in the future.

The tools used for development of agent-based systems are examined.

Following a review of the software tools, the specific problems taken into
account for our study in the present research have been presented. These
problems and solutions justify the suitability of agent paradigm in the current
context.