Tivoli Monitoring: UNIX Log Agent


Version 6.1.0

User’s Guide

SC32-9471-00
 Tivoli Monitoring: UNIX Log Agent
®


Version 6.1.0

User’s Guide

SC32-9471-00
 Note
Before using this information and the product it supports, read the information in Appendix F, “Notices,” on page 89.

First Edition (November 2005)

This edition applies to version 6.1 of the IBM Tivoli Monitoring: UNIX Log Agent (5724-C04) and to all subsequent
releases and modifications until otherwise indicated in new editions.
© Copyright International Business Machines Corporation 2005. All rights reserved.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
Contents
Tables . . . . . . . . . . . . . . . v Monitored Logs workspace . . . . . . . . 18
Typical scenarios . . . . . . . . . . . 18
About this guide . . . . . . . . . . vii
Who should read this guide . . . . . . . . . vii Chapter 5. Attributes reference . . . . 23
What this guide contains . . . . . . . . . . vii About attributes . . . . . . . . . . . . . 23
Publications . . . . . . . . . . . . . . viii More information about attributes . . . . . . . 23
Prerequisite publications . . . . . . . . . viii Attribute groups and attributes for the Monitoring
Related publications . . . . . . . . . . viii Agent for UNIX Logs . . . . . . . . . . . 24
Accessing terminology online . . . . . . . ix Log Entries attributes . . . . . . . . . . 24
Accessing publications online . . . . . . . ix Monitored Logs attributes . . . . . . . . 25
Ordering publications . . . . . . . . . . ix Disk capacity planning for historical data . . . . 26
Accessibility . . . . . . . . . . . . . . ix
Tivoli technical training . . . . . . . . . . . x Chapter 6. Situations reference . . . . 29
Support information . . . . . . . . . . . . x About situations . . . . . . . . . . . . . 29
Conventions used in this guide . . . . . . . . x More information about situations . . . . . . . 29
Typeface conventions . . . . . . . . . . x Predefined situations . . . . . . . . . . . 30
Operating system-dependent variables and paths xi HACMP_acquire_service_addr situation . . . . 30
HACMP_acquire_takeover_ addr situation . . . 31
Chapter 1. Overview of the Monitoring HACMP_config_too_long situation . . . . . 31
Agent for UNIX Logs . . . . . . . . . 1 HACMP_event_error situation . . . . . . . 31
IBM Tivoli Monitoring overview . . . . . . . . 1 HACMP_fail_standby situation . . . . . . . 31
Features of the Monitoring Agent for UNIX Logs . . 1 HACMP_get_disk_vg_fs situation . . . . . . 31
Monitoring Agent for UNIX Logs components . . . 2 HACMP_join_standby situation . . . . . . 32
User interface options . . . . . . . . . . . 3 HACMP_network_down situation . . . . . . 32
HACMP_network_down_complete situation . . 32
HACMP_network_up situation . . . . . . . 32
Chapter 2. Requirements and HACMP_network_up_complete situation . . . 32
configuration for the monitoring agent . 5 HACMP_node_down situation . . . . . . . 33
Requirements for the monitoring agent . . . . . 5 HACMP_node_down_complete situation . . . 33
Specifying the log files to monitor . . . . . . . 6 HACMP_node_down_local situation . . . . . 33
Customer configuration file . . . . . . . . 6 HACMP_node_down_local_complete situation . 33
Customer configuration file format . . . . . . 7 HACMP_node_down_remote situation . . . . 33
Syslog daemon configuration file . . . . . . 7 HACMP_node_down_rmt_complete situation . . 34
Environment variables for the Monitoring Agent for HACMP_node_up situation . . . . . . . . 34
UNIX Logs . . . . . . . . . . . . . . . 8 HACMP_node_up_complete situation . . . . 34
Environment variable syntax . . . . . . . . 9 HACMP_node_up_local situation . . . . . . 34
Dynamically refreshing the monitoring agent . . . 9 HACMP_node_up_local_complete situation . . 34
Sending a refresh signal to the monitoring agent . . 10 HACMP_node_up_remote situation . . . . . 35
Generic User Log Support (GULS) . . . . . . . 10 HACMP_node_up_remote_complete situation . . 35
HACMP_release_service_addr situation . . . . 35
Chapter 3. How to use the Monitoring HACMP_release_takeover_addr situation . . . 35
Agent for UNIX Logs . . . . . . . . . 11 HACMP_release_vg_fs situation . . . . . . 35
View real-time data about UNIX logs . . . . . . 11 HACMP_start_server situation . . . . . . . 36
Investigate an event . . . . . . . . . . . 12 HACMP_stop_server situation . . . . . . . 36
Recover the operation of a resource . . . . . . 12 HACMP_swap_adapter situation . . . . . . 36
Customize your monitoring environment . . . . 13 HACMP_swap_adapter_complete situation . . . 36
Monitor with custom situations that meet your UNIX_LAA_Bad_su_to_root_Warning situation 36
requirements . . . . . . . . . . . . . . 14 UNIX_LAA_Log_Size_Warning situation . . . 37
Collect and view historical data . . . . . . . 15
Chapter 7. Take Action commands
Chapter 4. Workspaces reference . . . 17 reference . . . . . . . . . . . . . . 39
About workspaces . . . . . . . . . . . . 17 About Take Action commands . . . . . . . . 39
More information about workspaces . . . . . . 17 More information about Take Action commands . . 39
Predefined workspaces . . . . . . . . . . 17 Predefined Take Action commands . . . . . . 39
Log Entries workspace . . . . . . . . . 17

© Copyright IBM Corp. 2005 iii

Chapter 8. Policies reference . . . . . 41 Trace logging . . . . . . . . . . . . . . 68
About policies . . . . . . . . . . . . . 41 Overview of log file management . . . . . . 68
More information about policies . . . . . . . 41 Examples of trace logging . . . . . . . . 69
Predefined policies . . . . . . . . . . . . 41 Principal trace log files . . . . . . . . . 69
Setting RAS trace parameters . . . . . . . 72
Appendix A. Generic user log support 43 Problems and workarounds . . . . . . . . . 73
Installation and configuration problem
Format command . . . . . . . . . . . . 43
determination . . . . . . . . . . . . 73
Example format command . . . . . . . . 43
Problem determination for remote deployment 78
Format comand syntax . . . . . . . . . 45
Situation problem determination . . . . . . 79
Support information . . . . . . . . . . . 83
Appendix B. Tuning format commands Searching knowledge bases . . . . . . . . 83
with the kulmapper utility. . . . . . . 61 Obtaining fixes . . . . . . . . . . . . 84
Using the kulmapper utility . . . . . . . . . 62 Receiving weekly support updates . . . . . 84
Analyzing User log files and testing format Contacting IBM Software Support . . . . . . 84
commands . . . . . . . . . . . . . . . 63
Appendix E. Accessibility . . . . . . . 87
Appendix C. IBM Tivoli Enterprise Navigating the interface using the keyboard . . . 87
Console event mapping . . . . . . . 65 Magnifying what is displayed on the screen . . . 87

Appendix D. Problem determination . . 67 Appendix F. Notices . . . . . . . . . 89

Gathering product information for IBM Software Trademarks . . . . . . . . . . . . . . 91
Support . . . . . . . . . . . . . . . 67
Built-in problem determination features . . . . . 67 Index . . . . . . . . . . . . . . . 93
Problem classification . . . . . . . . . . . 68

iv IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Tables
1. Monitoring agent requirements . . . . . . 5 18. Integer family data types . . . . . . . . 55
2. Monitoring Agent for UNIX Logs customer 19. Floating point family data types . . . . . 55
configuration file format . . . . . . . . 7 20. Escape character sequence . . . . . . . 56
3. Monitoring Agent for UNIX Logs environment 21. Overview of event slots to event classes 65
variables (ul.ini file) . . . . . . . . . . 8 22. Information to gather before contacting IBM
4. Viewing real-time data about UNIX logs 11 Software Support . . . . . . . . . . 67
5. Investigating an event . . . . . . . . . 12 23. Trace log files for troubleshooting agents 70
6. Recovering the operation of a resource . . . 13 24. Problems and solutions for installation and
7. Customizing your monitoring environment 13 configuration . . . . . . . . . . . . 75
8. Monitoring with custom situations . . . . . 15 25. General problems and solutions for
9. Collecting and viewing historical data . . . 16 uninstallation . . . . . . . . . . . . 77
10. Log entry fields and their descriptions . . . 20 26. Remote deployment problems and solutions 79
11. Sample Log Entry workspace . . . . . . 21 27. Specific situation problems and solutions 79
12. Capacity planning for historical data . . . . 27 28. Performance Impact by attribute group 80
13. Monitoring Agent for UNIX Logs valid size 29. Problems with configuring situations that you
option and data type combinations. . . . . 47 solve in the Situation Editor . . . . . . . 81
14. Monitoring Agent for UNIX Logs valid 30. Problems with configuration of situations that
alphanumeric data types . . . . . . . . 48 you solve in the Workspace area . . . . . 82
15. Log entries table view column mapping names 50 31. Problems with configuration of situations that
16. Log Entries table view mapping options 52 you solve in the Manage Tivoli Enterprise
17. Mapping precision and the data types Monitoring Services window . . . . . . . 82
specified . . . . . . . . . . . . . 54

© Copyright IBM Corp. 2005 v

vi IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
About this guide
The IBM Tivoli Monitoring: UNIX Log Agent User’s Guide provides information
about using the IBM® Tivoli® Monitoring: UNIX® Log Agent.

Use the requirements and configuration chapter in this guide along with the IBM
Tivoli Monitoring Installation and Setup Guide to install and set up the software.

Use the information in this guide along with the IBM Tivoli Monitoring User’s Guide
to monitor UNIX (and Linux®) logs.

Who should read this guide

This guide is for system administrators who install and use the Monitoring Agent
for UNIX Logs to monitor and manage UNIX log resources.

Readers should be familiar with the following topics:

v Tivoli Enterprise™ Portal interface
v IBM Tivoli Monitoring application software
v IBM Tivoli Enterprise Console® (optional)
v UNIX logs
v AIX® operating systems
v Solaris operating environments
v HP-UX operating systems
v Linux operating systems

What this guide contains

This guide contains the following chapters:
v Chapter 1, “Overview of the Monitoring Agent for UNIX Logs,” on page 1
Provides an introduction to the Monitoring Agent for UNIX Logs.
v Chapter 2, “Requirements and configuration for the monitoring agent,” on page
5
Provides requirements and configuration information specific to the Monitoring
Agent for UNIX Logs.
v Chapter 3, “How to use the Monitoring Agent for UNIX Logs,” on page 11
Provides a list of tasks to perform when using the monitoring agent, a list of
procedures for completing each task, and references for where to find
information about the procedures. After completing installation and
configuration and becoming familiar with the information in Chapter 1 of this
guide, use this chapter to see how you can use the monitoring agent.
v Chapter 4, “Workspaces reference,” on page 17
Provides an overview of workspaces, references to additional information about
workspaces, and descriptions of predefined workspaces in this monitoring agent.
v Chapter 5, “Attributes reference,” on page 23
Provides an overview of attributes, references to additional information about
attributes, descriptions of the attribute groups and attributes in this monitoring
agent, and disk space requirements for historical data.

© Copyright IBM Corp. 2005 vii

 v Chapter 6, “Situations reference,” on page 29
Provides an overview of situations, references to additional information about
situations, and descriptions of the predefined situations in this monitoring agent.
v Chapter 7, “Take Action commands reference,” on page 39
Provides detailed information about the Take Action commands, references to
additional information about Take Action commands, and descriptions of the
Take Action commands provided in this monitoring agent.
v Chapter 8, “Policies reference,” on page 41
Provides an overview of policies, references for detailed information about
policies, and descriptions of the predefined policies included in this monitoring
agent.
v Appendix A, “Generic user log support,” on page 43
Provides generic user log support for this monitoring agent.
v Appendix B, “Tuning format commands with the kulmapper utility,” on page 61
Provides information about the kulmapper utility.
v Appendix C, “IBM Tivoli Enterprise Console event mapping,” on page 65
Provides an overview of the IBM Tivoli Enterprise Console event mapping
information for this monitoring agent.
v Appendix D, “Problem determination,” on page 67
Provides information about troubleshooting the various components of the
Monitoring Agent for UNIX Logs, information about log files and messages, and
information about your options for obtaining software support.
v Appendix E, “Accessibility,” on page 87
Provides information about the accessibility features in the Monitoring Agent for
UNIX Logs.
v Appendix F, “Notices,” on page 89
Provides IBM and Tivoli notices and trademark information as it applies to the
Monitoring Agent for UNIX Logs.

Publications
This section lists publications relevant to the use of the Monitoring Agent for UNIX
Logs. It also describes how to access these publications online and how to order
these publications.

Prerequisite publications
To use the information in this guide effectively, you must have some knowledge of
IBM Tivoli Monitoring products, which you can obtain from the following
documentation:
v IBM Tivoli Monitoring Administrator’s Guide
v IBM Tivoli Monitoring Installation and Setup Guide
v IBM Tivoli Monitoring Problem Determination Guide
v IBM Tivoli Monitoring: Upgrading from Tivoli Distributed Monitoring
v IBM Tivoli Monitoring User’s Guide
v Introducing IBM Tivoli Monitoring Version 6.1

Related publications
The following documents also provide useful information:
v IBM Tivoli Enterprise Console Adapters Guide

viii IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 v IBM Tivoli Enterprise Console Event Integration Facility User’s Guide
v IBM Tivoli Enterprise Console Reference Manual
v IBM Tivoli Enterprise Console Rule Developer’s Guide

Accessing terminology online

The Tivoli Software Glossary includes definitions for many of the technical terms
related to Tivoli software. The Tivoli Software Glossary is available at the following
Tivoli software library Web site:

http://publib.boulder.ibm.com/tividd/glossary/tivoliglossarymst.htm

The IBM Terminology Web site consolidates the terminology from IBM product
libraries in one convenient location. You can access the Terminology Web site at the
following Web address:

http://www.ibm.com/ibm/terminology

Accessing publications online

IBM posts publications for this and all other Tivoli products, as they become
available and whenever they are updated, to the Tivoli software information center
Web site. Access the Tivoli software information center by first going to the Tivoli
software library at the following Web address:

http://www.ibm.com/software/tivoli/library

Scroll down and click the Product manuals link. In the Tivoli Technical Product
Documents Alphabetical Listing window, click M to access all of the IBM Tivoli
Monitoring product manuals.

Note: If you print PDF documents on other than letter-sized paper, set the option
in the File → Print window that allows Adobe Reader to print letter-sized
pages on your local paper.

Ordering publications
You can order many Tivoli publications online at the following Web site:

http://www.elink.ibmlink.ibm.com/public/applications/
publications/cgibin/pbi.cgi

You can also order by telephone by calling one of these numbers:

v In the United States: 800-879-2755
v In Canada: 800-426-4968

In other countries, contact your software account representative to order Tivoli

publications.

Accessibility
Accessibility features help users with a physical disability, such as restricted
mobility or limited vision, to use software products successfully. With this product,
you can use assistive technologies to hear and navigate the interface. You can also
use the keyboard instead of the mouse to operate most features of the graphical
user interface.

About this guide ix

 For additional information, see Appendix E, “Accessibility,” on page 87.

Tivoli technical training

For Tivoli technical training information, refer to the following IBM Tivoli
Education Web site:

http://www.ibm.com/software/tivoli/education/

Support information
“Support information” on page 83 describes the following options for obtaining
support for IBM products:
v “Searching knowledge bases” on page 83
v “Obtaining fixes” on page 84
v “Contacting IBM Software Support” on page 84

Conventions used in this guide

Unless otherwise noted, general references in this document to UNIX logs also
apply to Linux logs.

This guide uses several conventions for special terms and actions, and operating
system-dependent commands and paths.

Typeface conventions
This guide uses the following typeface conventions:
Bold
v Lowercase commands and mixed case commands that are otherwise
difficult to distinguish from surrounding text
v Interface controls (check boxes, push buttons, radio buttons, spin
buttons, fields, folders, icons, list boxes, items inside list boxes,
multicolumn lists, containers, menu choices, menu names, tabs, property
sheets), labels (such as Tip:, and Operating system considerations:)
v Keywords and parameters in text
Italic
v Words defined in text
v Emphasis of words
v New terms in text (except in a definition list)
v Variables and values you must provide
Monospace
v Examples and code examples
v File names, programming keywords, and other elements that are difficult
to distinguish from surrounding text
v Message text and prompts addressed to the user
v Text that the user must type
v Values for arguments or command options

x IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Operating system-dependent variables and paths
The direction of the slash for directory paths might vary in this documentation. No
matter which type of slash you see in the documentation, if using UNIX, use a
forward slash (/).

The names of environment variables are not always the same in Windows® and
UNIX. For example, %TEMP% in Windows is equivalent to $TMPDIR in UNIX.

For environment variables, if using UNIX, use $variable.

About this guide xi

xii IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Chapter 1. Overview of the Monitoring Agent for UNIX Logs
The Monitoring Agent for UNIX Logs provides you with the capability to monitor
UNIX logs and to perform basic actions with UNIX logs. This chapter provides a
description of the features, components, and interface options for the Monitoring
Agent for UNIX Logs.

IBM Tivoli Monitoring overview

IBM Tivoli Monitoring is the base software for the Monitoring Agent for UNIX
Logs. IBM Tivoli Monitoring provides a way to monitor the availability and
performance of all the systems in your enterprise from one or several designated
workstations. It also provides useful historical data that you can use to track trends
and to troubleshoot system problems.

You can use IBM Tivoli Monitoring to do the following:

v Monitor for alerts on the systems that you are managing by using predefined
situations or custom situations.
v Establish your own performance thresholds.
v Trace the causes leading to an alert.
v Gather comprehensive data about system conditions.
v Use policies to perform actions, schedule work, and automate manual tasks.

The Tivoli Enterprise Portal is the interface for IBM Tivoli Monitoring products. By
providing a consolidated view of your environment, the Tivoli Enterprise Portal
permits you to monitor and resolve performance issues throughout the enterprise.

See the IBM Tivoli Monitoring publications listed in “Prerequisite publications” on

page viii for complete information about IBM Tivoli Monitoring and the Tivoli
Enterprise Portal.

Features of the Monitoring Agent for UNIX Logs

On a typical UNIX system, many log files are scattered throughout the file system.
The kernel, various utilities, and user applications create these logs to alert an
administrator to events such as security violations and software or hardware
failures.

As the number of computers that a system administrator oversees increases, the

task of managing these logs and utilizing the information they contain becomes
increasingly difficult. Additionally, no standardized format exists for UNIX log
files; therefore there is no simple way to analyze the data.

Each site handles log management differently and can adopt a strategy that falls
between two extremes:
v Discard all log data immediately
v Store all log data indefinitely

While the first choice conserves disk space and the second allows problems to be
diagnosed at a later time, neither strategy allows you to anticipate problems or
respond to them in a timely manner.

© Copyright IBM Corp. 2005 1

 The Monitoring Agent for UNIX Logs allows you to manage and utilize log files
more effectively.
v You can create situations that fire when specific messages are written to a log so
that you can take a more proactive approach to managing the systems for which
you are responsible. This means you can respond to events as soon as they occur
and take action to prevent potential problems from developing.
v Because the Monitoring Agent for UNIX Logs screens all log entries forwarding
only selected entries to the Tivoli Enterprise Portal, it eliminates the need to
manually analyze large log files.
v By shifting the emphasis of management from post-mortem diagnosis to
real-time response, the monitoring agent allows you to increase the amount of
log data collected by system daemons and user applications while decreasing
the amount of data archived and stored for historical debugging and analysis.
v You can easily retrieve log entries that occurred within a certain time span from
any monitored log. Data from different log types can be presented in a common
format within a Tivoli Enterprise Portal workspace.

The Monitoring Agent for UNIX Logs monitors and provides reports for the
following types of logs:
v Syslogs
v Utmp style logs
v Errlogs (AIX platforms only)
v User-defined ASCII logs

User-defined ASCII logs are supported through the Generic User Log Support
(GULS) feature. GULS requires that you supply a format command in the
configuration file that describes a log’s format to the monitoring agent. See
Appendix A, “Generic user log support,” on page 43 for further details.

Monitoring Agent for UNIX Logs components

After you install and set up the Monitoring Agent for UNIX Logs, you have an
environment that contains the client, server, and monitoring agent implementation
for IBM Tivoli Monitoring that contains the following components:
v Tivoli Enterprise Portal client with a Java-based user interface for viewing and
monitoring your enterprise.
v Tivoli Enterprise Portal Server that is placed between the client and the Tivoli
Enterprise Monitoring Server and enables retrieval, manipulation, and analysis
of data from the monitoring agents.
v Tivoli Enterprise Monitoring Server, which acts as a collection and control point
for alerts received from the monitoring agents, and collects their performance
and availability data.
v Monitoring Agent for UNIX Logs installed on the systems or subsystems that
you want to monitor. This monitoring agent collects and distributes data to a
Tivoli Enterprise Portal Server.

For both of the IBM Tivoli Monitoring environments (IBM Tivoli Monitoring 5.x
and IBM Tivoli Monitoring 6.1), IBM Tivoli Enterprise Console is an optional
component, which acts as a central collection point for events from a variety of
sources, including those from other Tivoli software applications, Tivoli partner
applications, custom applications, network management platforms, and relational

2 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 database systems. You can view these events through the Tivoli Enterprise Portal,
and you can forward events from IBM Tivoli Monitoring situations to the IBM
Tivoli Enterprise Console component.

User interface options

Installation of the base software and other integrated applications provides the
following interfaces that you can use to work with your resources and data:
Tivoli Enterprise Portal browser client interface
The browser interface is automatically installed with Tivoli Enterprise
Portal. To start Tivoli Enterprise Portal in your Internet browser, enter the
URL for a specific Tivoli Enterprise Portal browser client installed on your
Web server.
Tivoli Enterprise Portal desktop client interface
The desktop interface is a Java-based graphical user interface (GUI) on a
Windows or a Linux workstation.
IBM Tivoli Enterprise Console
Event management application
Manage Tivoli Enterprise Monitoring Services window
The window for the Manage Tivoli Enterprise Monitoring Services utility is
used for configuring the monitoring agent and starting Tivoli services not
already designated to start automatically.

Chapter 1. Overview of the Monitoring Agent for UNIX Logs 3

4 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Chapter 2. Requirements and configuration for the monitoring
agent
This chapter contains information about the following topics and procedures
relevant to the installation and configuration of the Monitoring Agent for UNIX
Logs:
v “Requirements for the monitoring agent”
v “Specifying the log files to monitor” on page 6
v “Environment variables for the Monitoring Agent for UNIX Logs” on page 8
v “Dynamically refreshing the monitoring agent” on page 9
v “Sending a refresh signal to the monitoring agent” on page 10
v “Generic User Log Support (GULS)” on page 10

Requirements for the monitoring agent

In addition to the requirements described in the IBM Tivoli Monitoring Installation
and Setup Guide, the Monitoring Agent for UNIX Logs has the requirements listed
in Table 1.
Table 1. Monitoring agent requirements
Operating system UNIX
Operating system versions v AIX, v5.1, 5.2, 5.3 (32-bit or 64-bit)
v HP-UX, v11i (32-bit or 64-bit)
v Solaris, v8, 9, 10 (32-bit or 64-bit)
v Linux:
– Linux on zSeries
- RHEL AS 3.0 (31-bit or 64-bit)
- RHEL AS 4.0 (31-bit or 64-bit)
- SLES 8 (31-bit or 64-bit)
- SLES 9 (31-bit or 64-bit)
– Linux on Intel
- RHEL AS/ES 2.1
- RHEL AS/ES 3.0
- RHEL AS/ES 4.0
- SLES 8
- SLES 9
The Linux version must support the Korn shell (ksh)
and Motif Window Manager (libmotif) for installation
of the monitoring agent.
v A POSIX-compliant threads package must be installed
on the monitored system.
Memory v 128 MB RAM at a minumum, 512 MB or higher for
better performance
Disk space v 30 MB of disk space (100 MB for Linux)
v Historical data disk space: see “Disk capacity planning
for historical data” on page 26

© Copyright IBM Corp. 2005 5

 Table 1. Monitoring agent requirements (continued)
Operating system UNIX
Other requirements v TCP/IP
v The monitoring agent must have the permissions
necessary to perform requested actions. For example, if
the user ID you used to log onto the system to install
the monitoring agent (locally or remotely) does not
have the permission to perform a particular action
being monitored by the monitoring agent (such as
running a particular command or reading a monitored
log file), the monitoring agent will be unable to
perform the requested action.

Specifying the log files to monitor

The runtime environment and behavior of the Monitoring Agent for UNIX Logs is
controlled through both a configuration file and environment variables. The
configuration file indicates which files the monitoring agent is to monitor.

When the monitoring agent starts, it looks at two files to determine which logs to
monitor:
v Customer configuration file
v Syslog daemon configuration file

If, for any reason, the monitoring agent is unable to find at least one log to
monitor from either file, it writes a message to the RAS log that contains the text
Agent has no work to do. Exiting... and then automatically terminates.

Customer configuration file

The monitoring agent first looks for a customer configuration file. The absolute or
relative file name of this customer configuration file is specified in the monitoring
agent’s ul.ini file through the KUL_CONFIG_FILE environment variable. (More
information about the ul.ini file is provided in “Environment variables for the
Monitoring Agent for UNIX Logs” on page 8.) The customer configuration file
contains a line for each log to be monitored that includes the absolute file name of
the log along with the log’s type. If the monitoring agent is able to start
monitoring at least one of the logs specified in the customer configuration file, it
will not interrogate the syslog daemon configuration file.

A default customer configuration file is shipped with the product and is called
kul_configfile. This file is installed into the install_dir/config directory. All entries
in the default file are commented out.

Note: All references to install_dir refer to the destination directory that was
specified when the monitoring agent was installed.

Important! If the Monitoring Agent for UNIX Logs is installed on top of a previous
version of the agent (the OMEGAMON® agent), the file
install_dir/config/kul_configfile will be replaced. Rename the file or copy it to
another location before installing version 6.1 of the monitoring agent.

6 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Customer configuration file format
Each entry in the customer configuration file consists of a single line with the
following fields, which must occur in the order given. Each field is delimited by
one or more space and/or tab characters and all fields except the first must be
preceded by a semicolon (;). There must be no white space between the semicolon
and the first character of the field it delimits.
Table 2. Monitoring Agent for UNIX Logs customer configuration file format
Field Description
1 Absolute file name of monitored log.
2 Debug mode (optional: default = N)

N = debug mode off; Y = debug mode on

If debug mode is on, each entry written to the monitored log will be
recorded in a debug log. In addition, the formatted entry that is passed to a
situation is also written to the debug log. All logs that are monitored in
debug mode write to the same debug log.

The debug log is specified in the monitoring agent ul.ini file using the
AGENT_DEBUG_LOG environment variable. If this variable is undefined or
the log cannot be opened, no debug logging occurs. Each time the
monitoring agent is started, new events will be appended to the end of the
existing debug log.
3 Log type (optional: default = S)
v S = syslog
v E = errlog
v A = utmp log
v U = user-defined log
4 Format command. This command is valid only for type ‘E’ (errlog) and type
‘U’ (user-defined) logs.

For type ‘E’ logs, the format command must consist of a an errpt command
that includes the ‘-c’ (concurrent mode) option. The default value is:
errpt -c -smmddhhmmyy

For user-defined logs, the format command describes both the format of the
log and how data will be mapped and formatted in the Tivoli Enterprise
Portal Log Entries table view. There is no default.

For additional information on composing format commands, refer “Format

command” on page 43.

Syslog daemon configuration file

Kernel daemons and user applications use the UNIX syslog facility to record
messages in a log. By using the syslog facility an application ensures that its log
entries conform to a standard format.

The actual logging activities are performed by the syslog daemon, syslogd, which
is controlled through a configuration file usually called /etc/syslog.conf. This file
is usually maintained by the system administrator.

The syslog.conf file is used to indicate to which syslog messages are to be written
that have a given severity and that originate from a given application. This allows

Chapter 2. Requirements and configuration for the monitoring agent 7

 you to consolidate messages from more than one source into a single log file.
Through the syslog facility, you can also direct messages to be written to a
different system, known as the loghost.

The monitoring agent will attempt to build a default list of logs to monitor from
the syslog daemon configuration file under the following circumstances:
v The KUL_CONFIG_FILE environment variable is undefined.
v The specified customer configuration file does not exist or cannot be opened.
v There are no log names in the customer configuration file.
v None of the logs contained in the customer configuration file are valid.

The file that the monitoring agent reads to build the default monitored logs list is
called /etc/syslog.conf, but this can be overridden using the KUL_SYSLOG_CONF
environment variable.

If you are interested only in monitoring syslogs, you can omit the
KUL_CONFIG_FILE environment variable from the ul.ini file, or you can leave the
variable unassigned, thereby letting the monitoring agent determine which syslogs
are active on each system based on the syslogd configuration file.

Environment variables for the Monitoring Agent for UNIX Logs

Environment variables are specified in the monitoring agent’s ul.ini file and allow
you to communicate to the agent information such as the name of your customer
configuration file. The location of the agent’s ul.ini file is: install_dir/config/ul.ini.
The table describes some of the variables you can include.
Table 3. Monitoring Agent for UNIX Logs environment variables (ul.ini file)
Variable name Purpose
KUL_CONFIG_FILE The absolute or relative file name of the Monitoring Agent
for UNIX Logs customer configuration file. The default
value is:
install_dir/config/kul_configfile
KUL_SYSLOG_CONF The absolute or relative file name of the syslog daemon
configuration file. The default name is:
/etc/syslog.conf
AGENT_DEBUG_LOG The absolute or relative file name of the Monitoring Agent
for UNIX Logs debug log. This file is used to record pre-
and post-formatted images of each entry written to a log
that is being monitored in debug mode. The default value
is:
install_dir/logs/ul_debug.log
KUL_MAX_ROWS The maximum number of rows to be returned by the
Monitoring Agent for UNIX Logs for a Log Entries table
view request. The default value is 1000.

8 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 Table 3. Monitoring Agent for UNIX Logs environment variables (ul.ini file) (continued)
Variable name Purpose
KBB_RAS1 Specifies which trace entries to include in the monitoring
agent’s runtime log. The syntax is:
classes (COMP:component classes) (UNIT:unit classes)

A class specified outside of a parenthesis is global, that is, it

applies to all components and units. Valid classes are:
v ERROR
v FLOW
v STATE
v DETAIL
v ALL

Including one or more classes within parentheses includes

entries of the class that are generated by the associated
component or unit. A useful component to trace is
“MANAGER”; tracing units of “kul” and/or “kra” can also
be informative.
CTIRA_HOSTNAME Overrides the name with which the monitoring agent
identifies itself to the server. The default is the computer
name.
CTIRA_NODETYPE Overrides the default suffix that is appended to the host
name to differentiate the Monitoring Agent for UNIX Logs
from another UNIX monitoring agent running on the same
computer. The default is KUL.

Environment variable syntax

The syntax for defining an environmental variable depends on the shell used to
interpret the script. In the Bourne and Korn shells, the following defines the
variable “VAR” assigning to it the value “VALUE” and makes it available to other
programs invoked subsequently in the same script:
VAR=VALUE; export VAR

If you are using the C shell, the following command would produce the same
result:
setenv VAR VALUE

Examples

If you are using the Bourne or Korn shells, use the following commands to assign
a value of install_dir/config/myconfig to the KUL_CONFIG_FILE variable.
KUL_CONFIG_FILE=install_dir/config/myconfig
export KUL_CONFIG_FILE

If you’re using the C shell, use the following command:

setenv KUL_CONFIG_FILE install_dir/config/myconfig

Dynamically refreshing the monitoring agent

After the monitoring agent starts, you can dynamically change the list of logs
being monitored on managed systems; it is not necessary to stop and restart the
monitoring agent.

Chapter 2. Requirements and configuration for the monitoring agent 9

 In addition, if one or more monitors were either unable to start, or terminated
abnormally, they can be restarted without the need to restart the monitoring agent.
In both cases, it is only necessary to send the monitoring agent a refresh signal.

Sending a refresh signal to the monitoring agent

Use the following procedure to dynamically change the list of logs being
monitored on a managed system, or to restart individual monitors.
1. Start a Telnet session or other remote login procedure to the managed system
on which you want to change the monitored logs list.
2. Modify the customer configuration file (if you have specified the
KUL_CONFIG_FILE environment variable) or the syslog daemon configuration
file.
3. Send the monitoring agent a refresh signal:
kill -HUP agentPID
4. Open the Monitored Logs table view for the appropriate managed system. The
result is that monitoring has begun for logs that were added to the
configuration file, and stopped for logs that were deleted from the
configuration file.

Note: A refresh occurs only if the monitoring agent determines that the
configuration file has been modified since the agent was started, or since the
previous refresh. If you have not modified the configuration file, but want to
restart a monitor, change the modification date of the configuration file prior
to sending a refresh signal.

To change the modification date of the configuration file prior to sending a signal,
issue the following command from the install_dir/config directory on the managed
system where the monitoring agent is running:
touch kul_configfile

Generic User Log Support (GULS)

The Monitoring Agent for UNIX Logs has built into it the ability to monitor three
types of standard UNIX logs: syslogs, errlogs, and utmp logs. To monitor a log of
one of these three types, it is only necessary to specify the name and type of the
log in the configuration file because the monitoring agent already understands
how to interpret the data within each log record and map it into the Log Entries
table view.

To monitor an ASCII log that does not conform to any of the three supported
types, see Appendix A, “Generic user log support,” on page 43.

10 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Chapter 3. How to use the Monitoring Agent for UNIX Logs
After you have installed and configured the Monitoring Agent for UNIX Logs, you
can begin using this monitoring agent to monitor your resources.

This chapter provides information about how to use the Monitoring Agent for
UNIX Logs to perform the following tasks:
v “View real-time data about UNIX logs”
v “Investigate an event” on page 12
v “Recover the operation of a resource” on page 12
v “Customize your monitoring environment” on page 13
v “Monitor with custom situations that meet your requirements” on page 14
v “Collect and view historical data” on page 15

For each of these tasks, there is a list of procedures that you perform to complete
the task. For the procedures, there is a cross-reference to where you can find
information about performing that procedure. Information about the procedures is
located in subsequent chapters and appendixes of this user’s guide and in the IBM
Tivoli Monitoring documentation.

View real-time data about UNIX logs

After you install, configure, and start the Monitoring Agent for UNIX Logs,
monitoring starts.

Table 4 contains a list of the procedures for viewing the real-time data about UNIX
logs that the monitoring agent collects through the predefined situations. The table
also contains a cross-reference to where you can find information about each
procedure.
Table 4. Viewing real-time data about UNIX logs
Procedure Where to find information
View the hierarchy of your monitored IBM Tivoli Monitoring User’s Guide:
resources from a system point of view ″Navigating through workspaces″ (in
(Navigator view organized by operating ″Monitoring: real-time and event-based″
platform, system type, monitoring agents, chapter)
and attribute groups).
View the indicators of real or potential
problems with the monitored resources
(Navigator view).
View changes in the status of the resources IBM Tivoli Monitoring User’s Guide: ″Using
that are being monitored (Enterprise workspaces″ (in ″Monitoring: real-time and
Message Log view). event-based″ chapter)
View the status of the agents in the Chapter 4, “Workspaces reference,” on page
managed enterprise that you are monitoring 17 in this guide
(Monitoring Agent Status view).

© Copyright IBM Corp. 2005 11

 Table 4. Viewing real-time data about UNIX logs (continued)
Procedure Where to find information
View the number of times an event has been IBM Tivoli Monitoring User’s Guide: ″Using
opened for a situation during the past 24 workspaces″ (in ″Monitoring: real-time and
hours (Open Situations Count view). event-based″ chapter)

Chapter 4, “Workspaces reference,” on page

17 in this guide

Chapter 6, “Situations reference,” on page 29

in this guide
Manipulate the views in a workspace. IBM Tivoli Monitoring User’s Guide: ″Using
views″ (in ″Monitoring: real-time and
event-based″ chapter)

Investigate an event
When the conditions of a situation have been met, an event indicator is displayed
in the Navigator. When an event occurs, you want to obtain information about that
event so you can correct the conditions and keep your enterprise running
smoothly. The situation must be associated with a Navigator Item in order to
appear.

Table 5 contains a list of the procedures for investigating an event and a

cross-reference to where you can find information about each procedure.
Table 5. Investigating an event
Procedure Where to find information
Determine which situation raised the event IBM Tivoli Monitoring User’s Guide: ″Opening
and identify the attributes that have values the situation event workspace″ (in
that are contributing to the alert. ″Monitoring: real-time and event-based″
chapter, ″Responding to alerts″ section)
Review available advice. Chapter 4, “Workspaces reference,” on page
17 in this guide
Notify other users that you have taken IBM Tivoli Monitoring User’s Guide:
ownership of the problem related to an ″Acknowledging an event″ (in ″Monitoring:
event and are working on it. real-time and event-based″ chapter,
″Responding to alerts″ section)
Remove the event from the Navigator. IBM Tivoli Monitoring User’s Guide: ″Closing
the situation event workspace″ (in
″Monitoring: real-time and event-based″
chapter, ″Responding to alerts″ section)

Recover the operation of a resource

When you find out that a resource is not operating as desired, you can control it
manually or automatically using Take Action commands.

Table 6 on page 13 contains a list of the procedures for recovering the operation of
a resource and a cross-reference to where you can find information about each
procedure.

12 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 Table 6. Recovering the operation of a resource
Procedure Where to find information
Take an action on a resource manually. IBM Tivoli Monitoring User’s Guide:
v ″Other views″ (in ″Custom workspaces″
chapter, ″Workspace views″ section)
v ″Take action – Reflex automation″ (in
Situations for event-based monitoring″
chapter, ″Event-based monitoring
overview″ section)

Chapter 7, “Take Action commands

reference,” on page 39 in this guide
Take an action on a system condition IBM Tivoli Monitoring User’s Guide:
automatically by setting up a situation to ″Situations for event-based monitoring″
run a Take Action command. chapter
v ″Customizing a situation″
v ″Creating a situation″
v ″Specify an action to take″
v ″Distribute the situation″

Chapter 7, “Take Action commands

reference,” on page 39 in this guide
Take multiple actions on system conditions IBM Tivoli Monitoring User’s Guide: ″Policies
automatically using a policy. for automation″ chapter
v ″Creating a policy″
v ″Maintaining policies″
Take actions across systems, monitoring v ″Workflows window″
agents, or computers using a policy.
Chapter 8, “Policies reference,” on page 41 in
this guide

Customize your monitoring environment

You can change how your monitoring environment looks by creating new
workspaces with one or more views in it.

Table 7 contains a list of the procedures for customizing your monitoring

environment and a cross-reference to where you can find information about each
procedure.
Table 7. Customizing your monitoring environment
Procedure Where to find information
Display data in tables or charts (views) in IBM Tivoli Monitoring User’s Guide:
Tivoli Enterprise Portal. v ″Custom workspaces″
v ″Table and chart views″
Display an overview of changes in the status IBM Tivoli Monitoring User’s Guide: ″Message
of the situations for your monitored log view″ (in ″Event views: message log,
resources (Message Log View). situation event console, graphic, and Tivoli
Enterprise Console″ chapter)

Chapter 3. How to use the Monitoring Agent for UNIX Logs 13

 Table 7. Customizing your monitoring environment (continued)
Procedure Where to find information
Specify which attributes to retrieve for a IBM Tivoli Monitoring User’s Guide: ″Creating
table or chart so you can retrieve only the custom queries″ (in ″Table and chart views″
data you want by creating custom queries. chapter)

Chapter 5, “Attributes reference,” on page 23

in this guide
Build links from one workspace to another. IBM Tivoli Monitoring User’s Guide:
v ″Link from a workspace″ (in ″Custom
workspaces″ chapter)
v ″Link from a table or chart″ (in ″Table and
chart views″ chapter)
Identify which predefined situations started IBM Tivoli Monitoring User’s Guide: ″What
running automatically when you started the the enterprise workspace shows″ (in
Tivoli Enterprise Monitoring Server. ″Monitoring: real-time and event-based″
chapter, ″Using workspaces″ section)

Chapter 6, “Situations reference,” on page 29

in this guide
Determine whether to run situations as Chapter 6, “Situations reference,” on page 29
defined, modify the values in situations, or in this guide
create new situations to detect possible
problems.

Monitor with custom situations that meet your requirements

When your environment requires situations with values that are different from
those in existing situations, or when you need to monitor conditions not defined
by the existing situations, you can create custom situations to detect problems with
resources in two ways:
v Create an entirely new situation
v Create a situation by copying and editing a predefined situation

Note: When you create and run a situation, an IBM Tivoli Enterprise Console
event is created. For information on how to define event severities from
forwarded IBM Tivoli Monitoring situations and other event information,
see the IBM Tivoli Monitoring Administrator’s Guide.

You can specify the following information for a situation:

v Name
v Attribute group and attributes
v Qualification to evaluate multiple rows when a situation has a multiple-row
attribute group (display item)
v Formula
v Take Action commands
v Run at startup
v Sampling interval
v Persistence
v Severity
v Clearing conditions

14 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 v Expert Advice
v When a true situation closes
v Available Managed Systems

Table 8 contains a list of the procedures for monitoring your resources with custom
situations that meet your requirements and a cross-reference to where you can find
information about each procedure.
Table 8. Monitoring with custom situations
Procedure Where to find information
Create an entirely new situation. IBM Tivoli Monitoring User’s Guide: ″Creating
a new situation″ (in ″Situations for
event-based monitoring″ chapter, ″Creating a
situation″ section)

Chapter 5, “Attributes reference,” on page 23

in this guide
Create a situation by copying and editing a IBM Tivoli Monitoring User’s Guide:
predefined situation. ″Customizing a Situation″ (in ″Situations for
event-based monitoring″ chapter)

Chapter 6, “Situations reference,” on page 29

in this guide

Chapter 5, “Attributes reference,” on page 23

in this guide
Run a situation on a managed system. IBM Tivoli Monitoring User’s Guide:
″Situations for event-based monitoring″
chapter
v ″Associate situations with navigator
items″
v ″Distribute the situation″ (in ″Customizing
a situation″ section)
v ″Start, stop, or delete a situation″

Collect and view historical data

When you collect historical data, you specify the following configuration
requirements:
v Attribute groups for which to collect data
v Collection interval
v Summarization and pruning of attribute groups
v Roll-off interval to a data warehouse, if any
v Where to store the collected data (at the agent or the Tivoli Enterprise
Monitoring Server)

Table 9 on page 16 contains a list of the procedures for collecting and viewing
historical data and a cross-reference to where you can find information about each
procedure.

Chapter 3. How to use the Monitoring Agent for UNIX Logs 15

 Table 9. Collecting and viewing historical data
Procedure Where to find information
Configure and start collecting short-term IBM Tivoli Monitoring User’s Guide:
data (24 hours). ″Historical reporting″ (in ″Table and chart
views″ chapter)
Configure and start collecting longer-term
data (more than 24 hours). IBM Tivoli Monitoring Administrator’s Guide
View historical data in the Tivoli Enterprise
Portal. “Disk capacity planning for historical data”
on page 26
Create reports from historical data using
third-party reporting tools.
Filter out unwanted data to see specific
areas of interest.

16 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Chapter 4. Workspaces reference
This chapter contains an overview of workspaces, references for detailed
information about workspaces, and descriptions of the predefined workspaces
included in this monitoring agent.

About workspaces
A workspace is the working area of the Tivoli Enterprise Portal application
window. At the left of the workspace is a Navigator that you use to select the
workspace you want to see.

As you select items in the Navigator, the workspace presents views pertinent to
your selection. Each workspace has at least one view. Some views have links to
workspaces. Every workspace has a set of properties associated with it.

This monitoring agent provides predefined workspaces. You cannot modify or

delete the predefined workspaces, but you can create new workspaces by editing
them and saving the changes with a different name.

More information about workspaces

For more information about creating, customizing, and working with workspaces,
see the IBM Tivoli Monitoring User’s Guide.

For a list of the predefined workspaces for this monitoring agent and a description
of each workspace, refer to the Predefined workspaces section below and the
information in that section for each individual workspace.

Predefined workspaces
The Monitoring Agent for UNIX Logs provides the following predefined
workspaces:
v Log Entries
v Monitored Logs

The remaining sections of this chapter contain descriptions of each of these

predefined workspaces.

Log Entries workspace

The Log Entries workspace displays entries from any monitored log that occurred
within a specified time range. The same format is used for all logs, regardless of
their type. After the Tivoli Enterprise Portal has retrieved and displayed the entries
for the required time range, you can perform additional sorting and filtering. This
workspace is comprised of three views:
v Log Entries (table view)
v Log Size (bar chart)
v Number of Events (bar chart)

The Log Entries table view provides entry data and a description of each entry in
the monitored log. The Log Size chart depicts the size of each monitored log file, in

© Copyright IBM Corp. 2005 17

 bytes. The Number of Events chart depicts the total number of events detected by
the monitor since the monitor was first started. Based on the information that this
workspace provides, you can make changes, and set up situations.

Monitored Logs workspace

The Monitored Logs workspace provides basic information about the logs you are
monitoring. Workspace columns display:
v Logs that you have elected to monitor
v Basic information about each log, such as the log size and the time at which the
last log was modified
v Status of each monitor
v Time at which each monitor started or stopped
v Number of events detected by each monitor

This workspace is comprised of three views:

v Log Size (bar chart)
v Monitored Logs (table view)
v Number of Events (bar chart)

The Log Size chart depicts the size of each monitored log file, in bytes. The
Monitored Logs table view lists a variety of status details associated with the logs
you are monitoring. The Number of Events chart depicts the total number of
events detected by the monitor since the monitor was first started. Based on the
information that this workspace provides, you can make changes, and set up
situations.

Typical scenarios
This section illustrates how you can use the workspaces to monitor logs in some
typical scenarios.

Security issues
A common technique used by hackers to gain unauthorized access to your systems
is to guess the password for a known userid, often for the superuser. Failed logon
attempts are often recorded in a log. For instance, if someone issues the “su”
command to change the user ID to which they are currently logged on and enters
an invalid password for the new user ID, an entry is usually written to the file
/usr/adm/suaudit. You can create a situation that alerts you to possible break-in
attempts when repeated login failures to a user ID occur within a short period of
time.

Display the Monitored Logs workspace for the system in question to confirm that
you are monitoring the appropriate log. Display the Log Entries workspace for the
log to verify the format of the message written when a login attempt fails, for
example, “BAD SU from user1 to root”. Construct a situation that will fire if a
message indicating a logon failure to “root” is detected more frequently than
would normally be expected.

File server problems

To enable the sharing of data between multiple systems, many sites designate one
system as a file server and utilize a facility such as NFS to service remote file
access requests. In these environments, network outages or problems with the

18 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

server itself can impact many users. Often, a message is written to a log, for
example, /var/adm/messages, if an NFS request issued by a client on a remote
system fails.

Display the Monitored Logs workspace for a client system to confirm that you are
monitoring the appropriate log. Display the Log Entries workspace for the log to
verify the format of the message written when an NFS request fails, for example,
“NFS server system1 not responding”. Construct a situation that fires if a message
indicating a server problem is detected more frequently then would normally be
expected.

Monitoring user and third-party vendor applications

You can monitor any application that is already writing messages to one of the
supported standard log types (for example, syslog, utmp and errlog), by simply
adding an entry for the log in configuration file and restarting or refreshing the
monitoring agent.

If you want to monitor an application that is not already writing to a log file and
you can modify the application, add the necessary syslog system calls (or, on AIX
platforms, errlog system calls), to the code to produce the desired messages and
add the new logs to the configuration file.

If you want to monitor an application that you cannot modify and which is
writing messages to an ASCII log in a non-standard format, add the log to the
configuration file as normal but set its type to ‘U’ (user-defined). User-defined logs
require a format command that the monitoring agent uses to read log entries and
map the data into the Log Entries workspace.

Suppose you wish to monitor a log called /usr/adm/logs/mylog that is

comprised of entries such as those following:
MSG123456I/1024 10/04/05 12:15:32 region1 : Application 8 started
MSG234567W/2048 10/04/05 13:01:31 region2 : No journal files
opened
MSG345678E/4096 10/04/05 14:57:02 region1 : Unable to open file
‘FILE1’

The message identifier is terminated by either ‘I’ (informational), ‘W’ (warning) or

‘E’ (Error) depending on the severity of the message. You want to create a situation
that will fire only when type ‘E’ messages are written. In the Log Entries
workspace for this log, you want the message identifier to prefix the message text
and both to be displayed in the description column. Lastly, the decimal number
that appears after the message identifier is a reason code that you wish to convert
to hexadecimal and display in the class column prefixed by the literal “RC =“.

The following is the entry that you would include in the configuration file that will
format the log entries appropriately both for your situation and for report requests.
See “Customer configuration file format” on page 7 for details on the format of the
configuration file.

Note: The entire entry must be contained on a single line.

/usr/adm/logs/mylog ;N ;U ;a,”%9s%c/%d %d/%d/%d %d:%d:%d %s
:%[^\n]” , desc type class = “RC = %x” month day year hour
min sec source desc = “ %s”

Analyzing this line one component at a time from left to right, you can see that the
first item specifies the absolute file name of the log you want to monitor, in this

Chapter 4. Workspaces reference 19

 case “/usr/adm/logs/mylog”. Following the white space delimiter and the
semicolon (which indicates the start of the next item), is an ‘N’ stating that this log
is not being monitored in debug mode.

The ‘U’ item indicates that the log type is user-defined which mandates the
presence of the last item, the format command.

The format command starts with an ‘a’ (ASCII) and a comma. Everything between
the double quotes that follow is part of the format description that tells the
monitoring agent the format of the log. This format description allows you to
break each log entry into arbitrary fields consisting of one or more characters.
Following the format description is a comma that is followed by the mapping
specifications. These indicate into which column of the Log Entries workspace each
field must be mapped.

In this example, the format description breaks each log entry into 11 fields. Each
field is then mapped into one column of the Log Entries workspace by the 11
mapping specifiers.
Table 10. Log entry fields and their descriptions
Mapping and formatting data into the Log Entries
Field Scan directive workspace
1 %9s Consumes the first 9 characters of the message identifier
and is mapped into the “description” column.
2 %c Consumes the 10th character, in this example the message
severity indicator, (I, W or E). This is mapped into the
“type” column.
3 /%d Consumes the ‘/’ character and the following integer. The
‘/’ literal causes the ‘/’ character to be discarded but the
integer is mapped into the ‘class’ column. The mapping
specification for the class column includes a format specifier
“RC=%x”. This inserts the literal “RC =” into the column
and converts the integer to hexadecimal.
4 %d Consumes the white space and the following integer
mapping it into the “month” component of the Entry Time
column.
5 /%d Consumes and discards the ‘/’ character and then consumes
the following integer mapping it into the “day” component
of the Entry Time column.
6 /%d Consumes and discards the ‘/’ character; then consumes the
following integer mapping it into the “year” component of
the Entry Time column.
7 %d Consumes the next integer mapping it into the ‘hour’
component of the Entry Time column.
8 :%d Consumes and discards the ‘:’ character and then consumes
the following integer mapping it into the ‘minute’
component of the Entry Time column.
9 :%d Consumes and discards the ‘:’ character and then consumes
the following integer mapping it into the ‘second’
component of the Entry Time column.
10 %s Consumes and discards the white space preceding the next
character; then maps the next character string, (terminated
by white space), into the ‘source’ column.

20 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 10. Log entry fields and their descriptions (continued)
Mapping and formatting data into the Log Entries
Field Scan directive workspace
11 :%[^\n] The white space preceding the colon and the colon literal
consumes and discards all white space preceding the colon
and the colon itself in a log entry. The scanset discards all
white space between the colon and message content in a log
entry and then maps all remaining characters in the entry
into the ‘description’ column. Since the message identifier is
also being mapped into this same column, this mapping
specification includes a format specifier, “ %s”, which
simply inserts a space between the two mapped fields.

The resulting Log Entries workspace appears as follows:

Table 11. Sample Log Entry workspace
Entry time Description Source System Class Type
10/04/05 MSG345678 region1 RC = 1000 E
14:57:02 Unable to
open file
‘FILE1’
10/04/05 MSG234567 region2 RC = 800 W
13:01:3 No journal
files opened
10/04/05 MSG123456 region1 RC = 400 I
12:15:32 Application 8
started

You can now create a situation that fires if any ’E’ type messages are written to
this log file by including the following predicates:
v Log_Entries.Log_Name= mylog
v Log_Entries.Type= E

Resetting a situation using the Until predicate

When fired, situations that are based on the Log Entries workspace will remain in
a raised state. From the Events View, you have the option to reset a situation that
has fired thereby changing the managed object’s state back to the normal OK state.

You can also include an ‘Until’ predicate in the situation that causes it to be reset
automatically. The Until predicate allows you to specify that the situation is to be
reset after a certain time interval or when another situation is true.

This can be useful if you are monitoring events that occur in pairs, for example:
server redwood not responding
server redwood OK

In this case, you might create a situation called “Server_OK” that monitors a
certain log file looking for messages that contain the text “redwood OK”. Then
create another situation called “Server_Error” monitoring the same log but looking
for the text “redwood not responding”. In this second situation, include an Until
predicate. Open the Until settings page and refer to the “Reset this situation when”

Chapter 4. Workspaces reference 21

 box that contains 3 radio buttons. Select the button called “Another situation is
TRUE” and in the “Resetting situation” box enter the name of the first situation,
“Server_OK”.

Distribute both situations as usual to the managed systems on which you want
them to run. Add the situation called “Server Error” to one of the states of a new
or existing template and drag the template to the Enterprise icon to create a
managed object. Assign the managed object to one or more of the managed
systems to which you distributed the situations

When a event is written to the monitored log containing the text “redwood not
responding”, the Server_Error situation fires causing the managed object to change
state. When a message is written to the log containing the text “redwood OK”, the
Server_Error situation is reset and the managed object state reverts to normal.

22 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Chapter 5. Attributes reference
This chapter contains information about the following topics:
v Overview of attributes
v References for detailed information about attributes
v Descriptions of the attributes for each attribute group included in this
monitoring agent
v Disk space requirements for historical data

About attributes
Attributes are the application properties being measured and reported by the
Monitoring Agent for UNIX Logs, such as the log size. Some monitoring agents
have fewer than 100 attributes, while others have over 1000.

Attributes are organized into groups according to their purpose. The attributes in a
group can be used in the following two ways:
v Chart or table views
Attributes are displayed in chart and table views. The chart and table views use
queries to specify which attribute values to request from a monitoring agent.
You use the Query editor to create a new query, modify an existing query, or
apply filters and set styles to define the content and appearance of a view based
on an existing query.
v Situations
You use attributes to create situations that monitor the state of your operating
system, database, or application. A situation describes a condition you want to
test. When you start a situation, the Tivoli Enterprise Portal compares the values
you have assigned to the situation attributes with the values collected by the
Monitoring Agent for UNIX Logs and registers an event if the condition is met.
You are alerted to events by indicator icons that appear in the Navigator.

Some of the attributes in this chapter are listed twice, with the second attribute
having a ″(Unicode)″ designation after the attribute name. These Unicode attributes
were created to provide access to globalized data. Use the globalized attribute
names because this is where the monitoring agent is putting the data. If you were
using a previous Candle® OMEGAMON release of this monitoring agent, you
must run the Application Migration Tool to create globalized attributes for your
customized queries, situations, and policies. See the IBM Tivoli Monitoring
Installation and Setup Guide for more information.

More information about attributes

For more information about using attributes and attribute groups, see the IBM
Tivoli Monitoring User’s Guide.

For a list of the attributes groups, a list of the attributes in each attribute group,
and descriptions of the attributes for this monitoring agent, refer to the Attribute
groups and attributes section in this chapter.

© Copyright IBM Corp. 2005 23

Attribute groups and attributes for the Monitoring Agent for UNIX Logs
This monitoring agent contains the following attribute groups:
v Log Entries
v Monitored Logs

The following sections contain descriptions of these attribute groups, which are
listed alphabetically. Each description contains a list of attributes in the attribute
group.

Log Entries attributes

Use the Log Entries attributes to create situations to monitor entries made to
monitored logs.

Class The class of entry for errlogs, indicated by S = software, H = hardware, or O

= errlogger. Valid entry is an alphanumeric text string, with a maximum length of
16 characters.

Description The content of the log entry. Valid entry is an alphanumeric text
string, with a maximum length of 256 characters.

Description (Unicode) The content of the log entry. Valid entry is an alphanumeric
text string, with a maximum length of 768 bytes. This attribute is globalized.

Entry Time The date and time, as set on the monitored system, indicating the
instance when the entry was written.

Frequency Threshold The number of times an event must occur within a

user-specified interval before a situation is raised. Valid entries are integers in the
range 0 to 99999. Note that Frequency Threshold does not display in the
workspace, although it can be used as a situation predicate.

Log Name The name of the monitored log. Valid entry is an alphanumeric text
string, with a maximum length of 128 characters.

Log Name (Unicode) The name of the monitored log. Valid entry is an
alphanumeric text string, with a maximum length of 384 bytes. This attribute is
globalized.

Log Path The absolute path name of the monitored log. Valid entry is an
alphanumeric text string, with a maximum length of 256 characters.

Log Path (Unicode) The absolute path name of the monitored log. Valid entry is an
alphanumeric text string, with a maximum length of 768 bytes. This attribute is
globalized.

Managed System The name of the node that the agent is monitoring. Valid entry is
an alphanumeric text string, with a maximum length of 64 characters.

Period Threshold The interval within which an event must occur at least a
user-specified number of times before a situation is raised. Valid entries are
integers in the range 0 to 31622400. Note that Period Threshold does not display in
the workspace, although it can be used as a situation predicate.

24 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 Source The application or resource that logged the entry. Valid entry is an
alphanumeric text string, with a maximum length of 32 characters.

Source (Unicode) The application or resource that logged the entry. Valid entry is
an alphanumeric text string, with a maximum length of 96 bytes. This attribute is
globalized.

System The system on which the entry was written. Valid entry is an
alphanumeric text string, with a maximum length of 32 characters.

Timestamp The date and time, as set on the monitored system, indicating the
instance when the agent collects information.

Type The type of entry for errlogs and utmp logs. For errlog entries, the entry
types include PEND, PERF, PERM, TEMP and UNKN. For utmp logs, the entry
types include Unused space, Run level, System boot time, User logon time, User
idle time, init process, getty waiting, User process, Zombie process, and
Accounting. Valid entry is an alphanumeric text string, with a maximum length of
16 characters.

Monitored Logs attributes

Use the Monitored Logs attributes to create situations to monitor logs on a remote
node.

Date Last Modified The date and time, as set on the monitored system, indicating
the instance when the log was last modified.

Debug Mode This attribute indicates whether or not preformatted and

postformatted events written to this log are also written to a debug log. Valid entry
is an alphanumeric text string, with a maximum length of one character (the valid
values are N for NO and Y for YES).

Format Command The name of the command to be invoked to format log entries.
For errlogs, this attribute represents the name of the command to be invoked to
format entries in ASCII format. For user logs, this attribute represents the format
command used to describe the log’s format and how the data must be mapped
and formatted in the Log Entries workspace. Valid entry is an alphanumeric text
string, with a maximum length of 256 characters.

Log Name The name of the monitored log. Valid entry is an alphanumeric text
string, with a maximum length of 128 characters.

Log Name (Unicode) The name of the monitored log. Valid entry is an
alphanumeric text string, with a maximum length of 384 bytes.

Log Path The absolute path name of the monitored log. Valid entry is an
alphanumeric text string, with a maximum length of 256 characters.

Log Path (Unicode) The absolute path name of the monitored log. Valid entry is an
alphanumeric text string, with a maximum length of 768 bytes. This attribute is
globalized.

Log Size The size of the monitored log file, in bytes. Valid entries are integer in the
range of 0 to 2147483647.

Chapter 5. Attributes reference 25

 Log Type The log type, indicated by S = syslog, E = errlog, A = admin log, or U =
user log. Valid entry is an alphanumeric text string, with a maximum length of one
character.

Managed System The name of the node that the agent is monitoring. Valid entry is
an alphanumeric text string, with a maximum length of 64 characters.

Monitor Start/Stop Time A time stamp indicating the time at which a monitor
started running (if the monitor status is running) or the time at which the monitor
terminated.

Monitor Status If the log monitor is active, the status is running; otherwise, the
status indicates the error that caused the monitor to terminate. Valid entry is an
alphanumeric text string, with a maximum length of 32 characters. Valid values
are:

Error: create child failed

Error: create pipe failed
Error: format command
Error: get pipe flag failed
Error: insufficient memory
Error: log open failed
Error: log read failed
Error: log rewind failed
Error: pipe FD to FP failed
Error: reset EOF failed
Error: seek for EOF failed
Error: set pipe flag failed
Error: unknown
Error: wait for event failed
Error: wait loop failed
Not started
Running
Stopped

Number of Events The number of events detected by the monitor since monitoring
started. Valid entries are integers in the range of 0 to 2147483647.

Number of Format Errors The number of events that the monitor was unable to
understand and format (and as a result, were discarded). Valid entries are integers
in the range of 0 to 2147483647.

Timestamp The date and time, as set on the monitored system, indicating the
instance when the agent collects information.

Disk capacity planning for historical data

Disk capacity planning for a monitoring agent is a prediction of the amount of disk
space to be consumed for each attribute group whose historical data is being
collected. Required disk storage is an important factor to consider when you are
defining data collection rules and your strategy for historical data collection.

Calculate expected disk space consumption by multiplying the number of bytes

per instance by the expected number of instances, and then multiplying that

26 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 product by the number of samples. Table 12 provides the following information
required to calculate disk space for the Monitoring Agent for UNIX Logs:
v DB table name is the table name as it would appear in the warehouse database, if
the attribute group is configured to be written to the warehouse.
v Bytes per instance (agent) is an estimate of the record length for each row or
instance written to the agent disk for historical data collection. This estimate can
be used for agent disk space planning purposes.
v Bytes per instance (warehouse) is an estimate of the record length for detailed
records written to the warehouse database, if the attribute group is configured to
be written to the warehouse. Detailed records are those that have been uploaded
from the agent for long-term historical data collection. This estimate can be used
for warehouse disk space planning purposes.
v Bytes per summarized instance (warehouse) is an estimate of the record length for
aggregate records written to the warehouse database, if the attribute group is
configured to be written to the warehouse. Aggregate records are created by the
Summarization agent for attribute groups that have been configured for
summarization. This estimate can be used for warehouse disk space planning
purposes.
v Expected number of instances is a guideline that can be different for each attribute
group, because it is the number of instances of data that the agent will return for
a given attribute group, and depends upon the application environment that is
being monitored. For example, if your attribute group is monitoring each
processor on your machine and you have a dual processor machine, the number
of instances is 2.
The IBM Tivoli Monitoring Installation and Setup Guide contains formulas that can
be used to estimate the amount of disk space used at the agent and in the
warehouse database for historical data collection of an attribute group.
Table 12. Capacity planning for historical data
Bytes per
Bytes per Bytes per summarized
Attribute instance instance instance Expected number of
group DB table name (agent) (warehouse) (warehouse) instances
Log Entries ULLOGENT 2864 2894 2931 One row for each entry
added to the log file
during the time period
selected by the user. (The
default time period is the
last 12 hours.) The
number of rows varies
depending on the type of
log and system activity.
Monitored ULMONLOG 1958 1987 2141 One row for each log file
Logs monitored.

For more information about historical data collection, see the IBM Tivoli Monitoring
Administrator’s Guide.

Chapter 5. Attributes reference 27

28 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Chapter 6. Situations reference
This chapter contains an overview of situations, references for detailed information
about situations, and descriptions of the predefined situations included in this
monitoring agent.

About situations
A situation is a logical expression involving one or more system conditions.
Situations are used to monitor the condition of systems in your network. You can
manage situations from the Tivoli Enterprise Portal by using the Situation editor.

The IBM Tivoli Monitoring agents that you use to monitor your system
environment are shipped with a set of predefined situations that you can use as-is
or you can create new situations to meet your requirements. Predefined situations
contain attributes that check for system conditions common to many enterprises.

Using predefined situations can improve the speed with which you can begin
using the Monitoring Agent for UNIX Logs. You can examine and, if necessary,
change the conditions or values being monitored by a predefined situation to those
best suited to your enterprise.

Note: The predefined situations provided with this monitoring agent are not
read-only. Do not edit these situations and save over them. Software updates
will write over any of the changes that you make to these situations.
Instead, clone the situations that you want to change to suit your enterprise.

You can display predefined situations and create your own situations using the
Situation editor. The left frame of the Situation editor initially lists the situations
associated with the Navigator item that you selected. When you click a situation
name or create a new situation, the right frame opens with the following tabs:
Formula
Condition being tested
Distribution
List of managed systems (operating systems, subsystems, or applications)
to which the situation can be distributed.
Expert Advice
Comments and instructions to be read in the event workspace
Action
Command to be sent to the system
Until Duration of the situation

More information about situations

The IBM Tivoli Monitoring User’s Guide contains more information about predefined
and custom situations and how to use them to respond to alerts.

For a list of the predefined situations for this monitoring agent and a description
of each situation, refer to the Predefined situations section below and the
information in that section for each individual situation.

© Copyright IBM Corp. 2005 29

Predefined situations
This monitoring agent contains the following predefined situations, which are
organized alphabetically:
v HACMP_acquire_service_addr
v HACMP_acquire_takeover_ addr
v HACMP_config_too_long
v HACMP_event_error
v HACMP_fail_standby
v HACMP_get_disk_vg_fs
v HACMP_join_standby
v HACMP_network_down
v HACMP_network_down_ complete
v HACMP_network_up
v HACMP_network_up_complete
v HACMP_node_down
v HACMP_node_down_complete
v HACMP_node_down_local
v HACMP_node_down_local_ complete
v HACMP_node_down_remote
v HACMP_node_down_rmt_ complete
v HACMP_node_up
v HACMP_node_up_complete
v HACMP_node_up_local
v HACMP_node_up_local_ complete
v HACMP_node_up_remote
v HACMP_node_up_remote_ complete
v HACMP_release_service_addr
v HACMP_release_takeover_addr
v HACMP_release_vg_fs
v HACMP_start_server
v HACMP_stop_server
v HACMP_swap_adapter
v HACMP_swap_adapter_complete
v UNIX_LAA_Bad_su_to_root_Warning
v UNIX_LAA_Log_Size_Warning

The situations for High Availability Cluster Multiprocessing (HACMP™) are

supported only on the AIX platform.

The remaining sections of this chapter contain descriptions of each of these

predefined situations. The situations are organized alphabetically.

HACMP_acquire_service_addr situation
Configures the boot address to the corresponding service address and starts
TCP/IP servers and network daemons by running the telnet -a command.

30 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ acquire_service_addr

HACMP_acquire_takeover_ addr situation

Acquires the takeover IP address by checking configured standby addresses and
swapping them with failed service addresses.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ acquire_takeover_addr

HACMP_config_too_long situation
Sends a periodic console message when a node has been in reconfiguration for
more than six minutes.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ config_too_long

HACMP_event_error situation
Occurs when an HACMP event script fails for some reason.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ event_error

HACMP_fail_standby situation
Sends a console message when a standby adapter fails or is no longer available
because it has been used to take over the IP address of another adapter.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ fail_standby

HACMP_get_disk_vg_fs situation
Acquires disk, volume group, and file system resources as part of takeover.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ get_disk_vg_fs

Chapter 6. Situations reference 31

 HACMP_join_standby situation
Sends a console message when a standby adapter becomes available.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log*AND *SCAN Log_Entries.Description
*EQ join_standby

HACMP_network_down situation
Occurs when the cluster determines that a network has failed. The event script
provided takes no default action because the appropriate action is site or LAN
specific.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log*AND *SCAN Log_Entries.Description
*EQ network_down

HACMP_network_down_complete situation
Occurs only after a network_down event has successfully completed. The event
script provided takes no default action because the appropriate action is site or
LAN specific.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ network_down_complete

HACMP_network_up situation
Occurs when the cluster determines that a network has become available. The
event script provided takes no default action because the appropriate action is site
or LAN specific.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ network_up

HACMP_network_up_complete situation
Occurs only after a network_up_event has successfully completed. The event script
provided takes no default action because the appropriate action is site or LAN
specific.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ network_up_complete

32 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

HACMP_node_down situation
Occurs when a node is detaching from the cluster, either voluntarily or because of
a failure. Depending on whether the node is local or remote, either the
node_down_local or node_down_remote subevent is called.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log*AND *SCAN Log_Entries.Description
*EQ node_down

HACMP_node_down_complete situation
Occurs only after a node_down event has successfully completed. Depending on
whether the node is local or remote, either the node_down_local or
node_down_remote_complete subevent is called.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log*AND *SCAN Log_Entries.Description
*EQ node_down_complete

HACMP_node_down_local situation
Releases resources taken from a remote node, stops application servers, releases a
service address taken from a remote node, releases concurrent volume groups,
unmounts file systems, and reconfigures the node to its boot address.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_down_local

HACMP_node_down_local_complete situation
Instructs the cluster manager to exit when the local node has completed detaching
from the cluster. This event occurs only after a node_down_local event has
successfully completed.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_down_local_complete

HACMP_node_down_remote situation
Unmounts any NFS file systems and places a concurrent volume group in
nonconcurrent mode when the local node is the only surviving node in the cluster.
If the failed node did not go down gracefully, acquires failed nodes resources: file
systems, volume groups, and disks and service address.

This situation is not activated at startup.

This situation has the following formula:

Chapter 6. Situations reference 33

 *IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_down_remote

HACMP_node_down_rmt_complete situation
Starts takeover application servers if the remote node did not go down gracefully.
This event occurs only after node_down_remote event has successfully completed.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_down_remote_complete

HACMP_node_up situation
Occurs when a node is joining the cluster. Depending on whether the node is local
or remote, either the node_up_local or node_up_remote subevent is called.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log*AND *SCAN Log_Entries.Description
*EQ node_up

HACMP_node_up_complete situation
Occurs only after a node_up has successfully completed. Depending on whether
the node is local or remote, either node_up_local_complete or
node_up_remote_complete subevent is called.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_up_complete

HACMP_node_up_local situation
When the local node attaches to the cluster, the HACMP_node_up_local situation
acquires the services address, clears the application server file, acquires file
systems, volume groups and disk resources, exports file systems and either
activates concurrent volume groups or puts them into concurrent mode depending
upon the status of the remote nodes.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_up_local

HACMP_node_up_local_complete situation
Starts application servers and then checks to see if an inactive takeover is needed.
This event only occurs after node_up_local event has successfully completed.

This situation is not activated at startup.

This situation has the following formula:

34 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 *IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_up_local_complete

HACMP_node_up_remote situation
Causes the local node to do an NFS mount only after the remote node starts and to
place the concurrent volume groups into concurrent mode.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_up_remote

HACMP_node_up_remote_complete situation
Allows the local node to do an NFS mount only after the remote node is
completely up. This event occurs only after a node_up_remote event has
successfully completed.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ node_up_remote_complete

HACMP_release_service_addr situation
Detaches the service address and reconfigures to the boot address.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ release_service_addr

HACMP_release_takeover_addr situation
Identifies a takeover address to be released because a standby adapter on the local
node is masquerading as the service address of the remote node. Reconfigures the
local standby into its original role.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ release_takeover_addr

HACMP_release_vg_fs situation
Releases volume groups and file systems that the local node took from the remote
node.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ release_vg_fs

Chapter 6. Situations reference 35

 HACMP_start_server situation
Starts application servers.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ start_server

HACMP_stop_server situation
Stops application servers.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log*AND *SCAN Log_Entries.Description
*EQ stop_server

HACMP_swap_adapter situation
Exchanges or swaps the IP addresses of two network interface. NIS and name
serving are temporarily turned off during the event.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ swap_adapter

HACMP_swap_adapter_complete situation
Occurs only after a swap_adapter event has successfully completed. Ensures that
the local Address Resolution Protocol (ARP) cache is updated by deleting entries
and pinging cluster IP addresses.

This situation is not activated at startup.

This situation has the following formula:

*IF *SCAN Log_Entries.Log_Name *EQ cluster.log *AND *SCAN Log_Entries.Description
*EQ swap_adapter_complete

UNIX_LAA_Bad_su_to_root_Warning situation
Raises an alert if a logon failure to root message is written to usr/adm/suaudit
more than three times within a minute.

This situation has the following formula:

Log_Entries.Log_Path EQ /usr/adm/
AND
Log_Entries.Log_Name EQ suaudit
AND
*SCAN Log_Entries.Description EQ ’Badsu’
AND
*SCAN Log_Entries.Description EQ ’to root’ AND
Log_Entries.Frequency_Threshold GT 3
AND
Log_Entries.Period_Threshold EQ 60

36 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

UNIX_LAA_Log_Size_Warning situation
Raises an alert if the size of any monitored log exceeds 10 MB.

This situation has the following formula:

Monitored_Logs.Size GT 10485760

Chapter 6. Situations reference 37

38 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Chapter 7. Take Action commands reference
This chapter contains an overview of Take Action commands and references for
detailed information about Take Action commands.

About Take Action commands

Take Action commands can be run from the desktop or included in a situation or a
policy.

When included in a situation, the command executes when the situation becomes
true. A Take Action command in a situation is also referred to as reflex automation.
When you enable a Take Action command in a situation, you automate a response
to system conditions. For example, you can use a Take Action command to send a
command to restart a process on the managed system or to send a text message to
a cell phone.

Advanced automation uses policies to perform actions, schedule work, and

automate manual tasks. A policy comprises a series of automated steps called
activities that are connected to create a workflow. After an activity is completed,
Tivoli Enterprise Portal receives return code feedback, and advanced automation
logic responds with subsequent activities prescribed by the feedback.

More information about Take Action commands

For more information about working with Take Action commands, see the IBM
Tivoli Monitoring User’s Guide.

Predefined Take Action commands

There are no predefined Take Action commands for this monitoring agent;
however, you can run commands yourself, and include those that you use often in
a list of available commands.

© Copyright IBM Corp. 2005 39

40 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Chapter 8. Policies reference
This chapter contains an overview of policies and references for detailed
information about policies.

About policies
Policies are an advanced automation technique for implementing more complex
workflow strategies than you can create through simple automation.

A policy is a set of automated system processes that can perform actions, schedule
work for users, or automate manual tasks. You use the Workflow Editor to design
policies. You control the order in which the policy executes a series of automated
steps, which are also called activities. Policies are connected to create a workflow.
After an activity is completed, Tivoli Enterprise Portal receives return code
feedback and advanced automation logic responds with subsequent activities
prescribed by the feedback.

Note: For monitoring agents that provide predefined policies, predefined policies
are not read-only. Do not edit these policies and save over them. Software
updates will write over any of the changes that you make to these policies.
Instead, clone the policies that you want to change to suit your enterprise.

More information about policies

For more information about working with policies, see the IBM Tivoli Monitoring
User’s Guide.

For information about using the Workflow Editor, see the IBM Tivoli Monitoring
Administrator’s Guide or the Tivoli Enterprise Portal online help.

Predefined policies
There are no predefined policies for this monitoring agent.

© Copyright IBM Corp. 2005 41

42 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Appendix A. Generic user log support
Generic User Log Support (GULS) allows you to monitor an ASCII log that does
not conform to any of the three supported types (syslogs, errlogs, and utmp logs).
This feature relies on a format command that you supply in the configuration file
entry for each user log you want to monitor (see “Customer configuration file
format” on page 7).

The format command describes:

v The format of the log to the monitoring agent
v How you want data that is read from the log to be mapped in the Tivoli
Enterprise Portal Log Entries table view

While the data is being mapped into the table view, you have the ability to
perform data type conversions (for example, decimal to hexadecimal), and do
formatting to clarify the table view and facilitate the creation of situations.

Format command
A format command is composed as follows:

A, “log format description” , data mapping [and formatting] specifications

A The letter A.
log format description
One or more scan directives enclosed within double quotation marks.
Details are provided in “Log format description” on page 45.
data mapping [and formatting] specifications
Indicates into which columns of the Log Entries table view the scanned
data is mapped. Details are provided in “Data mapping specifications” on
page 50.

The format description and formatting specifications both use a syntax based on
that used by the standard ‘C’ scanf and printf functions. The format command
syntax is, perhaps, best illustrated through a simple example. The format
command syntax is explained in detail after the example.

Example format command

Suppose you run an application at your site that produces an ASCII log,
myapp.log, and that you wish to monitor the messages written to this log. Suppose
also that a sample entry from this log is as follows:
MSG123 Dec 25 2004 03:15 pm system myapp: Server frodo not responding

The following format command enables the monitoring agent to monitor this log
allowing you to both create situations looking for specific messages and to display
the log’s contents within the Tivoli Enterprise Portal Log Entries table view.
A , “%s %s %d %d %d:%d %s %s %[^:] : %[^\n]” , type month day year
hour minute hour system source description

© Copyright IBM Corp. 2005 43

 myapp.log MSG123 Dec 25 2004 03 : 15 pm system1 myapp: Server frodo...\n

format A, "%s %s %d %d %d : %d %s %s %[^:]: %[^\n]"

type month day year hour min hour system source description
command

Monitoring
Agent

type MSG123

month Dec

day 25

year 2004

hour 3 pm

min 15

system system1

source myapp

description Server frodo not responding

Log Entries table view for myapp.log

Entry Time Description Source System Class Type
12/25/04 15:15:00 Server frodo not responding myapp system1 MSG123

Figure 1. Example format diagram

44 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Format comand syntax
A format command consists of two components:
v The log format description
v Data mapping and formatting specifications

Log format description

The format description is comprised of one or more scan directives enclosed within
double quotation marks (“...”). Generally, a scan directive identifies a field or group
of fields within a log entry, although a directive can identify a single character
within a log entry. A field is any sequence of nonwhite space characters terminated
by one or more white space characters (that is, a tab or blank).

For example, the sequence below consists of five fields:

Dec 25 2004 03:15 pm

In addition to fields with content that varies from one log entry to another, an
entry can contain fixed character strings that occur in the same relative location in
all log entries. These are termed literals. Literals can be embedded anywhere
within a format description.

A scan directive has the following format. (Items enclosed within brackets are
optional.)

%[(offset)][*][width][size]datatype

All scan directives must include at least a percent sign, ‘%’, and a datatype.

Each scan directive starts from where the previous one ended unless it is the first
(in which case it starts at column 1 in the log entry), or an offset option has been
included in the directive. Each scan directive consumes characters from a log entry
until any of the following occurs:
v An inappropriate character is encountered (that is, one that does not match the
expected data type).
v The field width, if specified, is exhausted.
v The end of the log entry is encountered.

Format description components

The following sections describe each of the format description components. To
simplify the discussion, all examples in the next section include only the relevant
scan directives from a format description. The corresponding mapping
specifications that must be included in a complete format command have been
omitted.

Literals: Literals describe a sequence of one or more characters that occur at the
same relative location in every entry in the log, and which you do not want to
map into a table view column.

Specifying a literal makes the monitoring agent look for and read those characters
from a log entry, and then discard them. If you include a literal, it must match
exactly the character sequence in a log entry, otherwise that entry is ignored.

For example, to read a time from a log that has the format:
03:15

Appendix A. Generic user log support 45

 The following scan directives can be used:
%d:%d

In this example, the colon ‘:’ preceding the second directive is a literal.

Any number of white space characters that immediately precede the start of a field
in a log entry are automatically consumed and discarded (unless the data type of
the next scan directive is a character, for example %c). To consume any number of
white space characters that are embedded within a literal in a log entry, include
one or more white space characters in the format description literal. For example,
suppose a log entry has the following format:
MSG123 < Code 9 > System1

If you want to extract only the message field, the code number, and the system,
use the following format description:
%s < Code%d >%s

The first directive scans in the message field. The single blank following the first
directive consumes all the white space between the message field and the ‘<‘ sign
in the log entry. Similarly, the single blank between the ‘<‘ sign and the literal
‘Code’ in the format description consumes all the white space between those same
literals within the log entry. No white space is required between the literal ‘Code’
and the ‘%d’ directive or between the literal ‘>‘ and the ‘%s’ directive because this
white space is automatically consumed.

To include a percent sign, ‘%’, in a literal, specify two adjacent percent sign
characters in the format description. For example, if you want to extract the
percentages from a log that contains the following three fields:
45% 82% 2%

Use the following format description:

%d%% %d%% %d%%

(The blanks embedded within the description are not required but clarify the
example.)

Offsets: Offsets allow you to specify the absolute column within a log entry at
which a scan directive starts; if no offset is specified, each succeeding scan starts
where the previous one ended. The first column in an entry is 1. Offsets can
facilitate the description of fixed field logs; that is, those in which a field starts in
the same column in every entry.

For example, suppose each log entry starts with a message number as in the
following example:
MSG123 Dec 25

If you want to extract only the message number, discarding the text “MSG” that
precedes it, you can use the following scan directive:
%(4)d

This causes the scan to start in column 4, skipping over and discarding the first
three characters.

46 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Field suppression: The character ‘*’ in a scan directive indicates that the scanned
data is suppressed. That is, the data is read from the log entry but discarded. For
example, suppose a log entry has the following form:
MSG123 Dec 25

If you want to skip over the message number field entirely, you can use this
format description:
%*s %s %d

These directives cause the message number field to be ignored but the month and
day are stored and mapped to a column. Since the data is discarded for scan
directives that are suppressed, such directives have no corresponding data
mapping specifier (which associates log data with a table view column).

Width: The width option allows you to specify the maximum number of
characters that is consumed from a log entry to satisfy a scan directive. For
example, suppose you want to extract only the first 2 digits from the message
number from the following log entry:
MSG123 Dec 25

You can use the following scan directives:

%*3s %2d %*d

The first directive, “%*3s”, discards the first three characters of the message
number field (the text “MSG”). The second directive, “%2d”, saves the digits “12”
for mapping and the last directive, “%*d”, discards any digits that remain in the
message field. The digit ‘3’ is consumed and discarded.

For scan directives that have a data type of “string” (that is, %s or %[]), the default
width is 31.

Size: The size option can be used in numeric (that is, non-string) directives and
controls the amount of storage that is reserved to hold a scanned number.
Allocating more storage allows larger numbers to be scanned and stored. Unless
you need to scan very large numbers or need to increase the precision, the default
sizes are probably sufficient. The effect of including a size option in a numeric scan
directive depends on the operating system on which the monitoring agent is run.

The valid size option and data type combinations that you can specify are listed in
the following table.
Table 13. Monitoring Agent for UNIX Logs valid size option and data type combinations
Size option Can be used with these data types
l d, i, o, u, x, e, f, g
ll d, i, o, u, x
L e, f, g
h d, i, o, u, x

If you explicitly include formatting in the data mapping specifier for a scanned
value rather than allowing the print directive to default, the size option that you
specify in the scan and print directives must be consistent.

Appendix A. Generic user log support 47

 Data type: The data type of a scan directive indicates whether the corresponding
characters in the log entry are alphanumeric or numeric and affects how the data is
stored by the monitoring agent once it has been scanned. Specifying that log data
is numeric instead of a simple alphanumeric string can simplify the format
description and allows the scanned data to be converted (for instance, displayed in
hexadecimal or scientific notation instead of decimal), as it is being mapped into a
table view column.

The following table lists the valid data types you can use in a scan directive to
describe alphanumeric data.
Table 14. Monitoring Agent for UNIX Logs valid alphanumeric data types
Data type Corresponding field in the log entry
s A sequence of nonwhite space characters. Characters from the log entry are
consumed until the first white space character is encountered or until the number of
characters specified in the field width has been exhausted. If no width is specified in
the directive, the default width is 31.
c A sequence of bytes. The number of bytes consumed is determined by the specified
width option. If no width is specified, the default is 1. Unlike all other data type
directives, white space immediately preceding the corresponding field in the log
entry is not automatically skipped. To skip over white space, you must explicitly
include a white space literal immediately preceding a directive with a data type of
character.

This feature is useful for describing fields in a log entry that might be blank
assuming the starting column and width of the optional field is known. For
example, suppose two entries from a log are as shown:
field1a field2a field3a
field1b field3b

Field 2, if present, always starts in column 12 and can be up to 9 characters long.

The following format description could be used:
%s %(12)9c %s

In this example, the second directive will store “field2a” when the first entry is
processed and will contain blank when the second entry is processed.

48 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 14. Monitoring Agent for UNIX Logs valid alphanumeric data types (continued)
Data type Corresponding field in the log entry
[inclusive scanset] or Any sequence of characters. A scanset data type is a generalized type ‘s’ (string) data
[^exclusive scanset type. In fact, the type ‘s’ directive can be expressed by the following exclusive
scanset:
%[^ \t\n]

This says that all non-blank, non-tab and non-newline characters will be consumed.
That is, the scan will end on the first white space character in the log entry. (See
“Escape characters” on page 55 for details on specifying escape characters in a
format command.)

In an inclusive scanset, characters from a log entry will be consumed until a

character is encountered that is not in the scanset. In an exclusive scanset (for
example, one that has a ‘^’ (circumflex) character immediately following the left
bracket), characters from a log entry will be consumed until a character is
encountered that is in the scanset.

A scanset allows a single scan directive to consume multiple log entry fields. For
example, a scanset that you might use frequently is one that is to “read all the
remaining characters in an entry”:
%[^\n]

That is, consume everything from the current position in an entry up to the newline
character, which marks the end of the entry.

A scanset directive can also be used to terminate a scan before a simple type ‘s’
variable would, that is, when a white space character is found. For example,
suppose a log entry has embedded within it either one of the following two field
sequences:
Error code:24
Warning code:16

If you want only to extract the numeric code itself, the following directives could be
used:
%*[^:]:%d

This format description consumes and discards (field is suppressed) all characters
until a colon is found (exclusive scanset), then consumes and discards the colon
itself (an embedded literal) and finally consumes and stores the numeric code.

As with a string data type, if you wish to consume more than 31 characters with a
scanset directive, you must include a maximum width option in the directive. For
example, to consume up to 60 characters from the current location up to the end of
the entry use:
%60[^\n]

Some operating systems support the use of a ‘-’ (dash) to represent a range of
characters, for example:
%[a-z]

This example includes all lowercase letters in the scanset. The character that
precedes the dash must be lexically less than the character following it otherwise the
dash stands for itself. Also, the dash stands for itself whenever it is the first or last
character in the scanset.

To include the right bracket in an inclusive scanset, it must immediately follow the
opening left bracket. To include the right bracket in an exclusive scanset, it must
immediately follow the circumflex character. In both cases, a right bracket so placed
is not considered the closing right bracket of the scanset.

Appendix A. Generic user log support 49

Table 14. Monitoring Agent for UNIX Logs valid alphanumeric data types (continued)
Data type Corresponding field in the log entry
d, u An optionally signed decimal integer.
i An optionally signed integer with a base determined by the first characters of the
number:
v If the first character is in the range 1 to 9, the base is 10.
v If the first character is 0 and the second character is in the range 0 to 7, the base is
8.
v If the first 2 characters are 0x (or 0X), the base is assumed to be 16; that is, all
characters in the range 0 to 9 and a to f (or A to F) are considered part of the
number.
o An optionally signed octal integer, that is, a string of integers in the range 0 to 7.
x An optionally signed hexadecimal integer, that is, a string of characters in the range
0 to 9 and a to f or A to F.
e,f,g An optionally signed string of digits that can contain a decimal point and/or an
exponent component that consists of an ‘E’ or an ‘e’ followed by an optionally
signed integer.

Data mapping specifications: The data mapping specifications that comprise the
second component of a format command are separated from the format description
by a single comma. Each mapping specification is separated from the next by
white space. Every non-suppressed scan directive in the format description must
have a single, corresponding data mapping specifier to indicate into which column
of the Log Entries table view the scanned data must be mapped. That is, the
general form of a format command is as follows:
A , “%scan1 %scan2 %*scan3 %scan4” , mapspec1 mapspec2
mapspec4

The third scan directive has no corresponding mapping specifier since it is

suppressed (*).

As the data is mapped into a table view column you can also optionally specify
how you want it formatted and (for data read in and stored in numeric form), that
numeric data is converted to a different type (for example, decimal to hexadecimal
or exponent form). See “Specifying log entry times” on page 57 for further details.

The columns into which scanned data can be mapped correspond to columns in
the Log Entries table view. The following table lists all the valid column names
and minimum abbreviations that can be used.
Table 15. Log entries table view column mapping names
Tivoli Enterprise Portal Log
Entries table view column Format command mapping
name name Minimum abbreviation
Entry Time month mo
day da
year ye
hour ho
minute mi
second se
Description description de

50 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 15. Log entries table view column mapping names (continued)
Tivoli Enterprise Portal Log
Entries table view column Format command mapping
name name Minimum abbreviation
Source source so
System system sy
Class class cl
Type type ty

Referring again to the example at the beginning of this section (“Example format
command” on page 43), notice that there are 10 scan directives and also that there
are 10 mapping specifications. Each successive scan directive, reading the format
description left to right, is associated with each successive mapping specifier. That
is, the first log entry field, MSG123, is read by the first scan directive, %s, and is
mapped to the first column specifier, type. The second field, Dec, is read by the
second scan directive, %s, and is mapped to the second column specifier, month,
and so on.

As indicated in this same example, it is possible to concatenate two or more

non-adjacent log entry fields into the same table view column. The log entry time
is in 12-hour format; that is:
03:15 pm

The corresponding format description and data mapping specifiers in the example
format command are:
“%d:%d %s” , hour minute hour

This causes ‘03’ to be read by the first scan directive, ‘%d’, and mapped into the
hour column. The next character in the log entry, the colon, matches the colon
literal in the format description and is discarded. The ‘15’ is read by the second
scan directive, ‘%d’, and is mapped into the minute column. The ‘pm’ is read by
the third scan directive, ‘%s’, and is also mapped into the hour column. The result
is that the hour and minute columns contain ‘3pm’ and ‘15’ respectively. (If the
monitoring agent is passed an hour of ‘3pm’ it converts this into 24-hour format
for display in the Log Entries table view. See “12-Hour format times” on page 57
for more information concerning valid date and time formats that can be passed to
the monitoring agent.)

The example above shows that it is necessary only to supply a column name into
which scanned log data is mapped. For each mapping specification that has no
explicit format specifier, default formatting is applied. The monitoring agent
expands the mapping specifiers in the previous example as follows:
“%d:%d %s” , hour=“%d” minute=“%d” hour=“%s”

How to override the default mapping format specifiers is the subject of the next
section.

Formatting mapped data: If no formatting is included for a given mapping

specification, a default format specification is assigned based on the data type and
data type size in the corresponding scan directive. To specify explicitly how
scanned data is formatted as they are mapped into a given column, follow the
column map name with an equal sign (=) and enclose your format specifier in
double quotation marks (“...”).

Appendix A. Generic user log support 51

 The syntax for a data mapping specification that includes a format specifier is as
follows:

MappingName[=“[literals]%[options][width]
[.precision][size]datatype[literals]”]

The MappingName corresponds to one of the full or abbreviated names from

columns 2 and 3 of Table 15 on page 50 and items in brackets are optional. Since
there is a one-to-one relationship between a scan directive in the format description
and a mapping specification, include at most one “%datatype” directive in a
mapping format specification.

Mapping format specifier components

The following paragraphs describe each of the mapping format specifier
components.

Literals: Literals can be included before the scanned data is mapped into a table
view column, afterwards, or both, and can serve to clarify the Log Entries table
view and facilitate the creation of situations. For example, suppose you are
monitoring a log that includes the following three fields:
... 13303 15 4 ...

The first field indicates a process identifier, the second represents a return code
and the third a severity. You might choose to map and format the data as follows:
“... %d %d %d ...” , ... source=“proc id. = %d” desc=“RC = %d ” desc=“;
Severity = %d”...

(The ellipses represent omitted fields.) This causes the following to be displayed in
the source and description columns in the Log Entries table view:

Source Description
proc id. = 13303 RC = 15 : Severity = 4

To include a single % (percent sign) in a column, include two consecutive %

characters in the format specifier. For example, desc=“%d%%” maps the integer 83
as “83%” into the description column.

Options: One or more options can be included in a mapping format specifier

although not all combinations are valid. The options and their meaning are shown
in the following table.
Table 16. Log Entries table view mapping options
Option Description Notes
‘ Formats integer portions for i, d, u, f, Effect depends on the locale setting
g and G data types with the on the managed system. Use the
appropriate thousands grouping locale -a command to view available
separator. locales and review the monitoring
agent log to determine the current
locale.
- Left-justifies data if number of Use with field width.
characters mapped is less than the
minimum field width (see below).

52 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 16. Log Entries table view mapping options (continued)
Option Description Notes
+ Inserts a ‘+’ or ‘-’ sign before a Valid only for signed numeric data
numeric value depending on types.
whether it is greater or less than
zero.
space Inserts a space character before a Valid only for signed numeric data
character positive numeric value; inserts a ‘-’ types. Ignored if ‘+’ option is also
sign before a negative value. specified.
# For ‘o’ data type, increases precision Not valid for c, d, s or u data types.
to force the first digit of the result to
be a zero.

For ‘x’ and ‘X’ data types, precedes a

nonzero result with ‘0x’ or ‘0X’
respectively.

For ‘e’, ‘E’, ‘f’, ‘g’ and ‘G’ data types,

the result always contains a decimal
point even if no digits follow it. For
‘g’ and ‘G’ data types, trailing zeros
are not removed from the result.
0 Pads to the field width with leading Valid only for numeric data types.
zeros. Ignored if ‘-’ option is also specified.
Ignored also for d, i, o, u, x and X
data types if precision is specified.

For example, suppose a log entry contains the character sequence “65000” and the
format command contains:
“... %d ...” , ... type=“%’0+9d”

The type column is displayed as:

+0065,000

Width: A decimal digit string included in a mapping format specifier signifies the
minimum width of the field into which the data is mapped. If the mapped data
contains fewer characters than the minimum field width, it is right justified and
padded on the left to the length specified by the field width. If the ‘-’ (left-justify)
option has been specified, the data is padded on the right.

For example, suppose the log entry contains the following fields:
1 789 82 4567

You supplied the following format command:

A, “%d%d%d%d” , desc=“%8d” desc=“%8d” desc=“%8d” desc=“%8d”

The description column is formatted as follows:

1 789 82 4567

A field width with a leading zero is interpreted as meaning that the field must be 0
padded.

Precision: A precision is specified by a [ . ] (dot) followed by a decimal digit

string. The effect of a precision depends on the type of data being mapped.

Appendix A. Generic user log support 53

 Table 17. Mapping precision and the data types specified
Data type Precision specifies
d, i, o, u, x, X The minimum number of digits to appear
e, E, f The number of digits to appear after the decimal point
g, G The maximum number of significant digits
strings The maximum number of bytes to be printed from a string

For example, suppose a log entry contains the following fields:

2 3.142857 123.45 On no account allow a vogon to read poetry at you

The format command is:

A,“%d%f%g%[^\n]” , de=“%.3d” de=“ %.3f” de=“ %.2g“ de=“ %.13s“

The description field contains the following:

002 3.143 1.2e+02 On no account

Size: The valid size options that you can specify in a format specification for
mapped data are the same as those that can be specified in a scan directive in the
log format description. See Table 13 on page 47 for a list of valid size and data
type combinations.

The data size specified in a mapping format specifier must be the same as that
specified in the corresponding scan directive.

Data type: As with the size option, the data type in a mapping format specifier
must be consistent with that in the corresponding scan directive. This means that if
data is scanned and stored as an integer, it must be mapped as an integer. If it is
scanned as a floating point number, it must be mapped as a floating point number.
If it is scanned as a character or character string it must be mapped as a character
or character string.

Numeric data can be scanned and stored in one of two families: the integer family
and the floating point family. Within each family, data can be represented in
different ways. An integer can be displayed in decimal, octal, hexadecimal and
unsigned formats. A floating point number can be displayed in decimal or
exponent notation. When numeric data is mapped, it is legal to use a different data
type to that in the scan directive as long as the format data type comes from the
same family. Put another way, it is not legal to mix data types from different
numeric families.

This feature allows you to perform type conversions as you map data into a table
view column to clarify the table view. For example, if a field in a log entry
contains the size of a file in bytes, you can display the size as a hexadecimal value
in the Log Entries table view by using the following in the format command:
A, “... %d ...” , ... desc = “File size is %#x bytes” ...

If the file size is, for example, 11259375 bytes, the description column will contain:
File size is 0xabcdef bytes

For an example that mixes data types from the floating point family, the size can
be displayed in exponent notation with the following format command:
A, “... %f ...” , ... desc = “File size is %.2e bytes” ...

54 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

This time, the description column would contain:
File size is 1.13e+07 bytes

The valid mapping data types are specified by family in the following tables.
Table 18. Integer family data types
Data type Format of data scanned and stored as integers
d, i Signed decimal numbers.
u Unsigned decimal numbers.
o Unsigned octal numbers.
x, X Unsigned hexadecimal numbers. The letters abcdef are used for x;
the letters ABCDEF are used for X.

Table 19. Floating point family data types

Data type Format of data scanned and stored as integers
f Signed decimal numbers with the number of digits after the decimal
point equal to the precision. If no precision is specified, the default is
6 digits.
e, E In exponent form, that is, [-].ddd+/-dd. One digit precedes the
decimal point and the precision specifies the number of digits that
follow it. The default precision is 6. The E data type produces a
number with E instead of e before the exponent.
g, G In either the f or g (G if E is used) formats, depending on the value
of the number with the precision specifying the number of significant
digits. The exponent form is used if the exponent is less than -4 or
greater than the precision.
c As a single character. The mapped data can have been read as a
single character, ‘%c’, or could be the first character of a string that
was stored using a scan directive such as ‘%s’, ‘%[]’, or ‘%nc’ (where
n is an integer field width).
s As a string. The mapped data must have been read and stored as a
string using a scan directive such as ‘%s’, ‘%[]’, or %nc’ (where n is
an integer field width greater than 1). All characters from the string
are printed up to the number of bytes indicated by the precision.

Escape characters: You might want to include characters in a format command

that either are not valid in the command itself (for example, the newline character),
or which have a special meaning to the monitoring agent when it is interpreting
the format command (for example, the double quote, “, character). Such characters
are represented in the format command by a sequence of two characters:
v The backward slash (\) escape character
v Following the backward slash, a character that represents the character code that
is being escaped

The backward slash character removes any special meaning from the following
character and causes the latter’s single character code value to be substituted
instead.

Escape characters can be included in three areas of a format command:

v In the format description
– As part of a literal

Appendix A. Generic user log support 55

 – Inside a scanset (%[]) type scan directive
v In a mapping specifier
– As part of a literal

For example, suppose you want to monitor a log that contains the following
sample entry:
“http://www.acme.com” : GET /download/Acme.exe

Suppose that you want to extract the character string between the set of double
quotation marks and everything after the colon. Also, assume you want to map the
first string into the source column of the Log Entries table view and that you want
to map the second string into the description column enclosing it in quotation
marks. The following format command accomplishes this:
A,”\”%[^\”]\” :%[^\n]” , source desc = “\”%s\””

Following the double quotation mark that starts the format description is an
escaped double quotation mark literal that consumes the double quotation mark
preceding “http:” in the sample log entry. The exclusive scanset scan directive
contains an escaped double quotation mark that terminates the first scan; that is,
“http://www.acme.com” is stored by the scanset directive. The escaped double
quotation mark, white space character, and colon literal following the scanset
directive consumes all characters up to the text “GET” in the sample log entry. The
second scanset consumes and stores all characters until a newline character is
encountered (end of line). The next non-escaped double quotation mark terminates
the format description component of the format command.

The mapping specification for the description column contains two escaped double
quotation mark literals surrounding the scanned string.

The following shows how the sample data above is displayed in the Log Entries
table view using this format command.

Description Source
“GET /download/Acme.exe” http://www.acme.com

The following table lists all the characters that can be represented by an escape
sequence and the associated escape sequence.
Table 20. Escape character sequence
Character Escape sequence representation
newline \n
horizontal tab \t
vertical tab \v
backspace \b
carriage return \r
backspace character (\) \\
single quotation mark (‘) \’
double quotation mark (“) \”
alert \a

56 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Specifying log entry times
The entry time displayed in the Log Entries table view is extracted from each log
entry and is not the time at which the log monitor detected the event. The format
command that you supply for each user-defined log must, therefore, include
scanning and mapping specifications for the entry time.

Since the format of dates and times varies so widely between logs, the Entry Time
column of the Log Entries table view is composed of six components: the year,
month, day, hour, minute, and second. You specify scanning and mapping pairs
explicitly for each component using one of the mapping names in Table 15 on page
50.

When the monitoring agent formats an entry from a log, it attempts to build a time
stamp with a format of:
mm/dd/yy hh:mm:ss

‘yy’ is the 2 digit year, the first ‘mm’ is the month, ‘dd’ is the day of the month,
‘hh’ is the hour (in 24-hour format), the second ‘mm’ is the minute and ‘ss’ is the
second. To do so, the monitoring agent expects that the data that is mapped into
each of the entry time component fields is a valid integer. The data type with
which each component was scanned and mapped is not important; what is
important is that the formatted result is an integer.

For example, suppose the date and time in a log entry is in the form:
MSG123 2005 03 03 10 15 56 ...

Use the following format command to extract and map the entry time components:
A , “%s %s %s %s %s %s %s” , desc year month day hour minute
second

The following format command is also valid:

A , “%s %d %d %d %d %d %d” , desc year month day hour minute
second

There are two exceptions to the requirement that all time components consist of
numeric data only: text months and 12-hour format times.

Text months: If the month of the log entry is in text form, for example, Jan, Feb
or JAN, FEB, you can read and map the month as a string. When the monitoring
agent is passed a month in this format, it will translate it to the appropriate
numeric value when constructing the entry time.

12-Hour format times: Some logs contain the entry time in 12-hour format, for
example, 03:15 pm. Since the monitoring agent displays entry times in 24-hour
format, for such logs you must pass the ‘am/pm’ indicator to the monitoring agent
in the hour column. An example format command for reading and mapping the
hour and minute from a log with this format follows.
“%d:%d%s” , hour minute hour

This concatenates the ‘am/pm’ indicator to the 12-hour value so that, using the
previous time as an example, the value “3pm” is passed to the monitoring agent.
When constructing the entry time for an event, the monitoring agent will translate
such a time to its equivalent 24-hour format, in this case ‘15’.

Appendix A. Generic user log support 57

 Hardcoding missing entry time components
If you omit a scan or map specification pair for the seconds component of the
entry time, it will default to zero. If you omit a scan or map pair for any of the
other entry time components, the monitoring agent will default the value to the
corresponding current value reported by the system clock both for real-time,
monitored events and for events formatted for a table view request (see the
exception which follows for table view requests if the omitted value is the year).
This can be appropriate for the purposes of monitoring but can lead to
unpredictable or misleading results for table view requests (which return all entries
from a given log that occurred within a certain time span).

If, for instance, the minute is not supplied within a log entry and you do not want
to let the minute default to the current system clock minute for either monitored
events or table view requests, you can hardcode a value such as ‘0’ for the minute
column. Suppose an entry from such a log has the following format:
MSG123 2005 Mar 6 10 pm Text of event ...

The following format command will hardcode a value of zero for the entry time
minute for every event:
A,“%s %d %s %d %d %s%c %[^\n]” , de ye mo da ho ho min=“0”
desc=“:%s”

The Tivoli Enterprise Portal Log Entries table view would contain the following in
the Entry Time and Description columns. (If omitted, seconds defaults to zero.)

Entry Time Description

03/06/05 22:00:00 Text of event

To include a mapping specifier for the ‘minute’ column, there must be an

associated scan directive. However, we are trying to hardcode a value of zero for
this column; the data consumed by the associated scan directive will not be used
and is, therefore, totally arbitrary.

In this example, a dummy scan directive is supplied ‘%c’ that consumes the single
space character between the ‘pm’ and the start of the actual message. Since this
space was going to be discarded anyway and does not affect the data in interest,
this scan directive’s sole purpose is to allow the inclusion of a ‘minute’ column
mapping specifier in which a ‘0’ character is forced to be displayed.

Defaulting the year entry-time component

Many logs omit the year from their event entry times. For example, the following
sample was taken from a syslog.
Mar 17 03:34:11 frodo unix: NFS server gandalf not responding

When monitoring such logs, the monitoring agent sets the entry-time year
component for each new event to the current system-clock year as described in
“Hardcoding missing entry time components.”

When handling table view requests for logs that do not include the entry-time
year, the monitoring agent attempts to determine the year of an event based on the
date in the next entry. This causes the monitoring agent to make an assumption
that a monitored log has never been inactive for a period one year or longer. To
show how this works, suppose two entries from a syslog are as follows. (New log
entries are appended to the end of the log so the entry dated December 31st is
older than that dated January 1st.)
58 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Dec 31 23:34:11 bilbo unix: NFS server gandalf not responding
Jan 1 03:34:11 frodo unix: NFS write error on host bilbo

Further, suppose that the current date is March 15th, 2005. If you issued a table
view request and specified in the time span dialog a time range of December 31st,
2004 at 11:00 p.m. to January 1st, 2005 at 4:00 a.m., the above entries are displayed
(in reverse chronological order) in the Log Entries table view as follows:

Entry Time Description

01/01/05 03:34:11 NFS server gandalf not responding
12/31/04 23:34:11 NFS write error on host bilbo

Appendix A. Generic user log support 59

60 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Appendix B. Tuning format commands with the kulmapper
utility
You can use the kulmapper utility to create and fine-tune format commands.

Running the utility invokes the same code as that used by the Monitoring Agent
for UNIX Logs to monitor user logs. This means that a format command that maps
a user log as desired when passed to kulmapper will also work correctly when
used by the Monitoring Agent for UNIX Logs to monitor that log. Conversely, if a
format command is passed to the utility to format a given user log, any errors in
the format command will produce exactly the same messages as those generated
by the monitoring agent if it were given the same format command to monitor the
same log.

It is easier to build and test format commands using the utility instead of the
Monitoring Agent for UNIX Logs for several reasons:
v The format command used by kulmapper is included in the same file that the
utility reads that contains your sample user log entries. This is of benefit not
only because it is easier to create a format command while you can see actual
log entries but also because it simplifies the task of managing multiple sample
user log files and their corresponding format commands.
v Using sample user log files means you can edit the log entries and play ″what
if″ scenarios to ensure your format command can handle the different messages
that can be written to the log.
v All messages written by kulmapper are sent to the standard output device,
which, by default, is your terminal. The monitoring agent, on the other hand,
sends any syntax and formatting error messages to its RAS log. If formatting is
successful, the results must be viewed in the Log Entries table view of Tivoli
Enterprise Portal.
v The default kulmapper RAS tracing options cause each line read from the
sample log file, including the format command itself, to be displayed followed
by the results of formatting and mapping that entry. This simplifies the task of
verifying that each sample log entry was formatted as expected. If an entry is
encountered that cannot be formatted, kulmapper stops after displaying the log
entry that caused the error and the error message describing the problem.
v After you have updated the format command in the kulmapper sample log file,
you test it by simply invoking the utility again and observing the results on
your terminal. Conversely, after updating a format command in the monitoring
agent’s configuration file, it is necessary to send the monitoring agent a refresh
signal so that it can learn and use the new format.

Using the kulmapper utility you can significantly reduce the time required
performing each ″change-test-observe″ cycle as you tune format commands.

© Copyright IBM Corp. 2005 61

Using the kulmapper utility
The kulmapper utility is invoked using a script called kulmapper that is located in
the install_dir/bin directory. The script accepts three parameters as follows:

kulmapper [install_dir] [filename]

All parameters are optional.

-h install_dir
The name of the top-level directory in which you installed the monitoring
agent.
-l log_filename
The name of a file that contains a format command and one or more
sample entries from your user log.

If unspecified, the input file defaults to install_dir/config/kulmapper.samp and the

number of entries to format defaults to 1. Here is a sample file that can be used by
kulmapper:
a,"%9s%c/%d %s %d/%d %d:%d:%d %s %s :%[^\n]" , desc type class =
"RC = %x" month day year hour min sec hour source desc = " %s"
MSG123456I/1024 Oct 04/05 11:15:32 am region1 : Application alpha
started
MSG234567W/2048 Oct 04/05 1:01:31 pm region2 : No journal files
opened
MSG345678E/4096 Oct 04/05 2:57:02 pm region1 : Unable to open file
’FILE1’

To see how the utility operates, simply change to the install_dir/bin directory and
enter the script name, kulmapper, at the command prompt without any
parameters. This formats and maps the first sample log entry in the file according
to the supplied format command.

For example, if your installation directory is /myinstall_dir, enter the following:

cd /myinstall_dir
kulmapper
a,"%9s%c/%d %s %d/%d %d:%d:%d %s %s :%[^\n]" , desc type class =
"RC = %x" month day year hour min sec hour source desc = " %s"
MSG123456I/1024 Oct 04/05 11:15:32 am region1: Application alpha
started
year: 05
month: Oct
day: 4
hour: 11am
minute: 15
second: 32
system:
source: region1
type: I
class: RC = 400

The contents of install_dir/config/kulmapper.samp is based on “Monitoring user

and third-party vendor applications” on page 19 and indicates the required format
of the kulmapper input file.

62 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Analyzing User log files and testing format commands
Before using kulagent or kulmapper, analyze the log file that you want to monitor
to determine how you want the data to be mapped to the Monitoring Agent for
UNIX Logs tables. The name of each attribute in the Monitoring Agent for UNIX
Logs tables is documented in Chapter 5, “Attributes reference,” on page 23. When
you have completed your analysis, write a format command, create a test log file
with the format command as the first record, and use kulmapper to test the
command.

For example, perform the following steps:

1. Locate a sample of the log file that you want to monitor and analyze each field
of the log file.
For example, your log file contains a message like the one following:
ERZ010117I/0150 11/01/05 10:20:51 xcicsol1 : CICS is performing a cold
start
2. Determine which field you want mapped to each mapping name.
The Monitoring Agent for UNIX Logs mapping names are: year, month, day,
hour, minute, second, type, system, source, class and description. In the log file
record example, map the date/time stamp to the date/time mapping names.
Next, let’s map the message number ERZ010117I/0150 to the type mapping
name. Next, map the CICS® name to the system mapping name, and the
remainder of the message to the description mapping name. Finally, the
remaining mapping names, source and class are not used, so the monitoring
agent will assign them blank values.
3. Write your data mapping specification.
From our analysis of the example, the mapping specification is:
Type month day year hour minute second system description
This is the order in which this data appears in the log record, so specify them
in that order.
4. Determine the character type of each field.
Basically, the character type will be a character string or a numeric string, and
you need to identify which type applies to each mapping name. For example, a
date can be 01/01/05 or 01 January, 2005, so the month can be numeric or
character. In the example, the following are included:
v ERZ010117I/0150 = character string
v 11/01/05 = numeric/numeric/numeric. Ignore the “/” character when the
data is mapped.
v 10:20:51 = numeric:numeric:numeric. Ignore the “:” character.
v xcicsol1 = character string
v : Ignore the colon that follows the system name.
v Keep the remainder of the message.
5. Write the format description.
Using the example, the format description is:
"%s %d/%d/%d %d:%d:%d %s : %[^\n]"
Note that the literal characters “/” and “:” are included in the format
description. This tells the Monitoring Agent for UNIX Logs to skip over them
and omit them from the mapping variable’s value. The format scan set
“%[^\n]” specifies that the data is character data and to include all of it up to
the new line character at the end of the log record.

Appendix B. Tuning format commands with the kulmapper utility 63

 6. Combine the mapping specification and the format description into a format
command.
The final format command for the example looks like this:
a,"%s %d/%d/%d %d:%d:%d %s : %[^\n]", type month day year hour
minute second system descr
7. Build the kulmapper test file and test.
All that remains to test the format command is to copy it as the first record of
the log file that you are analyzing, and test it with kulmapper.

64 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Appendix C. IBM Tivoli Enterprise Console event mapping
Each event class corresponds to an attribute group in the monitoring agent. For a
description of the event slots for each event class, see Table 21. For more
information about mapping attribute groups to event classes, see the IBM Tivoli
Monitoring Administrator’s Guide.

Generic event mapping provides useful event class and attribute information for
situations that do not have specific event mapping defined. BAROC files are found
on the Tivoli Enterprise Monitoring Server in the installation directory in TECLIB
(that is, install_dir/cms/TECLIB for Windows systems and
install_dir/tables/TEMS_hostname/TECLIB for UNIX systems). IBM Tivoli
Enterprise Console event synchronization provides a collection of ready-to-use rule
sets that you can deploy with minimal configuration. Be sure to install IBM Tivoli
Enterprise Console event synchronization to access the correct Sentry.baroc, which
is automatically included during base configuration of IBM Tivoli Enterprise
Console rules if you indicate that you want to use an existing rulebase. See the
IBM Tivoli Monitoring Installation and Setup Guide for details.

Each of the event classes is a child of KUL_Base. The KUL_Base event class can be
used for generic rules processing for any event from the Monitoring Agent for
UNIX Logs.
Table 21. Overview of event slots to event classes
IBM Tivoli Enterprise Console event class event slots
ITM_Monitored_Logs Monitored_Logs attribute group
v managed_system: STRING
v log_path: STRING
v log_name: STRING
v log_type: STRING
v monitor_status: STRING
v monitor_start_per_stop_time: STRING
v number_of_events: INTEGER
v number_of_format_errors: INTEGER
v log_size: INTEGER
v date_last_modified: STRING
v debug_mode: STRING
v format_command: STRING
v timestamp: STRING
v log_path_u: STRING
v log_name_u: STRING

© Copyright IBM Corp. 2005 65

 Table 21. Overview of event slots to event classes (continued)
IBM Tivoli Enterprise Console event class event slots
ITM_Log_Entries Log_Entries attribute group
v smanaged_system: STRING
v log_path: STRING
v log_name: STRING
v entry_time: STRING
v system: STRING
v kul_source: STRING
v type: STRING
v class: STRING
v description: STRING
v frequency_threshold: INTEGER
v period_threshold: INTEGER
v timestamp: STRING
v log_path_u: STRING
v log_name_u: STRING
v source_u: STRING
v description_u: STRING

66 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Appendix D. Problem determination
This appendix explains how to troubleshoot the IBM Tivoli Monitoring: UNIX Log
Agent. Troubleshooting, or problem determination, is the process of determining
why a certain product is malfunctioning.

Note: You can resolve some problems by ensuring that your system matches the
system requirements listed in Chapter 2, “Requirements and configuration
for the monitoring agent,” on page 5.

This appendix provides agent-specific problem determination information. See the

IBM Tivoli Monitoring Problem Determination Guide for general problem
determination information. Also see “Support information” on page 83 for other
problem-solving options.

Gathering product information for IBM Software Support

Before contacting IBM Software Support about a problem you are experiencing
with this product, gather the following information that relates to the problem:
Table 22. Information to gather before contacting IBM Software Support
Information type Description
Log files Collect trace log files from failing systems. Most logs are located in a logs subdirectory
on the host computer. See “Trace logging” on page 68 for lists of all trace log files and
their locations. See the IBM Tivoli Monitoring User’s Guide for general information about
the IBM Tivoli Monitoring environment.
UNIX logs information v Version number and patch level
v Sample application data file (if monitoring a file)
v Metafile (if problem is missing or invalid data in a workspace and the problem
originates in an application that you are monitoring)
Operating system Operating system version number and patch level
Messages Messages and other information displayed on the screen
Version numbers for Version number of the following members of the monitoring environment:
IBM Tivoli Monitoring v IBM Tivoli Monitoring. Also provide the patch level, if available.
v IBM Tivoli Monitoring: UNIX Log Agent
Screen captures Screen captures of incorrect output, if any.
(UNIX only) Core dump If the system stops on UNIX systems, collect core dump file from install_dir/bin directory,
files where install_dir is the directory path where you installed the monitoring agent.

Upload files for review to the following FTP site: ftp.emea.ibm.com. Log in as
anonymous and place your files in the directory that corresponds to the IBM Tivoli
Monitoring component that you use. See “Contacting IBM Software Support” on
page 84 for more information about working with IBM Software Support.

Built-in problem determination features

The primary troubleshooting feature in the IBM Tivoli Monitoring: UNIX Log
Agent is logging. Logging refers to the text messages and trace data generated by
the IBM Tivoli Monitoring: UNIX Log Agent and is always enabled. Messages and
trace data are sent to the files listed in Table 23 on page 70.

© Copyright IBM Corp. 2005 67

 Trace data captures transient information about the current operating environment
when a component or application fails to operate as designed. IBM Software
Support personnel use the captured trace information to determine the source of
an error or unexpected condition. See “Trace logging” for more information.

Problem classification
The following types of problems might occur with the IBM Tivoli Monitoring:
UNIX Log Agent:
v Installation and configuration
v General usage and operation
v Display of monitoring data

This appendix provides symptom descriptions and detailed workarounds for these
problems, as well as describing the logging capabilities of the monitoring agent.
See the IBM Tivoli Monitoring Problem Determination Guide for general problem
determination information.

Trace logging
Trace logs capture information about the operating environment when component
software fails to operate as intended. The principal log type is the RAS (Reliability,
Availability, and Serviceability) trace log. These logs are in the English language
only. The RAS trace log mechanism is available for all components of IBM Tivoli
Monitoring. Most logs are located in a logs subdirectory on the host computer. See
the following sections to learn how to configure and use trace logging:
v “Principal trace log files” on page 69
v “Examples: using trace logs” on page 71
v “Setting RAS trace parameters” on page 72

Note: The documentation refers to the RAS facility in IBM Tivoli Monitoring as
″RAS1″.

Typically, IBM Software Support applies specialized knowledge to analyze trace

logs to determine the source of problems. However, you can open trace logs in a
text editor such as vi to learn some basic facts about your IBM Tivoli Monitoring
environment as described in “Examples: using trace logs” on page 71.

Overview of log file management

Table 23 on page 70 provides the names, locations, and descriptions of RAS1 log
files. The log file names adhere to the following naming convention:
hostname_product_program_timestamp-nn.log

where:
v hostname is the host name of the machine on which the monitoring component is
running.
v product is the two-character product code. For Monitoring Agent for UNIX Logs,
the product code is ul.
v program is the name of the program being run.
v timestamp is an 8-character hexadecimal timestamp representing the time at
which the program started.

68 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 v nn is a rolling log suffix. See “Examples of trace logging” for details of log
rolling.

Examples of trace logging

For example, if a UNIX logs monitoring agent is running on computer ″server01″,
the RAS log file for the Monitoring Agent for UNIX Logs might be named as
follows:
server01_ul_kulagent_437fc59-01.log

For long-running programs, the nn suffix is used to maintain a short history of log
files for that startup of the program. For example, the kulagent program might
have a series of log files as follows:
server01_ul_kulagent_437fc59-01.log
server01_ul_kulagent_437fc59-02.log
server01_ul_kulagent_437fc59-03.log

As the program runs, the first log (nn=01) is preserved because it contains program
startup information. The remaining logs ″roll." In other words, when the set of
numbered logs reach a maximum size, the remaining logs are overwritten in
sequence.

Each time a program is started, a new timestamp is assigned to maintain a short

program history. For example, if the Monitoring Agent for UNIX Logs is started
twice, it might have log files as follows:
server01_ul_kulagent_437fc59-01.log
server01_ul_kulagent_437fc59-02.log
server01_ul_kulagent_437fc59-03.log

server01_ul_kulagent_537fc59-01.log
server01_ul_kulagent_537fc59-02.log
server01_ul_kulagent_537fc59-03.log

Each program that is started has its own log file. For example, the Monitoring
Agent for UNIX Logs would have agent logs in this format:
server01_ul_kulagent_437fc59-01.log

Other logs have a similar syntax as in the following example:

server01_ul_kulmapper_447fc59-01.log

where kulmapper is the program name.

Note: When you communicate with IBM Software Support, you must capture and
send the RAS1 log that matches any problem occurrence that you report.

Principal trace log files

Table 23 on page 70 contains locations, file names, and descriptions of trace logs
that can help determine the source of problems with agents.

Appendix D. Problem determination 69

Table 23. Trace log files for troubleshooting agents
System where log File name and path Description
is located
On the computer The RAS1 log files are named Traces activity of the monitoring agent.
that hosts the hostname_ul_program_timestamp-nn.log and are Note: Other RAS1 logs have a similar
monitoring agent located in the install_dir/logs path. syntax and are located in this directory
Note: File names for RAS1 logs include a path.
See “Definitions of hexadecimal time stamp.
variables” on page
71 for descriptions Also on UNIX, a log with a decimal time stamp
of the variables in is provided: hostname_ul_timestamp.log and
the file names in hostname_ul_timestamp.pidnnnnn in the
column two. install_dir/logs path, where nnnnn is the
process ID number.
The *.LG0 file is located in the install_dir/logs A new version of this file is generated
path. every time the agent is restarted. IBM Tivoli
Monitoring generates one backup copy of
the *.LG0 file with the tag .LG1. View .LG0
to learn the following details regarding the
current monitoring session:
v Status of connectivity with the
monitoring server.
v Situations that were running.
v The success or failure status of Take
Action commands.
On the Tivoli On UNIX: The candle_installation.log file in the Provides details about products that are
Enterprise install_dir/logs path. installed.
Monitoring Server Note: Trace logging is enabled by default.
On Windows: The file in the A configuration step is not required to
See “Definitions of install_dir\InstallITM path. enable this tracing.
variables” on page
71 for descriptions The Warehouse_Configuration.log file is located Provides details about the configuration of
of the variables in in the following path on Windows: data warehousing for historical reporting.
the file names in install_dir\InstallITM.
column two. The RAS1 log file is named Traces activity on the monitoring server.
hostname_ms_timestamp-nn.log and is located in
the following path:
v On Windows: install_dir\logs
v On UNIX: install_dir/logs
Note: File names for RAS1 logs include a
hexadecimal time stamp

Also on UNIX, a log with a decimal time stamp

is provided: hostname_ms_timestamp.log and
hostname_ms_timestamp.pidnnnnn in the
install_dir/logs path, where nnnnn is the
process ID number.

70 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 23. Trace log files for troubleshooting agents (continued)
System where log File name and path Description
is located
On the Tivoli The RAS1 log file is named Traces activity on the portal server.
Enterprise Portal hostname_cq_timestamp-nn.log and is located in
Server the following path:
v On Windows: install_dir\logs
See “Definitions of
variables” for v On UNIX: install_dir/logs
descriptions of the Note: File names for RAS1 logs include a
variables in the hexadecimal time stamp
file names in
column two. Also on UNIX, a log with a decimal time stamp
is provided: hostname_cq_timestamp.log and
hostname_cq_timestamp.pidnnnnn in the
install_dir/logs path, where nnnnn is the process ID
number.
The TEPS_ODBC.log file is located in the following When you enable historical reporting, this
path on Windows: install_dir\InstallITM. log file traces the status of the warehouse
proxy agent.
Definitions of variables for RAS1 logs:
v hostname is the host name of the machine on which the agent is running.
v install_dir represents the directory path where you installed the IBM Tivoli Monitoring component. install_dir can
represent a path on the computer that hosts the monitoring server, the monitoring agent, or the portal server.
v product is the two character product code. For Monitoring Agent for UNIX Logs, the product code is ul.
v program is the name of the program being run.
v timestamp is an eight-character hexadecimal time stamp representing the time at which the program started.
v nn is a rolling log suffix. See “Examples of trace logging” on page 69 for details of log rolling.

See the IBM Tivoli Monitoring Installation and Setup Guide for more information on
the complete set of trace logs that are maintained on the monitoring server.

Examples: using trace logs

Typically IBM Software Support applies specialized knowledge to analyze trace
logs to determine the source of problems. However, you can open trace logs in a
text editor such as vi to learn some basic facts about your IBM Tivoli Monitoring
environment. You can use the ls -ltr command to list the log files in the
install_dir/logs directories, sorted by time they were last updated.
Example one
This excerpt shows the typical log for a failed connection between a
monitoring agent and a monitoring server with the host name server1a:
(Thursday, August 11, 2005, 08:21:30-{94C}kdcl0cl.c,105,"KDCL0_ClientLookup") status=1c020006,
"location server unavailable", ncs/KDC1_STC_SERVER_UNAVAILABLE
(Thursday, August 11, 2005, 08:21:35-{94C}kraarreg.cpp,1157,"LookupProxy") Unable to connect to
broker at ip.pipe:: status=0, "success", ncs/KDC1_STC_OK
(Thursday, August 11, 2005, 08:21:35-{94C}kraarreg.cpp,1402,"FindProxyUsingLocalLookup") Unable
to find running CMS on CT_CMSLIST <IP.PIPE:#server1a>
Example two
The following excerpts from the trace log for the monitoring server show the
status of an agent, identified here as ″Remote node.″ The name of the
computer where the agent is running is SERVER5B:
(42C039F9.0000-6A4:kpxreqhb.cpp,649,"HeartbeatInserter") Remote node SERVER5B:KUL is ON-LINE.
. . .
(42C3079B.0000-6A4:kpxreqhb.cpp,644,"HeartbeatInserter") Remote node SERVER5B:KUL is OFF-LINE.

Key points regarding the preceding excerpt:

Appendix D. Problem determination 71

 v The monitoring server appends the KUL product code to the server
name to form a unique name (SERVER5B:KUL) for this instance of
Monitoring Agent for UNIX Logs. This unique name enables you to
distinguish multiple monitoring products that might be running on
SERVER5B.
v The log shows when the agent started (ON-LINE) and later stopped
(OFF-LINE) in the environment.
v For the sake of brevity an ellipsis (...) represents the series of trace log
entries that were generated while the agent was running.
v Between the ON-LINE and OFF-LINE log entries, the agent was
communicating with the monitoring server.
v The ON-LINE and OFF-LINE log entries are always available in the
trace log. All trace levels that are described in “Setting RAS trace
parameters” provide these entries.

Setting RAS trace parameters

Objective
Pinpoint a problem by setting detailed tracing of individual components of the
monitoring agent and modules.

Background Information
Monitoring Agent for UNIX Logs uses RAS1 tracing and generates the logs
described in Table 23 on page 70. The default RAS1 trace level is ERROR.

RAS1 tracing has control parameters to manage to the size and number of RAS1
logs. Use the procedure described in this section to set the parameters.

Note: The KBB_RAS1_LOG parameter also provides for the specification of the
log file directory, log file name, and the inventory control file directory and
name. Do not modify these values or log information can be lost.

Before you begin

See “Overview of log file management” on page 68 to ensure that you understand
log rolling and can reference the correct log files when you managing log file
generation.

After you finish

Monitor the size of the logs directory. Default behavior can generate a total of 45 to
60 MB for each agent that is running on a computer. For example, each database
instance that you monitor could generate 45 to 60 MB of log data. See the
″Procedure″ section to learn how to adjust file size and numbers of log files to
prevent logging activity from occupying too much disk space.

Regularly prune log files other than the RAS1 log files in the logs directory. Unlike
the RAS1 log files which are pruned automatically, other log types can grow
indefinitely, for example, the logs in Table 23 on page 70 that include a process ID
number (PID).

Consider using collector trace logs (described in Table 23 on page 70) as an

additional source of problem determination information.

72 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 Note: The maximum error tracing setting can generate a large amount of trace
logging. Use them only temporarily, while you are troubleshooting
problems. Otherwise, the logs can occupy excessive amounts of hard disk
space.

Procedure
Specify RAS1 trace options in the install_dir/config/ul.ini file. You can
manually edit the configuration file to set trace logging:
1. Open the trace options file: /install_dir/config/ul.ini.
2. Edit the line that begins with KBB_RAS1= to set trace logging preferences.
For example, if you want detailed trace logging, set the Maximum Tracing
option:
export KBB_RAS1=’ERROR (UNIT:kul ALL) (UNIT:kra ALL)’
3. Edit the line that begins with KBB_RAS1_LOG= to manage the generation of
log files:
v Edit the following parameters to adjust the number of rolling log files and
their size.
– MAXFILES: the total number of files that are to be kept for all startups of
a given program. Once this value is exceeded, the oldest log files are
discarded. Default value is 9.
– LIMIT: the maximum size, in megabytes (MB) of a RAS1 log file. Default
value is 5.
v IBM Software Support might guide you to modify the following parameters:
– COUNT: the number of log files to keep in the rolling cycle of one
program startup. Default value is 3.
– PRESERVE: the number of files that are not to be reused in the rolling
cycle of one program startup. Default value is 1.

Note: The KBB_RAS1_LOG parameter also provides for the specification of

the log file directory, log file name, and the inventory control file
directory and name. Do not modify these values or log information can
be lost.
4. Restart the monitoring agent so that your changes take effect.

Problems and workarounds

The following sections provide symptoms and workarounds for problems that
might occur with Monitoring Agent for UNIX Logs:
v “Installation and configuration problem determination” on page 73
v “Problem determination for remote deployment” on page 78
v “Situation problem determination” on page 79

Note: You can resolve some problems by ensuring that your system matches the
system requirements listed in Chapter 2, “Requirements and configuration
for the monitoring agent,” on page 5.
This appendix provides agent-specific problem determination information. See the
IBM Tivoli Monitoring Problem Determination Guide for general problem
determination information.

Installation and configuration problem determination

This section provides tables that show solutions for installation, configuration, and
uninstallation problems.

Appendix D. Problem determination 73

 Unique troubleshooting approach for Monitoring Agent for UNIX
Logs
Configuration of Monitoring Agent for UNIX Logs depends on a unique
configuration file (kul_configfile) that is located on each computer that hosts the
agent software. This file specifies the following details for the monitoring of logs:
v Which log files to target for monitoring
v How to parse each line in a log
v How to map parsed strings into the database
This configuration step is unique to Monitoring Agent for UNIX Logs. Typically
you specify the details of monitoring in the Situation Editor of the Tivoli Enterprise
Portal, and you benefit from the features of the Situation Editor and the
verification cues that you can see in the portal. These features and cues are not all
available for troubleshooting Monitoring Agent for UNIX Logs. Instead you have
the unique troubleshooting approaches that are described in the following sections
of this document:
v Chapter 2, “Requirements and configuration for the monitoring agent,” on page
5
– “Specifying the log files to monitor” on page 6
– “Customer configuration file” on page 6
– “Customer configuration file format” on page 7
– “Syslog daemon configuration file” on page 7
– “Environment variables for the Monitoring Agent for UNIX Logs” on page 8
– “Environment variable syntax” on page 9
– “Dynamically refreshing the monitoring agent” on page 9
v Appendix B, “Tuning format commands with the kulmapper utility,” on page 61
v Check the format strings that specify the log files, data, and fields that you
monitor.
v Location defined by environment KUL_CONFIG_FILE variable described in
Chapter 2, “Requirements and configuration for the monitoring agent,” on page
5.
v View Log workspace in Portal. Appendix B, “Tuning format commands with the
kulmapper utility,” on page 61 describe how to set up the portal for this
purpose.

Note: See Table 24 on page 75 to learn about a problem that affects users who
have a previous version of Monitoring Agent for UNIX Logs.

74 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 24. Problems and solutions for installation and configuration
Problem Solution
When you upgrade to IBM Tivoli v Scenario 1, you upgrade monitoring agents: Fixpacks for Candle,
Monitoring, you might need to apply Version 350, are delivered as each monitoring agent is upgraded to
fixpacks to Candle, Version 350, agents. IBM Tivoli Monitoring.
Note: The IBM Tivoli Monitoring download image or CD provides
application fixpacks for the monitoring agents that are installed
from that CD (for example, the agents for operating systems such
as Windows, Linux, UNIX, and i5/OS). The upgrade software for
other agents is located on the download image or CDs for that
specific monitoring agent, such as the agents for database
applications.
v Scenario 2, you do not upgrade monitoring agents: If you do not
upgrade the monitoring agent to IBM Tivoli Monitoring, the agent
continues to work. However, you must upgrade to have all the
functionality that IBM Tivoli Monitoring offers.
Note: You might have to install fixpacks for 350 agents that you
choose not to upgrade to IBM Tivoli Monitoring. Likewise, you
might have to install fixpacks for any 350 agents that do not have
an equivalent in IBM Tivoli Monitoring.

Presentation files and customized The upgrade from version 350 to IBM Tivoli Monitoring handles
Omegamon DE screens for Candle export of the presentation files and the customized Omegamon DE
monitoring agents need to be upgraded to a screens.
new Linux on z/Series system.
(UNIX only) During a command-line The system prompts you to ignore the warning and re-install the
installation, you choose to install a component (″Yes″) or to stop installation of the component (″No″).
component that is already installed, and v If you select, ″Yes,″ you overwrite the current installation of the
you see the following warning: component.
WARNING - you are about to install Note: If you had previously applied a fixpack or other modification
the SAME version of "component" to the component, those changes would be overwritten.
v If you select, ″No,″ you must exit and restart the installation
where component is the name of the process. You cannot return to the list where you selected
component that you are attempting to components to install. When you run the installer again, do not
install. attempt to install any component that is already installed, unless
you want the installer to overwrite it.
(Monitoring Agent for UNIX Logs only) The This problem occurs because the installer overwrites a pre-existing
install_dir/config/kul_configfile copy of the kul_configfile file. Retrieve the prior version of the
configuration file is empty following the kul_configfile file from your archives or recreate the file from scratch.
installation of a new version of the Log Note: You should rename the file or copy it to another location before
Agent. Settings created for a previous upgrading to ITM 6.1.
version of this agent are lost.
The product fails to do a monitoring The monitoring agent must have the permissions necessary to
activity that requires read, write, or execute perform requested actions. For example, if the user ID you used to log
permissions. For example, the product onto the system to install the monitoring agent (locally or remotely)
might fail to read a log. does not have the permission to perform a monitoring operation (such
as running a command), the monitoring agent is not able perform the
operation.
While installing the agent from a CD, the This error is caused by low disk space. Although the install.sh script
following message is displayed and you are indicates that it is ready to install the agent software, the script
not able to continue the installation: considers the size of all tar files, not the size of all the files that are
install.sh warning: unarchive of contained within the tar file.Run the df -k command to check whether
"/cdrom/unix/cienv1.tar" may the file systems have enough space to install agents.
have failed

Appendix D. Problem determination 75

Table 24. Problems and solutions for installation and configuration (continued)
Problem Solution
Installing as root: The product has been When you install the product as root the files in the install_dir
installed as root, which is not directory are owned by root. You must change the status of the files
recommended. Without re-installing the as follows:
product, how can you change from root to 1. While logged on as root, run the install_dir/bin/UnSetRoot
a different user account? script, as in this example:
UnSetRoot [ -h CANDLEHOME ] userID
The script resets all the files under the install_dir directory.
2. Run the install_dir/bin/SetPerm command. SetPerm sets root
permission for specific IBM Tivoli Monitoring agent files.

About installing as root: Normally, do not use the root user account
to install or to start the Monitoring Agents for UNIX, for Linux, and
for UNIX Logs. If you use the root user account to install the product,
the files do not receive the correct permissions, and product behavior
is unpredictable.

To create a stable installation of the product, use one of the following

options:
v Create a user account with all the authority and permissions to
install and run commands. For example, create a tivoli user
account.
—OR—
v Use any user account other than root that has the required
authority and permissions.
Cannot locate the KDCB0_HOSTNAME Go to install_dir/config and edit the corresponding .ini file. Set the
setting. KDCB0_HOSTNAME parameter followed by the IP address. If you
use multiple network interface cards (NICs), give the primary IP
address of the network interface.
The Monitoring Agent for UNIX Logs You can collect data to analyze this problem as follows:
repeatedly restarts. 1. Access the install_dir/config/ul.ini file, which is described in
“Setting RAS trace parameters” on page 72.
2. Add the following line: KBB_SIG1=trace –dumpoff
Agents in the monitoring environment use Configure both the monitoring server and the Warehouse proxy server
different communication protocols. For to accept multiple protocols, as described in the IBM Tivoli Monitoring
example, some agents have security enabled Installation and Setup Guide.
and others do not.
Creating a firewall partition file: The How it works: When the agents start, they search
partition file enables an agent to connect to KDCPARTITION.TXT for the following matches:
the monitoring server through a firewall. v An entry that matches the partition name OUTSIDE.
v An entry that also includes a valid external address.
For more information, see the IBM Tivoli Monitoring Installation and
Setup Guide.
You see the following error: Confirm that the password within the Tivoli Enterprise Monitoring
Hub not registered with location Server is correct.
broker. Error-code 1195.

76 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 24. Problems and solutions for installation and configuration (continued)
Problem Solution
The agent is started and running but not Perform the following steps:
displaying data in the Tivoli Enterprise 1. Check the UNIX agent log files to see whether there are network
Portal. connectivity problems.
2. If there are no connection problems, check whether the agent has
terminated.
3. If the agent is not terminated, confirm that you have added
application support for the Monitoring Agent for UNIX in the
Tivoli Enterprise Monitoring Server as described in IBM Tivoli
Monitoring Installation and Setup Guide.
The system experiences high CPU usage View the memory usage of the kulagent process. If CPU usage seems
after you install or configure Monitoring to be excessive, recycle the monitoring agent.
Agent for UNIX Logs.
You see the following message: KFWITM083W You see this message because some links do not have default
Default link is disabled for the workspaces. Right-click the link to access a list of workspaces to
selected object; please verify link and select.
link anchor definitions.
When you edit the configuration for an The original configuration settings might include non-ASCII
existing monitoring agent, the values characters. These values were stored incorrectly and result in the
displayed are not correct. incorrect display. Enter new values using only ASCII characters.

Table 25. General problems and solutions for uninstallation

Problem Solution
On Windows, uninstallation of IBM Tivoli Be sure that you follow the general uninstallation process described in
Monitoring fails to uninstall the entire the IBM Tivoli Monitoring Installation and Setup Guide:
environment. 1. Uninstall monitoring agents first, as in the following examples:
v Uninstall a single monitoring agent for a specific database.
—OR—
v Uninstall all instances of a monitoring product, such as IBM
Tivoli Monitoring for Databases.
2. Uninstall IBM Tivoli Monitoring.
The way to remove inactive managed When you want to remove a managed system from the navigation
systems (systems whose status is OFFLINE) tree, right-click the item that you want to remove, and select Remove
from the Enterprise navigation tree in the managed system.
portal is not obvious.

Unique names for monitoring components

IBM Tivoli Monitoring might not be able to generate a unique name for monitoring
components due to the truncation of names that the product automatically
generates.

IBM Tivoli Monitoring automatically creates a name for each monitoring

component by concatenating the subsystem name, host name, and product code
separated by colons (subsystem_name:hostname:KUL).

Note: When you monitor a multinode system, such as a database, IBM Tivoli
Monitoring adds a subsystem name to the concatenated name, typically a
database instance name.
The length of the name that IBM Tivoli Monitoring generates is limited to 32

Appendix D. Problem determination 77

 characters. Truncation can result in multiple components having the same
32-character name. If this problem happens, shorten the hostname portion of the
name as follows:
1. Open the configuration file for the monitoring agent:
install_dir/config/ul.ini.
2. Find the line the begins with CTIRA_HOSTNAME=.
3. Type a new name for host name that is a unique, shorter name for the host
computer. The final concatenated name including the subsystem name, new
host name, and KUL, cannot be longer than 32 characters.

Note: You must ensure that the resulting name is unique with respect to any
existing monitoring component that was previously registered with the
Tivoli Enterprise Monitoring Server.
4. Save the file.
5. Restart the agent.
6. If you do not find the files mentioned in Step 1, perform the workarounds
listed in the next paragraph.

If you do not find the files mentioned in the preceding steps, perform the
following workarounds:
1. Change CTIRA_HOSTNAME environment variable in the configuration file of
the monitoring agent.
v Find the KULENV file in the same path mentioned in the preceding row.
v For z/OS agents, find the RKANPAR library.
v For i5/OS agents, find the QAUTOTMP/KMSPARM library in member
KBBENV.
2. If you cannot find the CTIRA_HOSTNAME environment variable, you must
add it to the configuration file of the monitoring agent:
v On Windows: Use the Advanced > Edit Variables option.
v On UNIX and Linux: Add the variable to the config/product_code.ini and to
config/product_code.config files.
v On z/OS: Add the variable to the RKANPAR library, member
Kproduct_codeENV.
v On i5/OS: Add the variable to the QAUTOTMP/KMSPARM library in
member KBBENV.
3. Some monitoring agents (for example, the monitoring agent for MQ Series) do
not reference the CTIRA_HOSTNAME environment variable to generate
component names. Check the documentation for the monitoring agent that you
are using for information on name generation. If necessary, contact IBM
Software Support.

Problem determination for remote deployment

Table 26 on page 79 lists problems that might occur with remote deployment. This
appendix provides agent-specific problem determination information. See the IBM
Tivoli Monitoring Problem Determination Guide for general problem determination
information.

This section describes problems and solutions for remote deployment and removal
of agent software Agent Remote Deploy:

78 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Table 26. Remote deployment problems and solutions
Problem Solution
While you are using the remote deployment feature Do not close or modify this window. It is part of the
to install Monitoring Agent for UNIX Logs, an installation process and will be dismissed automatically.
empty command window is displayed on the target
computer. This problem occurs when the target of
remote deployment is a Windows computer. (See the
IBM Tivoli Monitoring Installation and Setup Guide for
more information on the remote deployment
feature.)
The removal of a monitoring agent fails when you This problem might happen when you attempt the remote
use the remote removal process in the Tivoli removal process immediately after you have restarted the
Enterprise Portal desktop or browser. Tivoli Enterprise Monitoring Server. You must allow time for
the monitoring agent to refresh its connection with the Tivoli
Enterprise Monitoring Server before you begin the remote
removal process.

Situation problem determination

This section provides information about both general situation problems and
problems with the configuration of situations. Some of the problems are generally
valid, but do not apply to Monitoring Agent for UNIX Logs. See the IBM Tivoli
Monitoring Problem Determination Guide for more information about problem
determination for situations.

General situation problems

Table 27 lists problems that might occur with specific situations.
Table 27. Specific situation problems and solutions
Problem Solution
You want to change the appearance of 1. Right-click an item in the Navigation tree.
situations when they are displayed in a
2. Select Situations in the pop-up menu. The Situation Editor window is
Workspace view.
displayed.
3. Select the situation that you want to modify.
4. Use the Status pull-down menu in the lower right of the window to
set the status and appearance of the Situation when it triggers.
Note: This status setting is not related to severity settings in IBM
Tivoli Enterprise Console.
Monitoring activity requires too much Check the RAS trace logging settings that are described in “Setting RAS
disk space. trace parameters” on page 72. For example, trace logs grow rapidly when
you apply the ALL logging option.
Monitoring activity requires too many Table 28 on page 80 describes the performance impact of specific attribute
system resources. groups. If possible, decrease your use of the attribute groups that require
greater system resources.
A formula that uses mathematical This formula is incorrect because situation predicates support only logical
operators appears to be incorrect. For operators. Your formulas cannot have mathematical operators.
example, if you were monitoring Linux, Note: The Situation Editor provides alternatives to math operators.
a formula that calculates when Free Regarding the example, you can select % Memory Free attribute and
Memory falls under 10 percent of Total avoid the need for math operators.
Memory does not work: LT
#’Linux_VM_Stats.Total_Memory’ / 10

Appendix D. Problem determination 79

Table 27. Specific situation problems and solutions (continued)
Problem Solution
If you are running a Version 350 Access the database detail. In the ″release″ section change the version
Monitoring Agent for UNIX Logs and setting for the agent from 610 to 350. To enable Unicode and other
you choose to alter the views to include features, upgrade the monitoring agent to IBM Tivoli Monitoring, Version
a Version 610 UNICODE attribute, be 6.1.0.
aware that data for this attribute is not
displayed and you see a blank column
in this view.
Situations that you create display the For a situation to have the correct severity in TEC for those situations
severity UNKNOWN in IBM Tivoli which are not mapped, you need to ensure that an entry exists in the
Enterprise Console. tecserver.txt file for the situation and that SEVERITY is specified.

See the “Configuring Tivoli Enterprise Console integration” chapter in the

IBM Tivoli Monitoring Administrator’s Guide for more information.
You see the 'Unable to get attribute Ensure that the agent attribute files are installed on the Tivoli Enterprise
name' error in the Tivoli Enterprise Monitoring Server.
Monitoring Server log after creating a
situation. The following example shows a typical log entry when you have this
problem:
(4320916A.0049-F60:kfaottev.c,1572,"Translate_ResultBuffer") \
Unable to get attribute name for tablename/column \
<UAG524400.UA4>. Ignored.

Consider performance impact of each attribute group: Table 28 lists the impact
on performance (high, medium, or low) of each attribute group. The
multiple-instance attributes have been classified at the lowest level. That is, the
performance overhead will increase if you do not specify compare values for one
or more key values.

When you want to prevent impact on performance by any of the attribute groups
listed in Table 28 you must avoid referencing that attribute group, as suggested in
this list:
v Disable the attribute group.
v Never select workspaces that reference the attribute group.
v Disable situations that reference the attribute group by using the ″Undistributed
situations″ option in the Situation Editor.
v Disable historical reporting that references the attribute group.
v Avoid using the ″Auto Refresh″ refresh feature in a Workspace because this
option causes a refresh of data for all attribute groups.
See the IBM Tivoli Monitoring User’s Guide for additional information on controlling
attribute group usage.
Table 28. Performance Impact by attribute group
Attribute group High Medium Low
Log Entries U

By default the table associated with the attribute group shows 24 hours of
data. This set of data might be large.
Monitored Logs U

Problems with configuration of situations

Table 29 on page 81 lists problems that might occur with situations.

80 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 This section provides information for problem determination for agents. Be sure to
consult the IBM Tivoli Monitoring Problem Determination Guide for more general
problem determination information.
Table 29. Problems with configuring situations that you solve in the Situation Editor
Problem Solution
Note: To get started with the solutions in this section, perform these steps:
1. Launch the Tivoli Enterprise Portal.
2. Click Edit > Situation Editor.
3. In the tree view, choose the agent whose situation you want to modify.
4. Choose the situation in the list. The Situation Editor view is displayed.
The situation for a specific agent is Open the Situation Editor. Access the All managed servers view. If the situation
not visible in the Tivoli Enterprise is absent, confirm that application support for Monitoring Agent for UNIX
Portal. Logs has been added to the monitoring server. If not, add application support
to the server, as described in the IBM Tivoli Monitoring Installation and Setup
Guide.
The monitoring interval is too Access the Situation Editor view for the situation that you want to modify.
long. Check the Sampling interval area in the Formula tab. Adjust the time interval
as needed.
The situation did not activate at Manually recycle the situation as follows:
startup. 1. Right-click the situation and choose Stop Situation.
2. Right-click the situation and choose Start Situation.
Note: You can permanently avoid this problem by placing a check mark in the
Run at Startup option of the Situation Editor view for a specific situation.
The situation is not displayed. Click the Action tab and check whether the situation has an automated
corrective action. This action can occur directly or through a policy. The
situation might be resolving so quickly that you do not see the event or the
update in the graphical user interface.
An Alert event has not occurred Check the logs, reports, and workspaces.
even though the predicate has been
properly specified.
A situation fires on an unexpected Confirm that you have distributed and started the situation on the correct
managed object. managed system.
The product did not distribute the Click the Distribution tab and check the distribution settings for the situation.
situation to a managed system.

Appendix D. Problem determination 81

Table 29. Problems with configuring situations that you solve in the Situation Editor (continued)
Problem Solution
The situation does not fire. In the Formula tab, analyze predicates as follows:
1. Click the fx icon in the upper-right corner of the Formula area. The Show
Incorrect predicates are present in
formula window is displayed.
the formula that defines the
situation. For example, the a. Confirm the following details in the Formula area at the top of the
managed object shows a state that window:
normally triggers a monitoring v The attributes that you intend to monitor are specified in the formula.
event, but the situation is not true v The situations that you intend to monitor are specified in the formula.
because the wrong attribute is v The logical operators in the formula match your monitoring goal.
specified in the formula. v The numerical values in the formula match your monitoring goal.
b. (Optional) Click the Show detailed formula check box in the lower left
of the window to see the original names of attributes in the application
or operating system that you are monitoring.
c. Click OK to dismiss the Show formula window.
2. (Optional) In the Formula area of the Formula tab, temporarily assign
numerical values that will immediately trigger a monitoring event. The
triggering of the event confirms that other predicates in the formula are
valid.
Note: After you complete this test, you must restore the numerical values
to valid levels so that you do not generate excessive monitoring data based
on your temporary settings.

Table 30. Problems with configuration of situations that you solve in the Workspace area
Problem Solution
Situation events are not displayed Associate the situation with a workspace.
in the Events Console view of the Note: The situation does not need to be displayed in the workspace. It is
workspace. sufficient that the situation be associated with any workspace.
You do not have access to a Note: You must have administrator privileges to perform these steps.
situation. 1. Select Edit > Administer Users to access the Administer Users window.
2. In the Users area, select the user whose privileges you want to modify.
3. In the Permissions tab, Applications tab, and Navigator Views tab, select
the permissions or privileges that correspond to the user’s role.
4. Click OK.
A managed system seems to be 1. Select Physical View and highlight the Enterprise Level of the navigator
offline. tree.
2. Select View > Workspace > Managed System Status to see a list of
managed systems and their status.
3. If a system is offline, check network connectivity and status of the specific
system or application.

Table 31. Problems with configuration of situations that you solve in the Manage Tivoli Enterprise Monitoring Services
window
Problem Solution
After an attempt to restart the Check the system status and check the appropriate IBM Tivoli Monitoring logs.
agents in the Tivoli Enterprise
Portal, the agents are still not
running.
The Tivoli Enterprise Monitoring Check the system status and check the appropriate IBM Tivoli Monitoring logs.
Server is not running.

82 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Support information
If you have a problem with your IBM software, you have the following options for
obtaining support for software products:
v “Searching knowledge bases”
v “Obtaining fixes” on page 84
v “Receiving weekly support updates” on page 84
v “Contacting IBM Software Support” on page 84

Searching knowledge bases

You can search the available knowledge bases to determine whether your problem
was already encountered and is already documented.

Searching the information center

Note: If you print PDF documents on other than letter-sized paper, set the option
in the File > Print window that allows Adobe Reader to print letter-sized
pages on your local paper.

The documentation CD contains the publications that are in the product library.
The format of the publications is PDF, HTML, or both.

IBM posts publications for this and all other Tivoli products, as they become
available and whenever they are updated, to the Tivoli software information center
Web site. Access the Tivoli software information center by first going to the Tivoli
software library at the following Web address:

http://www.ibm.com/software/tivoli/library

Scroll down and click the Product manuals link. In the Tivoli Technical Product
Documents Alphabetical Listing window, click M to access all of the IBM Tivoli
Monitoring product manuals.

Searching the Internet

If you cannot find an answer to your question in the information center, search the
Internet for the latest, most complete information that might help you resolve your
problem.

The IBM Software Support Web site provides the latest information about known
product limitations and workarounds in the form of technotes for your product.
You can view this information at the following Web site:

http://www.ibm.com/software/support

To search for information on IBM products through the Internet (for example, on
Google), be sure to consider the following types of documentation:
v IBM technotes
v IBM downloads
v IBM Redbooks
v IBM developerWorks
v Forums and newsgroups

Appendix D. Problem determination 83

 Obtaining fixes
A product fix might be available to resolve your problem. To determine what fixes
are available for your IBM software product, follow these steps:
1. Go to the Software support Web site at
http://www.ibm.com/software/support.
2. Click the Download tab.
3. Select the operating system in the Operating system menu.
4. Type search terms in the Enter search terms field.
5. As appropriate, use other search options to further define your search.
6. Click Search.
7. From the list of downloads returned by your search, click the name of a fix to
read the description of the fix and to optionally download the fix.

For more information about the types of fixes that are available, see the IBM
Software Support Handbook at
http://techsupport.services.ibm.com/guides/handbook.html.

Receiving weekly support updates

To receive weekly e-mail notifications about fixes and other software support news,
follow these steps:
1. Go to the IBM Software Support Web site at
http://www.ibm.com/software/support.
2. Click My account in the upper right corner of the page.
3. Click Subscribe to IBM e-news. (If you have already subscribed and want to
modify your subscription preferences, click Modify subscriptions and follow
the instructions on screen.)
4. Follow the instructions on screen to provide the following data:
v Your personal contact information.
v Your areas of interest.
v The types of subscriptions and regional versions that you want to receive.
5. Review the subscription confirmation to confirm your settings.

Contacting IBM Software Support

IBM Software Support provides assistance with product defects.

Before contacting IBM Software Support, your company must have an active IBM
software maintenance contract, and you must be authorized to submit problems to
IBM. The type of software maintenance contract that you need depends on the
type of product you have:
v For IBM distributed software products (including, but not limited to, Tivoli,
Lotus, and Rational products, as well as DB2 and WebSphere products that run
on Windows, or UNIX operating systems), enroll in Passport Advantage in one
of the following ways:
Online
Go to the Passport Advantage Web site at
http://www.lotus.com/services/passport.nsf/
WebDocs/Passport_Advantage_Home and click How to Enroll.
By phone
For the phone number to call in your country, go to the IBM Software

84 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 Support Web site at
http://techsupport.services.ibm.com/guides/contacts.html and click the
name of your geographic region.
v For customers with Subscription and Support (S & S) contracts, go to the
Software Service Request Web site at
https://techsupport.services.ibm.com/ssr/login.
v For customers with IBMLink, CATIA, Linux, OS/390, iSeries, pSeries, z/Series,
and other support agreements, go to the IBM Support Line Web site at
http://www.ibm.com/services/us/index.wss/so/its/a1000030/dt006.
v For IBM eServer software products (including, but not limited to, DB2 and
WebSphere products that run in z/Series, pSeries, and iSeries environments),
you can purchase a software maintenance agreement by working directly with
an IBM sales representative or an IBM Business Partner. For more information
about support for eServer software products, go to the IBM Technical Support
Advantage Web site http://www.ibm.com/servers/eserver/techsupport.html.

If you are not sure what type of software maintenance contract you need, call
1-800-IBMSERV (1-800-426-7378) in the United States. From other countries, go to
the contacts page of the IBM Software Support Handbook on the Web at
http://techsupport.services.ibm.com/guides/contacts.html and click the name of
your geographic region for phone numbers of people who provide support for
your location.

To contact IBM Software support, follow these steps:

1. “Determining the business impact”
2. “Describing problems and gathering information”
3. “Submitting problems” on page 86

Determining the business impact

When you report a problem to IBM, you are asked to supply a severity level.
Therefore, you need to understand and assess the business impact of the problem
that you are reporting. Use the following criteria:
Severity 1
The problem has a critical business impact. You are unable to use the
program, resulting in a critical impact on operations. This condition
requires an immediate solution.
Severity 2
The problem has a significant business impact. The program is usable, but
it is severely limited.
Severity 3
The problem has some business impact. The program is usable, but less
significant features (not critical to operations) are unavailable.
Severity 4
The problem has minimal business impact. The problem causes little impact
on operations, or a reasonable circumvention to the problem was
implemented.

Describing problems and gathering information

When describing a problem to IBM, be as specific as possible. Include all relevant
background information so that IBM Software Support specialists can help you
solve the problem efficiently. To save time, know the answers to these questions:
v What software versions were you running when the problem occurred?

Appendix D. Problem determination 85

 v Do you have logs, traces, and messages that are related to the problem
symptoms? IBM Software Support is likely to ask for this information.
v Can you re-create the problem? If so, what steps were performed to re-create the
problem?
v Did you make any changes to the system? For example, did you make changes
to the hardware, operating system, networking software, and so on.
v Are you currently using a workaround for the problem? If so, be prepared to
explain the workaround when you report the problem.
See “Gathering product information for IBM Software Support” on page 67 for
further tips for gathering information for IBM Software Support.

Submitting problems
You can submit your problem to IBM Software Support in one of two ways:
Online
Click Submit and track problems on the IBM Software Support site at
http://www.ibm.com/software/support/probsub.html. Type your
information into the appropriate problem submission form.
By phone
For the phone number to call in your country, go to the contacts page of
the IBM Software Support Handbook at
http://techsupport.services.ibm.com/guides/contacts.html and click the
name of your geographic region.

If the problem you submit is for a software defect or for missing or inaccurate
documentation, IBM Software Support creates an Authorized Program Analysis
Report (APAR). The APAR describes the problem in detail. Whenever possible,
IBM Software Support provides a workaround that you can implement until the
APAR is resolved and a fix is delivered. IBM publishes resolved APARs on the
Software Support Web site daily, so that other users who experience the same
problem can benefit from the same resolution.

86 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

Appendix E. Accessibility
Accessibility features help users with physical disabilities, such as restricted
mobility or limited vision, to use software products successfully. The major
accessibility features in this product enable users to do the following:
v Use assistive technologies, such as screen-reader software and digital speech
synthesizer, to hear what is displayed on the screen. Consult the product
documentation of the assistive technology for details on using those technologies
with this product.
v Operate specific or equivalent features using only the keyboard.
v Magnify what is displayed on the screen.

In addition, the product documentation was modified to include the following

features to aid accessibility:
v All documentation is available in both HTML and convertible PDF formats to
give the maximum opportunity for users to apply screen-reader software.
v All images in the documentation are provided with alternative text so that users
with vision impairments can understand the contents of the images.

Navigating the interface using the keyboard

Standard shortcut and accelerator keys are used by the product and are
documented by the operating system. Refer to the documentation provided by
your operating system for more information.

Magnifying what is displayed on the screen

You can enlarge information on the product windows using facilities provided by
the operating systems on which the product is run. For example, in a Microsoft®
Windows environment, you can lower the resolution of the screen to enlarge the
font sizes of the text on the screen. Refer to the documentation provided by your
operating system for more information.

© Copyright IBM Corp. 2005 87

88 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Appendix F. Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in
other countries. Consult your local IBM representative for information on the
products and services currently available in your area. Any reference to an IBM
product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product,
program, or service that does not infringe any IBM intellectual property right may
be used instead. However, it is the user’s responsibility to evaluate and verify the
operation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matter
described in this document. The furnishing of this document does not give you
any license to these patents. You can send license inquiries, in writing, to:

IBM Director of Licensing

IBM Corporation
North Castle Drive
Armonk, NY 10504-1785 U.S.A.

For license inquiries regarding double-byte (DBCS) information, contact the IBM
Intellectual Property Department in your country or send inquiries, in writing, to:

IBM World Trade Asia Corporation

Licensing
2-31 Roppongi 3-chome, Minato-ku
Tokyo 106, Japan

The following paragraph does not apply to the United Kingdom or any other
country where such provisions are inconsistent with local law:

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS

PUBLICATION ″AS IS″ WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS
FOR A PARTICULAR PURPOSE.

Some states do not allow disclaimer of express or implied warranties in certain

transactions, therefore, this statement might not apply to you.

This information could include technical inaccuracies or typographical errors.

Changes are periodically made to the information herein; these changes will be
incorporated in new editions of the publication. IBM may make improvements
and/or changes in the product(s) and/or the program(s) described in this
publication at any time without notice.

Any references in this information to non-IBM Web sites are provided for
convenience only and do not in any manner serve as an endorsement of those Web
sites. The materials at those Web sites are not part of the materials for this IBM
product and use of those Web sites is at your own risk.

© Copyright IBM Corp. 2005 89

 IBM may use or distribute any of the information you supply in any way it
believes appropriate without incurring any obligation to you.

Licensees of this program who wish to have information about it for the purpose
of enabling: (i) the exchange of information between independently created
programs and other programs (including this one) and (ii) the mutual use of the
information which has been exchanged, should contact:

IBM Corporation
2Z4A/101
11400 Burnet Road
Austin, TX 78758 U.S.A.

Such information may be available, subject to appropriate terms and conditions,

including in some cases payment of a fee.

The licensed program described in this document and all licensed material
available for it are provided by IBM under terms of the IBM Customer Agreement,
IBM International Program License Agreement or any equivalent agreement
between us.

Any performance data contained herein was determined in a controlled

environment. Therefore, the results obtained in other operating environments may
vary significantly. Some measurements may have been made on development-level
systems and there is no guarantee that these measurements will be the same on
generally available systems. Furthermore, some measurement may have been
estimated through extrapolation. Actual results may vary. Users of this document
should verify the applicable data for their specific environment.

Information concerning non-IBM products was obtained from the suppliers of

those products, their published announcements or other publicly available sources.
IBM has not tested those products and cannot confirm the accuracy of
performance, compatibility or any other claims related to non-IBM products.
Questions on the capabilities of non-IBM products should be addressed to the
suppliers of those products.

All statements regarding IBM’s future direction or intent are subject to change or
withdrawal without notice, and represent goals and objectives only.

This information is for planning purposes only. The information herein is subject to
change before the products described become available.

This information contains examples of data and reports used in daily business
operations. To illustrate them as completely as possible, the examples include the
names of individuals, companies, brands, and products. All of these names are
fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.

COPYRIGHT LICENSE:

This information contains sample application programs in source language, which

illustrate programming techniques on various operating systems. You may copy,
modify, and distribute these sample programs in any form without payment to
IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating
system for which the sample programs are written. These examples have not been

90 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

 thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply
reliability, serviceability, or function of these programs. You may copy, modify, and
distribute these sample programs in any form without payment to IBM for the
purposes of developing, using, marketing, or distributing application programs
conforming to IBM’s application programming interfaces.

If you are viewing this information in softcopy form, the photographs and color
illustrations might not appear.

Trademarks
IBM, the IBM logo, AIX, Candle, CICS, DB2®, developerWorks®, eServer™,
HACMP, Hummingbird®, iSeries™, i5/OS, Lotus®, OMEGAMON, OS/390®,
OS/400, Passport Advantage®, pSeries®, Rational®, Redbooks™, Tivoli, the Tivoli
logo, Tivoli Enterprise, Tivoli Enterprise Console, WebSphere®, zOS®, and zSeries
are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both.

Microsoft, Windows, and Windows NT® are registered trademarks of Microsoft

Corporation in the United States, other countries, or both.

Intel®, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo,
Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium® are trademarks or
registered trademarks of Intel Corporation or its subsidiaries in the United States
and other countries.

Linux is a trademark of Linus Torvalds in the United States, other countries, or

both.

UNIX is a registered trademark of The Open Group in the United States and other
countries.

Other company, product, and service names may be trademarks or service marks
of others.

Appendix F. Notices 91
92 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide
Index
Numerics data (continued)
trace logs 68
12-hour format times 57 viewing 15
data mapping specifications 50
data provider
A See agent
accessibility ix, 87 data type in format description 48
agent database agent installation problems 75
trace logs 69 detecting problems, modifying situation values 14
agent installation problems 75 directory names, notation xi
ASCII logs, nonconforming 43 disk capacity planning for historical data 26
attribute groups disk space requirements 5
list of all 24
Log Entries 24
more information 23 E
overview 23 education
performance impact 80 see Tivoli technical training x
attributes entry time format
more information 23 12-hour format 57
overview 23 defaulting the year 58
missing components 58
specifying 57
B text months 57
books environment
feedback viii customizing 13
online viii features 1
ordering viii monitoring real-time 11
see publications ix real-time monitoring 11
built-in problem determination features 67 environment variables 8, 9
environment variables, notation xi
event
C mapping 65
events
calculate historical data disk space 26 investigating 12
capacity planning for historical data 26 workspaces 12
collecting data 15
commands, Take Action 39
components 2
configuration 5 F
customer configuration file 6 features, Monitoring Agent for UNIX Logs 1
customer configuration file format 7 field suppression in format description 47
environment variable syntax 9 files
environment variables 8 agent trace 69
specifying log files to monitor 6 installation trace 69
syslog daemon configuration file 7 other trace log 70
using nonstandard logs 10 trace logs 68
contacting support 84 fixes, obtaining 84
conventions format command 43, 61
operating system xi format description 45
typeface x format description components 45
customer configuration file 6 data mapping specifications 50
customer support data type 48
See support field suppression 47
customizing literals 45
monitoring environment 13 offsets 46
situations 14 size 47
width 47
formatting mapped data 51
D
data
collecting 15

© Copyright IBM Corp. 2005 93

G K
gathering support information 67 knowledge bases for support 83
Generic User Log Support (GULS) 10, 43 kulmapper utility 61
analyzing logs before using 63
testing format commands 63
H using 62
HACMP_acquire_service_addr 30
HACMP_acquire_takeover_addr situation 31
HACMP_config_too_long situation 31 L
HACMP_event_error situation 31 legal notices 89
HACMP_fail_standby situation 31 literals in format description 45
HACMP_get_disk_vg_fs situation 31 Log Entries attribute group 24
HACMP_join_standby situation 32 Log Entries workspace 17
HACMP_network_down situation 32 log entry fields 20
HACMP_network_down_complete situation 32 log entry times 57
HACMP_network_up situation 32 log format description 45
HACMP_network_up_complete situation 32 log format, generic user log 43
HACMP_node_down situation 33 command syntax 45
HACMP_node_down_complete situation 33 example 43
HACMP_node_down_local situation 33 log types supported 43
HACMP_node_down_local_complete situation 33 logging
HACMP_node_down_remote situation 33 agent trace logs 69, 70
HACMP_node_down_rmt_complete situation 34 built-in features 67
HACMP_node_up situation 34 installation log files 69
HACMP_node_up_complete situation 34 location and configuration of logs 68
HACMP_node_up_local situation 34 trace log files 68
HACMP_node_up_local_complete situation 34 logs, monitoring other types of 43
HACMP_node_up_remote situation 35
HACMP_node_up_remote_complete situation 35
HACMP_release_service_addr situation 35
HACMP_release_takeover_addr situation 35
M
manuals
HACMP_release_vg_fs situation 35
feedback viii
HACMP_start_server situation 36
online viii
HACMP_stop_server situation 36
ordering viii
HACMP_swap_adapter situation 36
see publications ix
HACMP_swap_adapter_complete situation 36
mapped data, formatting 51
historical data
mapping data 50
calculate disk space 26
mapping format specifier components 52
disk capacity planning 26
data type 54
historical data, collecting and viewing 15
escape characters 55
literals 52
options 52
I precision 53
IBM Software Support size 54
See support width 53
IBM Tivoli Enterprise Console memory requirements 5
event mapping 65 messages
IBM Tivoli Monitoring: UNIX Log Agent built-in features 67
performance considerations 79 modifying situation values to detect problems 14
information centers for support 83 Monitored Logs workspace 18
information, additional Monitoring Agent for UNIX Logs
attributes 23 components 2
policies 41 features 1
procedural 11 purposes 11
situations 29 using 11
Take Action commands 39 monitoring other log types 43
workspaces 17 monitoring, viewing the real-time environment 11
installation 5 months, text form 57
log file 69
problems 75
interface, user 3
Internet
N
notation
for product support 83
environment variables xi
investigating an event 12
path names xi
typeface xi

94 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide

O remote deployment
problem determination 78
offsets in format description 46 requirements
online publications disk space 5
accessing ix memory 5
for support 83 operating system 5
operating systems 5 other 6
operation of resource, recovering 12 resetting situations using Until predicate 21
ordering publications ix resource, recovering operation 12
other requirements 6

P S
sample Log Entry workspace 21
path names, for trace logs 68 scenarios, workspace 18
path names, notation xi situations
performance considerations 79 general problem determination 80
performance impact HACMP_acquire_service_addr 30
attribute groups 80 HACMP_acquire_takeover_addr situation 31
policies HACMP_config_too_long situation 31
more information 41 HACMP_event_error situation 31
overview 41 HACMP_fail_standby situation 31
predefined policies 41 HACMP_get_disk_vg_fs situation 31
problem determination 67, 73 HACMP_join_standby situation 32
built-in features 67 HACMP_network_down situation 32
describing problems 85 HACMP_network_down_complete situation 32
determining business impact 85 HACMP_network_up situation 32
information centers for 83 HACMP_network_up_complete situation 32
installation 75 HACMP_node_down situation 33
installation logs 69 HACMP_node_down_complete situation 33
knowledge bases for 83 HACMP_node_down_local situation 33
remote deployment 78 HACMP_node_down_local_complete situation 33
situations 79, 80 HACMP_node_down_remote situation 33
submitting problems 86 HACMP_node_down_rmt_complete situation 34
uninstallation 75 HACMP_node_up situation 34
uninstallation logs 69 HACMP_node_up_complete situation 34
problems HACMP_node_up_local situation 34
detecting 14 HACMP_node_up_local_complete situation 34
problems and workarounds 73 HACMP_node_up_remote situation 35
procedures 11 HACMP_node_up_remote_complete situation 35
publications HACMP_release_service_addr situation 35
accessing online ix HACMP_release_takeover_addr situation 35
feedback viii HACMP_release_vg_fs situation 35
for support 83 HACMP_start_server situation 36
online viii HACMP_stop_server situation 36
ordering viii, ix HACMP_swap_adapter situation 36
purposes HACMP_swap_adapter_complete situation 36
collecting data 15 list of all 30
customizing monitoring environment 13 more information 29
investigating events 12 overview 29
monitoring with custom situations 14 predefined 30
problem determination 67 resetting using Until predicate 21
recovering resource operation 12 specific problem determination 79
viewing data 15 UNIX_LAA_Bad_su_to_root_Warning 36
viewing real-time monitoring environment 11 UNIX_LAA_Log_Size_Warning 37
using attributes 23
values, modifying 14
Q size in format description 47
queries, using attributes 23 support
about 83
contacting 84
R describing problems 85
determining business impact of problems 85
real-time data, viewing 11 gathering information for 67
recovering the operation of a resource 12 information centers for 83
refresh signal 10 knowledge bases for 83
refreshing the monitoring agent 9 obtaining fixes 84
on Internet 83

Index 95
support (continued) workspaces (continued)
submitting problems 86 typical scenarios (continued)
weekly update option 84 security issues 18
syslog daemon configuration file 7

Y
T year, omitting from entry times 58
Take Action commands 12
list of all 39
more information 39
overview 39
predefined 39
tasks for using 11
text months 57
time stamps 57
Tivoli software information center ix
Tivoli technical training x
trace logs 68
directories 68
trademarks 91
training, Tivoli technical x
troubleshooting 67
tuning format commands 61
typeface conventions x

U
uninstallation
log file 69
problems 75
UNIX_LAA_Bad_su_to_root_Warning situation 36
UNIX_LAA_Log_Size_Warning situation 37
Until predicate 21
user interfaces options 3

V
values, modifying situations 14
variables, notation for xi
viewing data 15
viewing real-time monitoring environment 11
views
Log Entries workspace 17
Monitored Logs workspace 18

W
weekly update support option 84
width in format description 47
Windows agent installation problems 75
workarounds 73
remote deployment 78
situations 79
workspaces
event 12
list of all 17
Log Entries 17
Monitored Logs 18
more information 17
overview 17
predefined 17
sample Log Entry 21
typical scenarios 18
file server problems 18
monitoring applications 19

96 IBM Tivoli Monitoring: UNIX Log Agent: User’s Guide




Printed in USA

SC32-9471-00