Vista Plus

Technical Addendum
This document discusses some technical aspects of certain
Vista Plus functions. This information is presented in an
Addendum due to its technical nature and because it is
probably of interest to only a small percentage of Vista Plus
users.

VP050502-TA-EN-02

VistaPlus
TechnicalAddendum
VP050502-TA-EN-02
Rev: 2013-Feb-14
This documentation has been created for software version 5.5.2.
It is also valid for subsequent software versions as long as no new document version is shipped with the product or is
published at https://knowledge.opentext.com.

OpenTextSA
40 Avenue Monterey, Luxembourg, Luxembourg L-2163
Tel: 35 2 264566 1

OpenTextCorporation
275 Frank Tompa Drive, Waterloo, Ontario, Canada, N2L 0A1
Tel: +1-519-888-7111
Toll Free Canada/USA: 1-800-499-6544 International: +800-4996-5440
Fax: +1-519-888-0677
Email: [email protected]
FTP: ftp://ftp.opentext.com
For more information, visit http://www.opentext.com

Copyright20062012OpenTextCorporation
OpenText is a trademark or registered trademark of Open Text SA and/or Open Text ULC. The list of trademarks is not
exhaustive of other trademarks, registered trademarks, product names, company names, brands and service names
mentioned herein are property of Open Text SA or other respective owners.

Disclaimer
No Warranties and Limitation of Liability
Every effort has been made to ensure the accuracy of the features and techniques presented in this publication. However,
Open Text Corporation and its affiliates accept no responsibility and offer no warranty whether expressed or implied, for
the accuracy of this publication.

Table of Contents
Introduction ............................................................................................................................. 1
ImportFileInfo File Format ....................................................................................................... 2
Passwords and ImportFileInfo.......................................................................................... 4
Exporting LDAP User and Group Data.................................................................................. 5
What DSExporter Copies.................................................................................................. 5
What DSExporter Does Not Copy ................................................................................... 5
Output File Format............................................................................................................ 6
DSExporter Command Format........................................................................................ 6
After Running DSExporter................................................................................................. 7
Using the x Option During Report Capture ........................................................................ 8
x Prepend File Location.................................................................................................. 8
Sample Uses and Files ...................................................................................................... 9
Notes ................................................................................................................................ 10
Vista Plus Encryption ............................................................................................................. 11
Vista Plus Temporary File Use ............................................................................................... 12
The Warehouse Parameters Temporary Directory...................................................... 12
The TMP/TMPDIR Temporary Directory ......................................................................... 13
The .locks Subdirectory .................................................................................................. 13
Report Warehouse Folder and File Structure ..................................................................... 14
The server.debug.log Message File .................................................................................... 17
Circular Logging ............................................................................................................. 17
External Password Authorization ......................................................................................... 19
About the Sample Authorization Module.................................................................... 20
Building the Sample Code ............................................................................................ 21
Installing a Plug-in Security Module .............................................................................. 22
Using the Sample Module ............................................................................................. 22
Making a Custom Authorization Module..................................................................... 22
The Migration Process........................................................................................................... 24
The Migration Process .................................................................................................... 24
Offline Volume Structure................................................................................................ 25
VMTransports Temporary Files ...................................................................................... 26
Font Support in Web View When Using Xvfb ..................................................................... 28
Using Vista Plus Utilities .......................................................................................................... 29
The check_gens Utility.................................................................................................... 29
The del_gens Utility ......................................................................................................... 31
Adding Index Values to the Database .............................................................................. 34
Space Considerations.................................................................................................... 35
convert_gens Command Format................................................................................. 36

Vista Plus Technical Addendum

Table of Contents

Removing Index Values from the Database ..................................................................... 38

revert_gens Command Format .................................................................................... 38
Using revert_gens............................................................................................................ 39
Reclaiming the Database Space................................................................................. 40
Tuning Vista Plus Server Configuration................................................................................ 42
Scalability Configuration Options................................................................................. 42
Recommended Kernel Parameters on the Server Host ............................................. 46
MySQL Memory Limits..................................................................................................... 51
Preventing Lost Oracle Database Connections......................................................... 51

ii

Vista Plus Technical Addendum

Introduction
This document discusses some technical aspects of certain Vista Plus functions. It covers
these topics:

File format for importing user, group, and link information with the vadmin
ImportFileInfo command

Using the DSExporter command to export user data from an Active Directory or
LDAP database so it can be imported in to Vista Plus

Using the capture x option to prepend PCL files to captured reports

How Vista Plus encryption works

Temporary files and directories Vista Plus creates and uses

Report warehouse folder and file structure, including rendition files created by
epurposing

The server.debug.log message file

Creating and using an external password authorization routine

Details of the VMIdentify and VMTransport processes

Notes about font support when using Web View in the Xvfb environment

Using the check_gens and del_gens utilities

Using convert_gens to add index values from the report warehouse to the Vista Plus
database

Using revert_gens to remove index values from the Vista Plus database if desired

Topics related to tuning Vista Plus configuration and related settings to improve
performance and avoid certain types of errors. These include:

Scalability configuration options

Kernel parameter settings on Sun Solaris and HP-UX

Adjusting MySQL memory limits

Preventing lost connections to an Oracle database

This information is presented in an addendum rather than in the Vista Plus Server
Administration Guide due to its technical nature and because it is probably of interest to
only a small percentage of Vista Plus users.

Vista Plus Technical Addendum

ImportFileInfo File Format

ImportFileInfo File Format

The ImportFileInfo vadmin command loads multiple users, groups, and/or user-group
links into Vista Plus from a text file. It can also modify or delete existing users, groups,
and links. As described in the Vista Plus Server Administration Guide, the command format
is simple: the only required parameter for ImportFileInfo is the name of the text file
containing the information to import. Here is the syntax:
vadmin c=ImportFileInfo f=file [ptype=type] [password=password]

Based on the information in the text file, ImportFileInfo builds a batch file of the
appropriate vadmin commands, then submits it. The optional ptype and password
parameters determine whether users created by ImportFileInfo use passwords found in
the text file or are assigned different passwords. See the section Passwords and
ImportFileInfo, below.
Tip

The vadmin executable must exist in the current working directory where
you enter the ImportFileInfo command. This is true even if the vadmin
installation directory is included in your PATH statement.

The key to using ImportFileInfo successfully is the format of the data in the text file. A
single text file can contain all three types of records to updateusers, groups, and user
membership in groupsand all three actionsadding, modifying, and deleting. The file
format is as follows:

Each record occupies one line.

Each line consists of a number of fields; fields are delimited by the vertical bar
character ( | ). The number of fields depends on the type of recorduser, group, or
linkbeing updated.

The first field of the line determines the type of record to update:

0 = user

1 = group

2 = user-group link

The second field of the line determines the type of action to take:

0 = add new record

1 = modify existing record

2 = delete existing record

The rest of the line contains the information to add, modify, or delete.

Vista Plus Technical Addendum

ImportFileInfo File Format

A line for a user record has this format:

0|action|login-name|full-name|description|password|home-folder|
unix-uid|admin (y|n)*|primary-group|fax-number|pager-number|
email-address|ptrdef|home-address|department|start date|end date|
days till password expires|authorization method (v|u|x|a|n)*|force
password change? (y|n)*|

*: Allowable values for the field are shown in parentheses; enter only the value.

If you include ptype on the command line, any passwords in the file are ignored. See
Passwords and ImportFileInfo on page 4.

A line for a group record has this format:

1|action|group-name|description|home-folder|ptrdef|start date|end
date|

A line for a user-group link has this format:

2|action|user-name|group-name|start date|end date|

Blank spaces in data fields are significant; they are not ignored. Do not include a space
at the beginning or end of a field unless you want that space included when the data
is added to Vista Plus. For example, entering a user name as | Chris| is not the
same as |Chris|.

Not all data is required; required fields for each action are the same as for the
corresponding vadmin command. For example, a line which is adding a group must
start with 1|0 and include a group name and home folder; the other fields are
optional (but see the next item). See the Vista Plus Server Administration Guide for
descriptions of each field and which fields are required.

If you leave one or more fields out, you must include the correct number of vertical
bars as placeholders, even if the omitted fields are at the end of a line. There must be
one | character for each field, including blank fields. For example, to add the group
Writers with a Home folder of Marketing and a start date of 12/01/05, but no
description, default printer definition, or end date:
1|0|Writers||Marketing||12/01/05||

The two vertical bars between Writers and Marketing cause Marketing to be
treated properly as the Home folder rather than the description; the two vertical bars
after Marketing indicate the default printer definition is left blank, and the two
vertical bars at the end indicate the end date is blank. A group line contains eight
fields, including the 1 which marks it as a group record; the sample line includes eight
| characters, as it must.

The order of the records in the file does not matter. The groups are
added/deleted/modified first, then the user records, then the user-group links.

You can include comment lines in the file. Start each comment line with a pound sign
(#).

You can include blank lines in the file; they are ignored.

Vista Plus Technical Addendum

ImportFileInfo File Format

Notes

If a user or group record specifies a Home folder which doesnt exist, the folder is
created as a subfolder of the MAIN folder.

All groups included in either user or link records must either already exist or must be
created by a group record line in the same file. A group can be added only with a
group (type 1) line.

If you attempt to perform an add (action type 0) and the record already exists, no
action is performed. The existing record is not updated with new information. You
must use an update (action type 1) to update an existing record.

Passwords and ImportFileInfo

Depending on the source of the data for ImportFileInfo, your text file may or may not
contain user passwords. If there are passwords in the source text file and you want to use
them in Vista Plus, do not include the ptype parameter. If the source file doesnt have
passwords, or if you want to assign users new passwords during the import, you can use
ptype, and optionally password. There are three possible values for ptype:

common: Gives all users the password you specify using the password parameter. You
must include password with this setting.

random: If used without password, sets each password to the user name followed by
a five-digit random number. If you include password, sets each password to the
password entry followed by a five-digit random number. (If you set password to
nullpassword=the password is just the random number.) When you use
random, vadmin creates a text file called random.out which lists the user names and
the randomly-generated passwords.

username: Sets each password to the user name.

For example, to set all passwords to the word new followed by a five-digit random
number:
vadmin c=ifi file=file.txt ptype=random password=new

When you use random, users will not be able to log in to Vista Plus until an Administrator
tells them what their new passwords are, since each user will have a different random
number at the end of his or her password. Using common or username will let users log
in, but does not provide much security.
If you do not include ptype, ImportFileInfo uses any passwords found in the text file. If
there are no passwords in the text file, the users are created without passwords.
Including password without ptype has no effect; ImportFileInfo still uses passwords
from the text file, if there are any.
Note

Important! The ptype and password options are applied both to new users
being added and to existing users being modified by ImportFileInfo, and
override any passwords contained in the source text file. Be careful not to
inadvertently change passwords for existing users.

Vista Plus Technical Addendum

Exporting LDAP User and Group Data

Exporting LDAP User and Group Data

The Vista Plus DSExporter command takes user and group data from a Microsoft Active
Directory or other LDAP user database and exports it to a text file. You can thenafter
some modificationsuse the text file as the source file for the vadmin ImportFileInfo
command to create the groups and users in Vista Plus.
DSEXporter is frequently used together with the Vista Plus LDAP Authentication module
to create Vista Plus users for LDAP users, who can then sign in to Vista Plus using their
LDAP names and passwords. The LDAP Authentication module is described in the Vista
Plus Server Administration Guide.
Note

DSExporter is included in a full installation of the Vista Plus Windows server.

To install just DSExporter, perform a custom installation of the server and
select only the DSExporter option. See the Vista Plus Server Installation Guide
chapter on Windows server installation for more information.

What DSExporter Copies

DSExporter takes all of the information stored in LDAP for each group and user and
copies it to the output file. For groups, this is only the group name. For users, it includes:

User name

Full name

Description

E-mail address

Group memberships

If a user belongs to only one group, that group becomes his or her primary group when
imported into Vista Plus. If a user belongs to more than one group, all groups are added as
secondary groups; the user will not have a primary group when imported into Vista Plus
unless you take further action, as described below.

What DSExporter Does Not Copy

The information listed in the previous section is all that is available to DSExporter. It does
not include enough information to create valid Vista Plus user and group records:

All Vista Plus users must have either a home folder or a primary group. There are no
home folders in LDAP, and if a user belongs to multiple LDAP groups, DSExporter
has no way of knowing which should be the primary group in Vista Plus.

All Vista Plus groups must have home folders. There are no folders for groups in
LDAP.

Because this information does not exist in LDAP, but is required in Vista Plus, DSExporter
includes features to add a default primary group for each user and a home folder for each
group to the created text file, as described in the next section. This ensures that the text file

Vista Plus Technical Addendum

Exporting LDAP User and Group Data

contains at least the minimum information required to create valid users and groups in
Vista Plus.
Additionally, DSExporter cannot copy user passwords from LDAP, as they are encrypted.
If desired, you can use an ImportFileInfo option to set passwords for each user during
import. However, if the Vista Plus users will be using either operating system passwords
or their LDAP passwords (through the LDAP Authentication module) to log in to Vista
Plus, there is no need to create Vista Plus passwords for them.
Note

You can also type each users password into the text file before importing the
information into Vista Plus, but this is rarely practical, and would be a
security risk as the passwords would have to be entered in plain text.

Output File Format

DSExporter creates a text file containing the exported groups, users, and user-group links.
The file is formatted for input into Vista Plus using the vadmin ImportFileInfo command.
As not all data accepted by Vista Plus is available, many of the fields on each line are
blank. DSExporter places the correct number of vertical bars on each line so that it will
import into Vista Plus correctly. For more information, see ImportFileInfo File Format
on page 2.
The default output file name is DSExporter.txt in the current directory. You can change the
file name and location using the x option.

DSExporter Command Format

To export LDAP data to the intermediate text file, simply run DSExporter. You specify the
host or domain to copy data from. Options let you set the output file name, default
primary group and Home folder, and more.
The format for DSExporter is:
DSExporter s=source [t=w] [options]

Source is the name or IP address of an LDAP server, or a Microsoft Exchange or Active

Directory domain.
You must include t=w if the source is anything other than a Microsoft Exchange server.

Vista Plus Technical Addendum

Exporting LDAP User and Group Data

The available options are:

Option

Description

a=v|u|x|a|n

Sets the authorization method for all users in the output file:
v = Vista Plus passwords. This is the default.
u = User-provided authorization. Use this method if you want to
use the LDAP authentication module on a UNIX server host.
x = Unix passwords.
n = Windows passwords. Use this method if you want to use
LDAP passwords on a Windows server host.
a = Any authorization method.
Authorization methods are described in the Vista Plus Server
Administration Guide.

c=y|n

Setting to y forces users to change password when first logging in to

Vista Plus. Default is n. Has an effect only if authorization method is
v or a.

f=folder

Sets the home folder for all groups in the output file to folder. If you
don't use this option, all Home folders are set to HOME_FOLDER.

g=group

Sets the default primary group for users in the output file to group.
This is used for all users except those who belong to exactly one
group in the LDAP database. If you leave out this option, the default
primary group is PRIMARY_GROUP.

x=file

Sets the output file. You can enter a full path or just a file name. The
default output file is DSExporter.txt in the directory where you run
DSExporter. If the file already exists, the information is appended to
it; if it doesn't exist, it is created. If you include a path, the directory
must already exist.

After Running DSExporter

After you have run DSExporter, you can use its output file as the input file for vadmin
ImportFileInfo. However, you may want or need to make any or all of these changes to
the data in the output file before using it:

Setting the correct home folder for each group being created.

Setting the correct home folder for users who should have one.

Setting the correct primary group for users who were in more than one LDAP group.

Adding any other available data. It may be easier to add this data to the text file before
import rather than editing the groups or users in Vista Plus.

For a description of how to use vadmin ImportFileInfo and the information you can
import with it, see ImportFileInfo File Format on page 2.

Vista Plus Technical Addendum

Using the x Option During Report Capture

Using the x Option During Report Capture

The capture x option lets you prepend a file to a report during capture. You can use it
either on the capture command line or by including it in a capture configuration file. The
Vista Plus Server Administration Guide describes the syntax for using x; however, it does
not discuss what you may want to use this option for or the proper format for prepend
files.
x should be used only with ASCII text reports and some PCL reports. While x can add
any file before a captured report, it should be used only to prepend files containing PCL
codes. Its most common use is to emulate Printer-Specific Environment Settings usually
used by the printer a report is printed to. If a printer uses specific mode (portrait or
landscape) or font settings, reports sent to it dont need to include these settingsthey use
the mode and font preset for the printer. When one of these reports is captured to Vista
Plus, it may not display or print correctly because the report data does not include the
mode and font settings it needs.
For example, a printer could be set to always print in landscape mode using compressed
print. All reports sent to this printer use these settings; the reports themselves do not
include any commands setting the mode or compression. When one of these reports is
captured to Vista Plus and displayed, it will display in portrait mode using normal print:
its data may not all fit on the page, characters could overprint at the ends of lines, or there
could be other problems.
The x option can solve this problemand various othersby prepending to the report a
file containing a set of PCL codes. These PCL codes can replicate the Printer-Specific
Environment Settings so the report displays correctly in Vista Plus clients and prints
correctly when printed from Vista Plus.
Note

If you prepend a file containing PCL codes to a captured text report, the
captured report file is changed to a PCL file.

x Prepend File Location

When you use x, you specify the name of the file to prepend to the captured report. You
include just the file name, not a path. The file must exist in the prefiles subdirectory of the
Vista Plus installation directory on the server you are capturing the report to. If you use x
from a remote host, the prepend file must exist on the host being captured to, not on the
remote host where you are performing the capture.

Vista Plus Technical Addendum

Using the x Option During Report Capture

Sample Uses and Files

As mentioned above, the reason to use x is to add PCL control codes to the beginning of
captured reports. Here are two sample situations where x could be used.

UNIX Text Files

Unless otherwise specified, a PCL report expects DOS-style line termination: each line
ends with a carriage return/line feed pair. UNIX text files have only a line feed, not a
carriage return, at the end of each line. You can prepend a file to UNIX text files so line
termination will be correctly recognized. Here are the contents of the file, shown in both
hex and PCL representation. The file should be a continuous line of data ending with a
newline character.
Hex Representation:

1B 45 1B 26 6B 32 47

PCL Representation:

Esc E Esc & k

If you created this file and saved it in the prefiles subdirectory as, for example, unixterm,
you could prepend it to any captured UNIX text file to allow it to print properly on a PCL
printer.
Note

The NORMALIZE flag in the server.cfg file can also correct problems with
remote printing UNIX text files. See the Vista Plus Server Administration Guide.
This flag does not affect local printing or displaying reports with a Vista Plus
client.

Setting Landscape Mode and Compressed Print

Prepending the following sample file causes a report to print in landscape mode using
compressed print. You could use it in the situation described earlier, where the report is
usually printed to a printer which uses these Printer-Specific Environment Settings. While
it is shown in three lines here, this should be a continuous string of data ending with a
newline character.
Hex Representation:

1B 45 1B 28 38 55 1B 28

73 34 30 39 39 54

PCL Representation:

Esc E Esc (

Hex Representation:

1B 26 6C 31 4F 1B

PCL Representation:

Esc &

Hex Representation:

1B 26 6C 36 36 46 1B 28 73 31 36 2E 35 30 48

PCL Representation:

Esc &

Vista Plus Technical Addendum

U Esc (

26 6C 38 44 1B 26 6C 35 45

O Esc &

F Esc (

Esc & l

Using the x Option During Report Capture

Notes

10

The first character in any PCL prepend file should be the PCL reset sequence:
<Esc>E.

You can use a binary editor to create files containing PCL codes. You enter the
hexadecimal representation of each PCL code.

The x option may not work on all PCL files. Some PCL files, particularly those
generated by Windows applications, contain PJL header lines. These PJL header lines
will prevent the prepended PCL file from having any effect.

Vista Plus attempts to remove any initial CR/LF characters and the PCL reset
sequence from a report when a prepend file is being used. If a reset character follows
non-CR/LF characters in a PCL report, the prepended PCL file will most likely be
ignored.

The original file captured into Vista Plus is never modified by the capture process;
however, the prepend file is added to the copy of the original file kept in the Vista Plus
warehouse (the .v file). This allows you to remote print the report using the PCL
attributes in the prepend file.

If the prepend file you include with x doesnt exist, an error message is logged in the
Vista Plus server log and returned to the rcapture command or other capture tool
being used.

Vista Plus Technical Addendum

Vista Plus Encryption

Vista Plus Encryption

Vista Plus 4.1 and later includes server.cfg flags to enable encryption of information as it
passes between the various Vista Plus clients and the Vista Plus server. There are five
separate encryption flags, as listed in Appendix B of the Vista Plus Server Administration
Guide. Each flag controls encryption of information as it is passed between the Vista Plus
server and a particular client: vadmin, the Server Administration client, the Windows
Client, Web View, and capture clients.
These flags encrypt only network data packets being sent across the network or the
Internet. Vista Plus does not encrypt information stored in the report warehouse, even if
all encryption flags are set. When an encryption flag is on, the information is encrypted by
the server or client before transmission, and decrypted before being stored or displayed at
the other end.
Note

A users name and password are always encrypted when they are sent to the
server during login. This is not affected by the encryption flags.

Note

The encryption flag for Web View affects information only when it is being
sent between the Vista Plus server and the Web server where Web View is
installed. Whether or not data is encrypted between the Web server and the
users workstation is controlled by the Web server, not by Vista Plus.

Vista Plus Technical Addendum

11

Vista Plus Temporary File Use

Vista Plus Temporary File Use

Vista Plus creates temporary files and/or subdirectories in three directories on the Vista
Plus server:

The temporary directory set during installation and stored in warehouse parameters.

The directory named by the TMP (Windows) or TMPDIR (UNIX) statement in the
server.cfg file. If server.cfg does not have this statement, Vista Plus uses the directory set
by the equivalent system environment variableTMP or TMPDIR.

The .locks subdirectory of the Vista Plus server installation directory.

The tables below list the files created in each directory.

File name notes for the tables:

pid refers to the process ID on a UNIX server or the thread ID on a Windows server.

nnn indicates a numeric file name from 1 65535. These file names are assigned
sequentially by the operating system, not by Vista Plus.

GenID and RepID are the generation ID and the report ID, respectively.

The Warehouse Parameters Temporary Directory

12

File Name

Process

Description

temp_pid

Bundle distribution and

remote printing

Used during token replacement for bundle

banner pages.

bundle_print.pid

Bundle printing

Holds printable report data for bundle.

nnn

Capture

Store portions of the list of objects in the

generation when the list is too big for
memory.

OLTGenID\GenGenID
OLTGenID\RepRepID

Client-requested
archiving

Temporary storage of generation data to be

archived. The GenGenID file contains the
generation data in Vista Plus format;
RepRepID is a flat file containing the data.

Report file name

Conversion from 2.x (the

vconvert program)

Created for each generation during

conversion from 2.5, then deleted

_VISTA_seconds.dat

dircapture

Holds copy of file while time stamp is being

changed; seconds is number of seconds
since midnight.

vista_tempRepID_pid

Extracting a subreport
during search

Contains matching lines or pages. Used as

the source if the subreport is saved with
Store Report.

nnn

Index creation

Used during indexing.

lm_ip.out, lm_sem.out

Lock manager
initialization

Used during lock manager initialization.

Vista Plus Technical Addendum

Vista Plus Temporary File Use

File Name

Process

Description

report.000

Printing

Actual print file for report. The extension

may be 000, 001, etc.

vista_print.pid

Printing

Holds printable report data.

vista_mail.pid

Printing; sending e-mail

attachment

Holds the user-accessible data; removed

when action is complete.

tempRepID.pid

Search

Holds PCL version of extracted matching

pages or lines; used only with PCL reports.

nnn

Search

One or more temp files used during search.

_VISTA_time.cfg

Updating capture
configuration file

Original configuration file is deleted;

temporary file is saved as configuration file.

vistaportsessionnnn

Session management

Directory to hold persistent data for open

reports for session nnn. Port is the port
number of the Vista Plus server.

The TMP/TMPDIR Temporary Directory

File Name

Process

Description

VVSRnnn

vadmin gsr

Holds data during activity report

generation.

SCHnnn

Repository search

Holds temporary search data

vmt_n\GenID (n is
assigned by the
operating system)

VMTransport

Subdirectories to temporarily hold all

generation data (the generation itself,
indexes, etc.) during VMTransport. See
page 24 for more information.

vmt_n\VolID (n is
assigned by the
operating system)

VMTransport

Subdirectory to hold compressed

generations waiting to be copied to an
offline tape volume during VMTransport.
See page 24 for more information.

The .locks Subdirectory

File Name

Process

Description

GenID.act

Any process that opens a

generation fileWeb
View, VMTransport, and
so on

Contains a count indicating how many

processes currently have the GenID
generation open. When the last process
closes the generation, the file is deleted; it is
recreated when the generation is opened
again.

Vista Plus Technical Addendum

13

Report Warehouse Folder and File Structure

Report Warehouse Folder and File Structure

When you create an online volume for report storage, you specify the root path for Vista
Plus to use for the volume. Beneath this root path, Vista Plus creates a structured directory
tree containing the files for the report generations captured to that volume. The directory
tree has this structure:

Directly beneath the root path, there is a subdirectory called whnnnn. nnnn is the port
number for the warehouse. For example, if the warehouse uses the default port of
7980, the subdirectory is wh7980.

Beneath this subdirectory is a tree

based on the sequence numbers of
the captured generations. There is
a subdirectory for each digit of the
sequence number; the files for a
particular generation are stored in
a subdirectory contained in the
subdirectory corresponding to the
last digit of its sequence number.
This subdirectory which actually
contains the generation files is
named Gnnn; nnn is the sequence
number.
For example, if the warehouse port
number is 7980, the files for
generation 365 are in the directory
Wh7980/3/6/5/G365. The structure
is shown in the picture to the right.
Since not all sequence numbers
have the same number of digits, any subdirectory at any level can have both the 0, 1,
2, etc. subdirectories for the next digit of a sequence number, and a Gnnn subdirectory
containing generation files. In the picture, if there is a generation with a sequence
number of 36, the Wh7980/3/6 directory will have a G36 subdirectory in addition to the
0 through 9 subdirectories shown. Also, if there are generations with more than three
digits, as in most Vista Plus warehouses, there will be additional levels of
subdirectories.

14

Vista Plus Technical Addendum

Report Warehouse Folder and File Structure

The Gnnn subdirectory contains all the files for the report generation. This directory
will contain some combination of these files (* in the file name is the sequence number
of the generation):
File

Description

Notes

*.v

Original format report

file

Exists only if the warehouse parameter to store the

original format file is set for this report type.

*.i###

Index files

Indexes are numbered sequentially throughout the

warehouse. All generations of a report have the same
index file names; each one contains the index entries for
just that generation.

*.p###

Page security

Numbered sequentially throughout the warehouse.

Used only for page securities created before Vista Plus
5.5.1.

*.p

Page security

Contains page securities for the generation. Used for all

page securities created with Vista Plus 5.5.1 or later.

*.pidx

Page security index.

Index used to find individual page securities in the *.p

file. Used for all page securities created with Vista Plus
5.5.1 or later.

Note

When you re-index a generation, its page securities are re-created based
on the new index values. If you re-index a report or generation using
Vista Plus 5.5.1 or later, its page securities are moved from *.p### files to
the *.p and *.pidx files for the generation. (If you only re-index one index,
only pages securities that use that index are moved.)

*.o

Object file

Describes the contents of the generation: where each text

object goes; fonts to use; where bitmap images go, and so
on.

*.t

Text file

Contains the text objects for the report, with no

formatting or location information. All formatting and
location instructions are contained in the *.o file

*.b

Bitmaps

Contains all bitmap images for the generation; the *.o file
contains instructions for placing the bitmaps on the
page.

*.img

GIF files

GIF-format renditions of all images in the generation,

used to speed display of the generation at 100% zoom in
Web View. This file exists only if there was a
PARSE_RENDER_IMAGE=1 statement in server.cfg
when the generation was captured.

*.Z

Compressed report file

If the generation has been compressed, the directory

contains only a *.Z file. This file contains all information
from all of the other file types.

If you use epurposing, the Gnnn directory can contain these additional files and/or
subdirectories (* in the file name is the sequence number of the generation):

Vista Plus Technical Addendum

15

Report Warehouse Folder and File Structure

Directory or File

Description

Notes

frame

Files for frame-based

HTML rendition

Contains HTML files for table of contents and

for each page of a frame-based HTML
rendition. May also contain gif files for graphic
objects in the rendition.

id

Files for rendition

id is the page security ID. This subdirectory
based on page security may contain files for any rendition type
single-file HTML, frame-based HTML, text, or
PDF. If there is a frame-based rendition for the
page security, there is an id/frame directory to
contain the files.

*.pdf

PDF file

A single PDF file containing all the pages of

the generation or page security; the only file
for a PDF rendition.

*.txt

Text file

A single text file containing all the pages of the

generation or page security; the only file for a
text rendition.

*.html

HTML file

A single HTML file containing all the pages of

the generation or the page security; the only
file for a single-file HTML rendition.

*_o###.gif

GIF file

A GIF image file; ### is the object id of the

display item as stored in the object file (*.o) for
the generation; HTML rendition files may refer
to gif files for images.

*Page###.html

HTML page file

An HTML file containing a single page of the

report generation; ### is the page number. A
separate file is created for each page in a
frame-based rendition.

*toc.html

Table of contents file

A table of contents for a frame-based HTML

rendition.

*Imagepage-xxx.gif

Image file

Image file for HTML rendition, referenced by

appropriate html file; page is the page number
the image is found on; xxx is the sequential
number of the image on the page.

Note

16

For a rendition of the full generation, all html or pdf files are found in the
generation directory or its frame subdirectory; for renditions based on a page
security, they are found in the subdirectory for the page security or its frame
subdirectory. For page securities which Grant access to all pages, renditions
are placed in the generation directory, not in a separate subdirectory.

Vista Plus Technical Addendum

The server.debug.log Message File

The server.debug.log Message File

The server.debug.log file contains error and other messages generated by the Vista Plus
software. If a problem occurs, the contents of server.debug.log can help you and Vista Plus
customer support diagnose and correct it. server.debug.log is a plain text file and is created
in the Vista Plus server installation directory.
Some messages are more critical than others; Vista Plus assigns each message one of six
severity levels: Debug, Info, Log, Warning, Error, or Fatal. You can select which messages
are recorded in server.debug.log by using the LOGGING statement in the server.cfg file.
There are three possible settings for LOGGING:

OFF Logs messages of level Warning, Error, and Fatal. server.debug.log always
records these messages.

ON Logs messages of any level except Debug.

VERBOSE Logs messages of any severity level.

During normal operation, most installations leave LOGGING set to OFF. You would want
to set it to VERBOSE in these situations:

Before upgrading your version of Vista Plus. It can be very useful to have detailed
logging information in case of any problems during the upgrade.

If you are having any problems with Vista Plus, especially intermittent problems
whose cause is not apparent.

Any time you are asked to do so by Vista Plus customer support.

Circular Logging
The server.debug.log file can grow quicklyvery quickly if LOGGING is set to VERBOSE
and, if you do not regularly clear it, can eventually consume a great deal of disk space. To
avoid this, you can use the server.cfg statement CIRCULAR_DEBUG=size. Size is the
maximum size, in bytes, of server.debug.log. When server.debug.log reaches this size, it will
begin overwriting the oldest entries.
Once server.debug.log has filled up and is overwriting old entries with new ones, there is a
wrap point somewhere in the middle of the file: the place immediately following the
newest message where the next message will be written. Since messages are not all the
same length, this point may be in the middle of a line, with the newest message followed
by the end of the oldest message. The wrap point is marked by a | character.
Here are some other characteristics of circular logging:

The size in the CIRCULAR_DEBUG statement is in bytes. Messages in

server.debug.log can be up to 100 characters or longer; keep this in mind when deciding
on a maximum size.

The CIRCULAR_DEBUG size limit is not exact. When full-grown, the file will exceed
this limit by one line (probably around 70-100 bytes).

Vista Plus Technical Addendum

17

The server.debug.log Message File

18

Setting CIRCULAR_DEBUG to 0 or leaving it out of server.cfg turns off circular

logging; server.debug.log will grow until you clear it manually.

You can change the CIRCULAR_DEBUG setting at any time; to make the change take
effect you must stop and restart the Vista Plus server, as described in the Vista Plus
Server Administration Guide.

When CIRCULAR_DEBUG is on, the first line of server.debug.log contains

<circular>. If circular logging is off, the first line contains <.linear.> (there is no
special first line if you have never used circular logging).

Vista Plus Technical Addendum

External Password Authorization

External Password Authorization

You can instruct the Vista Plus server software to use an external authentication routine to
check the user name and password when a user attempts to log in; you set this option
individually for each user, as described in the Vista Plus Server Administration Guide. To
make this option work, you must write your own external routine which accepts the user
name and password from Vista Plus and returns an authentication code indicating
whether the entries are acceptable.
Note

While some users may use external authentication while other users use a
different type; all users who use external authentication must use the same
authentication routine. You cannot set different external routines for different
users.

On both UNIX and Windows servers, this routine should be a dll file. The file must:

Accept the user name and password passed to it by the Vista Plus server software.

Validate the user name and password, then return to the Vista Plus server a Boolean
value of True if the user should be allowed to log in, or False if the login request
should be refused. It can validate the name and password entries itself, or pass them
on to a third-party security application for the actual authentication.

On request, Vista Plus technical support can send you a sample authorization module.
You can then use this sample as a basis for your own routine. We recommend you use
these steps when creating an authorization module:
1.

Unpack and build the sample authorization module.

2.

Install the sample authorization module on a development machine, where a security

breach would cause no harm.

3.

Test the sample authorization module and familiarize yourself with the code.

4.

Modify the sample, developing your own module.

Warning Do not install the sample authorization module on a live Vista Plus server!
Install your own authorization module on a server with live data only after
it is completely developed, debugged, and tested. If the authorization
module is faulty, it could allow anyone to log into Vista Plus.
When you choose to use an external authorization module, you are taking
responsibility for the security of your Vista Plus database, and you do so at
your own risk. Any breach of security caused or aided by the use of any
external security software is not the responsibility of Open Text
Corporation.
Treat this document and the sample authorization module as you do other
sensitive or company-confidential material. Misuse of the information in
this document or the sample authorization module could jeopardize the
security of your Vista Plus data.

Vista Plus Technical Addendum

19

External Password Authorization

Tip

The sample module is written in C. You must use this code, modified as
described below, as the interface between Vista Plus and your authentication
routine. If you pass the user information on to another routine, it can be
written in any language.

About the Sample Authorization Module

The sample module consists of a kit that can be used to create a simple authorization plugin. The plug-in it creates accepts any user who enters a password of SAMPLE and denies
access if any other password is entered.
The compressed sample file contains these files and directories:
Sample code

Required include files

authorization.c

Actual sample code

AuthSample.cpp

Stub for sample code, required only for

Windows compiles

DLL.h

Include files

UserAuth.h
Make files

20

AuthSample.dsp

Project file when compiling for Windows

Makefile.SunOS

Makefile for Sun Solaris compiles

Makefile.AIX

Makefile for AIX compiles

Makefile.HP-UX

Makefile for HP-UX compiles

Vista Plus Technical Addendum

External Password Authorization

Building the Sample Code

After downloading the sample file to your disk and expanding it, follow one of the
procedures below to build the sample authorization module.

To build under UNIX

Use cd to change to the sample directory, then type the command to build the module for
your operating system. These are the three possible commands:
make -f Makefile.HP-UX
make -f Makefile.SunOS
make -f Makefile.AIX

This creates the intermediate file authorization.o and the final file AuthSample.dll in the
sample directory.
The makefiles support three different targets:

clean Deletes any existing AuthSample.dll and authorization.o files; does not create
new ones.

new Deletes any existing AuthSample.dll and authorization.o files; compiles source
files to create new AuthSample.dll and authorization.o.

all If the source files have changed, deletes any existing AuthSample.dll and
authorization.o files; compiles source files to create new AuthSample.dll and
authorization.o. This is the default.

To use one of these targets, add it at the end of the make command. For example:
make -f Makefile.HP-UX new

To build under Windows

1.

Launch AuthSample.dsp under Microsoft Visual Studio.

2.

Set the Active Configuration to either Release or Debug.

3.

Select Rebuild All.

When finished, the result will be one of the following, depending on your choice in step 2:

.\Release\AuthSample.dll if you chose Release

.\Debug\AuthSample.dll if you chose Debug

Vista Plus Technical Addendum

21

External Password Authorization

Installing a Plug-in Security Module

Installing the sample authorization module, or your own after you have created it, is easy.
Follow these steps:
1.

Place the dll fileAuthSample.dll for the sample modulein the desired directory.
This can be any directory; the bin subdirectory of the Vista Plus server installation
directory is often a good choice. For example, on a Windows server using the default
installation directory, the file would go in c:\Program Files\Open Text\Vista Plus\Vista
Plus Server\bin.

2.

Add a line to server.cfg that points to the new module. In the above example, it would
be:
AUTHENTICATOR=C:\Program Files\Open Text\Vista Plus\Vista Plus
Server\bin\AuthSample.dll

3.

If your server runs Windows, stop and restart the Vista Plus Server service. On a
UNIX server, stop and restart the Vista Plus server process.

Vista Plus will now use the authentication module for any user with User Supplied
Authorization Method selected in the user record.
Warning

To protect Vista Plus security when using a custom module, make sure only
authorized users can change or install the authorization module. Specifically,
make sure:
The server.cfg file is protected against modification, to prevent anyone from
changing the AUTHENTICATOR statement to point to a different
module.
Your authorization module is protected against modification and
replacement.

Using the Sample Module

The sample authorization module is deliberately simple. It serves only as an example of
how an authorization module plugs into the Vista Plus Server. If the user provides a
password of SAMPLE (it must be all upper case), access is allowed. If the password is
anything else, access is denied.
Before writing your own module, we suggest you test the sample module. Create Vista
Plus users with differing authorization methods and see what happens when you log in
using different types of passwords.

Making a Custom Authorization Module

Once you are familiar with when and how passwords are validated by a plug-in
authorization module, you can design your own. Among other benefits, using your own
module could provide one or more of the following:

22

Stronger security

Vista Plus Technical Addendum

External Password Authorization

Simpler administration by using an existing security subsystem to validate users

A log of user log-in attempts, both successful and unsuccessful

The heart of the plug-in authorization module is the routine IsNamePassValid() in

authorization.c. It takes a string for the name and a string for the password, and returns a
Boolean value indicating whether the combination of name and password is valid. TRUE
indicates that the user is valid; FALSE denies access. The routine in the sample module,
shown below, merely checks for a password of SAMPLE.
/*
* IsNamePassValid()
*
* This routine is the heart of the plug-in authorization system.
Given a
* user name and offered password, the routine validates the
combination and
* returns a Boolean:
.*
TRUE = the combination is correct
*
FALSE = the combination is incorrect, invalid, outdated, etc.
*/
BOOL USER_AUTH_API IsNamePassValid(
const char*UserName,
const char*Password)
{
BOOL
bUserIsValid;
/*
* Put your code here!
* This sample accepts any logon that uses a password of
"SAMPLE".
*/
bUserIsValid = (0 == strcmp(Password, "SAMPLE"));
return bUserIsValid;
} /* IsNamePassValid() */

To create your own authorization module, insert your validation code where indicated in
the sample. This code may validate the entries itself, or it may pass the name and
password on to a separate security program. If it passes the information on to another
program, it must also accept a returned Boolean value and pass that value back to Vista
Plus.
After compiling, be sure to test your module thoroughly; any mistakes or holes in your
code could give unauthorized users access to any or all Vista Plus features and
information. How to install your module so Vista Plus uses it is described in Installing a
Plug-in Security Module on page 22.

Vista Plus Technical Addendum

23

The Migration Process

The Migration Process

The Vista Plus migration processes (VMIdentify and VMTransport) move report
generations from online disk volumes to offline disks or tapes. This section describes how
this is done, the file naming conventions used when storing generation files on an offline
volume, and temporary files and directories VMTransport creates and uses.
This section does not describe how to use VMIdentify or VMTransport. For instructions
on using them and information on migration rules, please see the Vista Plus Server
Administration Guide.

The Migration Process

Migrating report generations from online to offline is a two-step process. First you run
VMIdentify to identify the report generations to be archivedor compressed or
deletedthen you run VMTransport, which actually removes the file(s) for the
generation from the online volume and writes the generation on the offline volume or
device. On a UNIX host, this can be any device; on a Windows host, it can be any mapped
drive letter.
VMIdentify identifies the generations which are due for a migration action based on
migration rules assigned to the various Vista Plus reports. For example, a migration rule
may say to migrate any generations of a report which have not been viewed by any user
for at least 30 days. The rule also includes what offline disk volume or other device the
generations should be moved to.
For each type of migration action (migrate, compress, and so on), VMIdentify creates a lis
file in the vmfiles subdirectory of the Vista Plus server installation directory. Each lis file
contains a header line followed by the generation ID of each generation for which that
action should be performed, one ID per line. There is no other information in the file. The
files which may be created are:

24

File Name

Contains

CompressGens.lis

Generations to compress

migrate_devID.lis

Generations to migrate to this device; ID is the device ID assigned by

Vista Plus when the device was created

migrate_volID.lis

Generations to migrate to this volume; ID is the volume ID assigned by

Vista Plus when the volume was created

DeleteOnline.lis

Online generations to delete; information about the generation is also

removed from Vista Plus database

DeleteRestored.lis

Previously archived and restored generations to delete from online

storage; offline copy is not deleted

DeleteOffline.lis

Generations to delete from offline storage; information about generation

is also removed from Vista Plus database

Vista Plus Technical Addendum

The Migration Process

File Name

Contains

DatabaseDelete.lis

Offline generations to remove from Vista Plus database; generation file

remains in offline storage, but cannot be accessed from Vista Plus;
normally used for generations archived to tape

restore.lis

Offline generations to restore to online volumes; restoration is requested

from Vista Plus viewer clients; VMIdentify cannot create a restore.lis file

VMIdentify creates a particular lis file only if it finds generations to perform that action
for.
VMTransport reads each lis file to see what generations each action should be performed
for, and performs the actions: compressing generations, moving the generations marked
for migration to the specified disk or tape, and so on. During this process, it creates
temporary files as needed; see VMTransports Temporary Files on page 26. It also
updates the Vista Plus database, noting the new location of each offline generation and
removing information about deleted generations. It processes the lis files in the order
given above; if there are multiple files of a typefor example, migrate_vol50 and
migrate_vol51it processes them in order by device or volume number.
Note

While the lis files are generally created by VMIdentify, they do not have to
be. VMTransport will read any text file with the correct name and containing
a list of generations. This means you can, if there is a need to, create a list of
files for VMTransport by hand.

Note

When restoring a generation, the generation file must be on the same offline
volume it was migrated to. If the file has been moved to another volumefor
example, by an archiving program outside Vista PlusVMTransport will not
be able to find it and restore the generation.

Offline Volume Structure

When VMTransport moves report generations to an offline volume, it compresses each
generation into a single file and writes the file to the directory defined as the root path for
the volume. The file name format is:

On a UNIX host: Genid.tar.z

On a Windows host: Genid.zip

id is the ID of the generation being archived. For example, when you migrate generation
4242 on a UNIX host, it creates Gen4242.tar.z in the root path of the offline volume.
Offline volumes do not contain a directory structure similar to that used in online
volumes of the report warehouse; all files are created in the root path. See page 14 for a
description of the directory structure of online volumes.
Note

When moving a generation from one online volume to another, VMTransport

simply moves the generation directory into the directory structure on the
destination volume. It does not compress the generation. The structure of
online directories is described on page 14.

Vista Plus Technical Addendum

25

The Migration Process

VMTransports Temporary Files

When migrating generations to an offline volume, VMTransport first compresses them.
This compression is a two-step process. The first step occurs in the generation directory;
the second is done in the temporary directory defined by the TMP or TMPDIR statement
in the server.cfg file (see page 12):
Note

In this section, GID always indicates the generation ID and RID always
indicates the report ID.

1.

First, VMTransport compresses all of the files in the generation directory into a single
file, also in the generation directory, called GID.zip.

2.

It then creates two subdirectories of the temporary directory defined by TMP or

TMPDIR. These directories are called vmt_n\GenGID and vmt_n\RptRID, where n is
a number assigned by the operating system.
VMTransport creates an additional compressed file in each of these directories:

In vmt_n\GenGID, it creates GenGIDUnload, containing the information from

the generation record in the Vista Plus database.

In vmt_n\RptRID, it creates RptRIDUnload, containing information from the

report record in the Vista Plus database.

3.

Next, it copies the GID.zip file from step 1 into the vmt_n\GenGID directory.

4.

Finally, in the temporary directory, it creates a single compressed file, called

GenGID.zip, containing the vmt_n\GenGID and vmt_n\RptRID directories. This is the
file that is actually moved to the offline volume.

5.

If the generation is being migrated to on offline tape volume, the GenGID.zip file is
copied to another subdirectory of the temporary directory, called vmt_n\VolID, where
VolID is the volume ID of the destination volume. All of the generations being
migrated to the volume are then copied to it from vmt_n\VolID, in one operation.

For example, for generation 789, belonging to report 321, VMTransport would create
these files:

In the generation directory, 789.zip

In the temporary directory, the subdirectory vmt_n\Gen789, containing the file

Gen789Unload. It would also copy 789.zip to the vmt_n\Gen789 directory.

In the temporary directory, the subdirectory vmt_n\Rpt321, containing the file

Rpt321Unload.

In the temporary directory, Gen789.zip, containing the vmt_n\Gen789 and

vmt_n\Rpt321 directories.

It then copies Gen789.zip is then copied directly to an offline disk volume, or to the
vmt_n\VolID temporary directory for an offline tape volume.

26

Vista Plus Technical Addendum

The Migration Process

Note

For the compress migration action, VMTransport only performs step 1 of this
process. It compresses all of the files in the generation directory into a single
file, but it does not take any information from the database.

During a restore, the process is reversed; the generations compressed archive file is
copied from the offline volume to the temporary subdirectory, where it is uncompressed
and the uncompressed files are copied back to the report warehouse.
All temporary directories and files created by VMTransport are removed when the
current processing option is complete for all generations.
Note

VMTransport does not create any temporary files when moving a generation
from one online volume to another online volume. It merely moves the
generation files directly to the destination volume.

Other Migration Temporary Files

VMIdentify and VMTransport create two other temporary files, one in the Vista Plus
server home directory and one in the servers vmfiles subdirectory:

Both VMIdentify and VMTransport create the file VMITsync.lck in the server home
directory when they begin running, and delete it when they finish. While this file
exists, you cannot start either VMIdentify or VMTransportthis keeps you from
running both programs at the same time, or running two instances of either one. If a
problem occurs and VMIdentify or VMTransport is interrupted, you may need to
delete this file by hand.

When it finishes, VMTransport creates the file VMTrsync.lck in the vmfiles

subdirectory. While this file exists, you cannot start VMTransport without specifying
a specific list file for it to use (see the Vista Plus Server Administration Guide). When
VMIdentify runs, it deletes this file.

Both VMITsync.lck and VMTrsync.lck are 0-byte files with no content.

Finally, VMTransport also uses the generation-specific act files in the Vista Plus servers
.locks subdirectory. As it processes each generation it increments and decrements the
count, or creates and removes the act file, as necessary. See page 13 for more information.

Vista Plus Technical Addendum

27

Font Support in Web View When Using Xvfb

Font Support in Web View When Using Xvfb

As described in the Vista Plus Server Installation Guide, Web View must be started in a
graphics environment. If it is not, it will not be able to display graphics, and will not be
able to display any report in layout mode.
As one possible solution to this, if the Web server host does not have a graphics
environment available, the Installation Guide suggests using a virtual X server such as Xvfb
(X server Virtual Frame Buffer) to provide a graphics environment for Web View. Xvfb
supports only a limited set of fonts. When the Web server uses Xvfb to provide a graphics
environment, Web View can display only fonts which are supported by Xvfb. If a report
contains fonts which Xvfb does not support, a users browser may hang when they
attempt to view the report in Layout mode, or the report may not display correctly.
There is nothing Web View can do about this situation: if a font is not supported by the
graphics environment it is running in, Web View cannot display it. To avoid this problem,
run Web View on a Windows host or, on a UNIX host, start it using OpenWin or CDE to
give it a graphics context.

28

Vista Plus Technical Addendum

Using Vista Plus Utilities

Using Vista Plus Utilities

Vista Plus includes two utilities to help you find and fix problems that could occur if the
Vista Plus database and the generation files in the report warehouse get out of synch.
These are check_gensthe check generations utility, and del_gensthe delete
generations utility.

The check_gens Utility

check_gens compares the contents of the Vista Plus report warehouse and the Vista Plus
database, looking for inconsistencies. Specifically, it will find and correct these conditions:

Empty generation directories in the report warehouse.

Generation files and directories in the report warehouse that dont have
corresponding records in the database.

Mismatches between the compression flag in the database and the generation files in
the report warehouse.

Generation files which are incorrectly in the top-level directory of the report
warehouse.

Generations with an original file size of zero in the database.

Index files in the report warehouse without corresponding entries in the database.

You can run check_gens just to report on these conditions, without making any changes,
or add options to correct some or all of them, as described below.
Note

In general, you should use check_gens only when recommended by, and
under the supervision of, Vista Plus Technical Support.

check_gens Command Format

check_gens [-c] [-e] [-g] [-i] [-p] [-s] [-nsize] [-Ppath] [-D] [-v]
[-h]
Option Meaning
-c

Fix compression flags so they accurately reflect the state of the generation files in the
report warehouse.

-D

Verbose mode; includes more detail in the log files.

-e

Remove empty directories from the report warehouse.

-g

Remove report warehouse files for generations without records in the database.

-h

Display the list of options.

-i

Remove index files without records in the database.

-n

Retrieve size generation records from the database at one time. If you do not include
this parameter, the number of records retrieved at once is set by the
VMIDENTIFY_CHUNKSIZE statement in server.cfg.

Vista Plus Technical Addendum

29

Using Vista Plus Utilities

Option Meaning
-P

Check only generations in the directory specified by path. Path must be the
complete path to a directory in the report warehouse. This lets you run check_gens
for a single generation or only certain generations. If you have multiple online
volumes, be sure to enter the path for the correct one. The directory structure of the
report warehouse is described on page 14.

-p

If any generation files are stored in the top-level directory of the report warehouse,
move them to the correct directory.

-s

If the original file size for a generation is zero, make it non-zero.

-v

Display the check_gens version.

To run check_gens
Warning

Before starting this process, make sure VMTransport is not running. If it is,
wait for it to finish before stopping the Vista Plus server.

1.

If the Vista Plus server is running on UNIX, make sure you are logged in as the user
who owns Vista Plus. On a Windows host, open a DOS command window.

2.

On UNIX, change to the Vista Plus home directory.

On Windows, change to the Vista Plus bin subdirectory.

3.

Stop the Vista Plus server. This disconnects any clients and stops any capture or
migration processes that are running:

On UNIX:

On Windows:

./vista_service t
vista_service t

4.

Open the server.cfg file and change the LOGGING setting to VERBOSE. Save the
change and close the file.

5.

Run check_gens in scan mode; this looks for possible problems in the generation
table:

On UNIX:

On Windows:

./check_gens
check_gens

You can include the -n and/or -D options if desired.

6.

When check_gens finishes, review its log files:

more check_gens.debug.log
more check_gens.log

30

Vista Plus Technical Addendum

Using Vista Plus Utilities

7.

If there are errors in the log files, you can run check_gens again, using one or more of
the repair options:
Error Type

Option to Use

Generation XXX not found in Database

check_gens g

Fix generations paths

check_gens p

Improper compression flag for Generation XXX

check_gens c

Remove obsolete indexes

check_gens i

Located empty directory

check_gens e

Note

8.

Important! If you have any questions or concerns about the contents of the
check_gens log files e-mail a copy to technical support for review before
using any of the repair options.

After finishing with check_gens, be sure to change LOGGING back to its usual
setting in server.cfg, then restart the Vista Plus server:

On UNIX:

On Windows:

./vista_service s
vista_service s

The del_gens Utility

Like check_gens, del_gens compares the contents of the Vista Plus database and the
report warehouse. Specifically, del_gens looks for cases where there is an entry for a
generation in the database but there are no files for that generation in the report
warehouse. It can also look for orphaned generation entries: generations listed for a
report that does not exist in the database. When it finds one of these inconsistencies, it can
remove the entry for the generation from the database.
del_gens checks only that the directory for the generation exists and has at least one file in
it. It does not verify that all the files that should exist for a generation are present. Except
when looking for generations which have no report, it does not check offline generations.
Note

In general, you should use del_gens only when recommended by, and under
the supervision of, Vista Plus Technical Support.

Vista Plus Technical Addendum

31

Using Vista Plus Utilities

del_gens Command Format

del_gens [-gstart-id] [-zend-ID] [-o] [-d] [-nsize] [-D] [-v] [-h]
Option

Meaning

-d

Display qualifying generations without removing

-D

Verbose outputlog file includes each directory that was examined

-g

Remove obsolete generations with IDs above the start-ID number

-h

Display the list of options

-n

Report list chunk size

-o

Remove generations that have no report; this option checks both online and offline
generations

-v

Display version information

-z

Remove obsolete generations with IDs below the end-ID number

To run del_gens
Warning

Before starting this process, make sure VMTransport is not running. If it is,
wait for it to finish before stopping the Vista Plus server.

1.

If the Vista Plus server is running on UNIX, make sure you are logged in as the user
who owns Vista Plus. On a Windows host, open a DOS command window.

2.

On UNIX, change to the Vista Plus home directory.

On Windows, change to the Vista Plus bin subdirectory.

3.

Stop the Vista Plus server; this disconnects any clients and stops any capture or
migration processes that are running:

On UNIX:

On Windows:

./vista_service t
vista_service t

4.

Open the server.cfg file and change the LOGGING setting to VERBOSE. Save the
change and close the file.

5.

First, run del_gens in display-only mode to see what generations may need to be
deleted:
del_gens -d [-o]

Include -o if you want to look for orphaned generations (ones whose report does not
exist in the database) rather than for generations that dont have files in the report
warehouse.

32

Vista Plus Technical Addendum

Using Vista Plus Utilities

6.

After reviewing the results of del_gens -d, you can run one of these commands to
delete unneeded generation records:

To remove the database records for generations that do not have files in the
report warehouse:
del_gens [-g startID] [-z endID]

Include -g startID to remove records only for generations with an ID above

startID. Include -z EndID to remove records only for generation with an ID
below EndID. You can include both to remove generations only within a
specific range of IDs.

To remove the database records for orphaned generations:

del_gens -o

7.

After finishing with del_gens, be sure to change LOGGING back to its usual setting
in server.cfg, then restart the Vista Plus server:

On UNIX:

On Windows:

./vista_service s
vista_service s

Tip

del_gens -o deletes only the database record for orphaned generations. There
could be files for those generations left in the report warehouse. We
recommend you run check_gens to look for this and delete any unneeded
files.

Vista Plus Technical Addendum

33

Adding Index Values to the Database

Adding Index Values to the Database

Starting with version 5.5, Vista Plus can store index values for captured generations in the
database as well as in the generation directory in the report warehouse. Having the index
values in the database can greatly speed up global index searches and searches across
multiple report generations, and also allows you to include offline generations in these
types of searches.
Unless the server.cfg file includes the statement StoreIndexValuesInDB=0 to turn the
feature off, Vista Plus 5.5 and later add index values to the database when a generation is
indexed during or after capture. However, generations captured before this feature was
available have index values stored only in the report warehouse. You can add these
existing index values to the report warehouse using the convert_gens utility.
convert_gens reads index values from the Vista Plus report warehouse and, if they are not
already there, inserts them into the Vista Plus database. It can do this for all generations,
only those captured after a certain date, or only a certain number of generations (for
example, the 50 most recent) for each report.
convert_gens only copies existing index values from the report warehouse to the
database. It does not perform a re-index based on current index definitions. Also, since it
uses values from the report warehouse, it works only on online generations. To add index
values for an offline generation to the database, you must restore the generation. After
you run convert_gens, you can delete or remigrate the generation and it will remain
searchable.
Depending on the number of generations and index values involved, convert_gens can
take a considerable time (up to hours or even days, depending on the number of index
values) to add the index values to the database.
Note

34

In most cases, if you are going to add index values to the database,
StoreIndexValuesInDB should either not be present in server.cfg, or should be
set to 1. If it is present and set to 0, index entries for newly captured
generations are not written to the database. The database would contain
entries only for the generations you converted with convert_gens and not for
others, which could mean that global index or multi-generation searches
would miss some matches.

Vista Plus Technical Addendum

Adding Index Values to the Database

Space Considerations
Adding index values to the Vista Plus database increases the size of the database.
However, in the long term, it may either increase or decrease the total space used by Vista
Plus.
The amount of space needed in the database depends on the number, type, and size of the
defined indexes and the number of generations you are converting. You can estimate the
exact amount of extra space running convert_gens will use by calculating the total size of
all the index files for the generations youre converting. These files are in the Vista Plus
report warehouse and have a .i extension. In most cases, the amount of data added to the
database is approximately twice the total size of all the .i files. (In addition to the actual
index values, convert_gens creates a key structure to speed searching, thus requiring the
extra space.) For text (as opposed to numeric) indexes, the requirements could be even
higher, up to three times the .i file size; short text indexes use relatively more space than
longer ones.
If you want to see how much space the .i files use so you can calculate how much space
convert_gens will use, you can add the size of the .i files for all of the generations you will
be converting. However, if any of those generations are compressed, the .i files are
contained in the compressed generation file. In that case, to estimate the total size of the .i
files, you could open a typical generation (which uncompresses its files), see the size of its
.i files, then multiply that by the number of generations.
Despite this size increase in the database, storing index values in the database could
decrease total Vista Plus disk space usage by decreasing the space needed for the report
warehouse. It does this in two ways:

With index values in the database, Vista Plus does not have to uncompress each
generation when you do a generation search or a global index search. A generation is
only uncompressed if you select and open it from the list of search results.

With index values in the database, you can search offline generations. This may let
you move generations offline sooner, freeing space in the online volumes.

Whether the space saved in the report warehouse offsets the additional space required in
the database depends on your specific circumstances.

Vista Plus Technical Addendum

35

Adding Index Values to the Database

convert_gens Command Format

convert_gens [-c] [-d zzz] [-l] [-n xxx]
Option

Meaning

-c

Convert the generations listed in the GensToConvert.txt file. Use after -l.

-d

List only generations captured in the last zzz days. Do not include older
generations. Works only with -l.

-h

Display the list of options.

-l

List generations selected for conversion in the file GensToConvert.txt. Do not write
index values to the database.

-n

List only the most recent xxx generations for each report. Do not include older
generations. Works only with -l.

Note

If you want to stop convert_gens while it is running, press one of these keys:
p to pause the process but not exit the program.
r to resume after pausing.
x to cancel. Any values already added to the database remain there. A
message tells you how many generations have been converted.
These options are available only while index values are being written to the
database, not while convert_gens is compiling the list of generations to
convert. This means they have no effect if you included the -l option.

Running convert_gens
There are two ways to use convert_gens: you can use one command to convert all existing
online generations, or you can first create a list of the generations to convert, optionally
limited by date or number of narrations per report, then convert the listed generations
using a second command.
Tip

You may want to run del_gens before convert_gens to check for missing
generation files. convert_gens cannot convert a generation which is in the
Vista Plus database but does not have files in the report warehouse. If it finds
one, it writes an error to server.debug.log.

To convert all generations

1.

If the Vista Plus server is running on UNIX, make sure you are logged in as the user
who owns Vista Plus. On a Windows host, open a DOS command window.

2.

On UNIX, change to the Vista Plus home directory.

On Windows, change to the Vista Plus bin subdirectory.

3.

Type this command:

convert_gens

Tip

36

On UNIX, include ./ before the command.

Vista Plus Technical Addendum

Adding Index Values to the Database

4.

convert_gens displays a message warning that its about to make database changes
and asks if you want to continue. Type y and press Enter to continue or just press
Enter to exit the program without converting any generations.
The conversion may take a considerable time, depending on the number of
generations. You can use the p, r, and x options while it is running, as described
above. When convert_gens finishes, it tells you how many generations were
converted.
Note

Important! Do not run two instances of convert_gens at the same time. This is
not supported.

To list generations, then convert them

On UNIX, include ./ before all commands.

Tip
1.

If the Vista Plus server is running on UNIX, make sure you are logged in as the user
who owns Vista Plus. On a Windows host, open a DOS command window.

2.

On UNIX, change to the Vista Plus home directory.

On Windows, change to the Vista Plus bin subdirectory.

3.

Type this command:

convert_gens -l [-n xxx] [-d zzz]

This listsin the file GensToConvert.txtthe generations to be converted. The list

includes the generation ID, report name, generation description, and the name of the
captured file.
If included, -n xxx lists only the most recent xxx generations for each report. If a
report has more than this many online generations, additional ones are not included.
If included, -d zzz lists only generations captured in the last zzz days. Older
generations are not included.
4.

To read the list from the file and perform the conversion, type this command:
convert_gens -c

5.

convert_gens displays a message warning that its about to make database changes
and asks if you want to continue. Type y and press Enter to continue or just press
Enter to exit the program without converting any generations.
The conversion may take a considerable time, depending on the number of
generations. You can use the p, r, and x options while it is running, as described
above. When convert_gens finishes, it tells you how many generations were
converted.
Note

Important! Do not run two instances of convert_gens at the same time. This is
not supported.

Vista Plus Technical Addendum

37

Removing Index Values from the Database

Removing Index Values from the Database

As discussed in Space Considerations on page 35, storing index values to the Vista Plus
database in addition to the report warehouse can greatly increase the size need for the
database. For this reason, you may change your mind and decide that the benefits of
having the index values in the database are outweighed by the extra space needed. If this
happens, you can set StoreIndexValuesInDB=0 in the server.cfg file so future captures and
indexing operations do not create entries in the database. You can also use the revert_gens
command to remove existing index entries from the database.
Warning

Before changing the StoreIndexValuesInDB setting or using revert_gens,

contact Vista Plus Technical Support. You should be certain of your decision
before making the change. Changing the StoreIndexValuesInDB setting from
1 to 0, then back again at a later time can cause data inconsistencies that can
hurt Vista Plus performance and potentially lead to other problems.

After you use revert_gens, you or your database administrator will need to perform
further actions to reclaim the space that had been used by the database index entries. How
to reclaim the space is described after the revert_gens command description.
revert_gens removes index entries only from the Vista Plus database. The index entries in
the report warehouse are not affected. As long as the IndexSearchMethod parameter in
server.cfg is set to either file or auto, all index searches will continue to work after you
run revert_gens.

revert_gens Command Format

revert_gens [-c] [-l] [-d xxx] [-n zzz] [-h]
Option

38

Meaning

-c

Revert the generations listed in the GensToRevert.txt file. Use after -l.

-d

List only generations captured in the last xxx days. Do not include older
generations. Works only with -l.

-h

Display the list of options.

-l

List generations selected for conversion in the file GensToRevert.txt. Do not write
index values to the database.

-n

List only the most recent zzz generations for each report. Do not include older
generations. Works only with -l.

Vista Plus Technical Addendum

Removing Index Values from the Database

Using revert_gens
Tip

In almost all cases, before you use revert_gens, you should make sure the
server.cfg property StoreIndexValuesInDB is set to 0. If it isnt, index values
will continue being added to the database for new generations and
generations that are re-indexed.

There are two basic ways to use revert_gens:

Run it without the -l option to find and immediately revert all generations that have
index entries in the database.

Run it first with the -l option to create a list of generations that need to be reverted,
then run it with the -c option to remove the index entries from the database. With this
method, you can use the -d or -n option to limit the generations affected.

Here are some example commands:

revert_gens
Removes all index entries for all generations from the database.

revert_gens -l
Lists all generations which have index entries in the database. The list is in the file
GenstoRevert.txt.

revert_gens -l -n10
Creates a list in GensToRevert.txt of generations with index entries in the database. The
list include only the most recent ten generations for any report. If a report has more
than ten online generations, older generations are not listed.

revert_gens -c
Removes from the database the index entries for the generations listed in
GensToRevert.txt.

Note

If you want to stop revert_gens while it is running, press one of these keys:
p to pause the process but not exit the program.
r to resume after pausing.
x to cancel. Any values already removed from the database remain
deleted. A message tells you how many generations have been reverted.
These options are available only while index values are being deleted from
the database, not while revert_gens is compiling the list of generations to
revert. This means they have no effect if you include the -l option.

Vista Plus Technical Addendum

39

Removing Index Values from the Database

Reclaiming the Database Space

revert_gens removes the index entries for the affected generations from the Vista Plus
database tables. However, to reclaim that space so it can be used for other purposes
requires additional actions at the database level. The procedure to do this differs
depending on whether you use Oracle or MySQL for the Vista plus database.

Reclaiming the Database Space in Oracle

The Oracle database software includes several different ways to reclaim unused space in
data files, such as export/import, manual tablespace reorganization, and Enterprise
Manager tablespace reorganization. Your Oracle DBA can decide which method to use
and perform the appropriate procedure to reclaim the space.

Reclaiming the Database Space in MySQL

How to free the database space previously used by the index entries depends on whether
the MySQL my.ini file sets innodb_file_per_table to 1 or 0.
Note

Either of these procedures should be performed by the MySQL database

administrator.

If innodb_file_per_table is set to 1, follow this procedure:

1.

Use the mysql command to start MySQL:

mysql -u name -p password

Name is the name of any MySQL user with permission to read and write the Vista
Plus database. Password is the password for that user.
2.

At the MySQL prompt, enter:

OPTIMIZE TABLE vp_index*

This optimizes all Vista Plus index tables. This could include some tables which were
not affected by the revert_gens command, and therefore have no space to be
reclaimed. Optimizing these tables does not do any harm. However, if you want to
optimize only the tables affected by revert_gens, you would need to look at the list of
affected generations, determine which reports those generations belong to, and see
what indexes are defined in those reports. You could then look in the index definition
table to see the table names for each index and optimize only those tables. It is
generally easier to optimize all the index tables.

40

Vista Plus Technical Addendum

Removing Index Values from the Database

If innodb_file_per_table is not set to 1, the only way to make the space available is
to dump all the data from the ibdata file, delete the file, then create a new file and restore
the data to it. Follow this procedure:
1.

Stop the Vista Plus server

2.

Use mysqldump to dump all your InnoDB tables:

mysqldump -u root -p password -A > dump.sql

Password is the password for the MySQL root user.

3.

Stop MySQL server.

4.

Remove all the existing tablespace files, including the ibdata and ib_log files. If you
want to keep a backup copy of the information, copy all the ib* files to another
location before the removing the files in your MySQL installation.

5.

Remove any .frm files for InnoDB tables.

6.

Edit the MySQL my.ini file and set innodb_file_per_table to 1.

7.

Configure a new tablespace.

8.

Restart MySQL server.

9.

Import the dump files.

mysql -u root -p password < dump.sql

Password is the password for the MySQL root user.

10. Restart the Vista Plus server.

Vista Plus Technical Addendum

41

Tuning Vista Plus Server Configuration

Tuning Vista Plus Server Configuration

Many configuration settings, both within Vista Plus and at the operating system and
database levels, interact to affect many aspects of Vista Plus performance: memory and
other system resource use, general response time,. search performance, and more.
Adjusting some of these settings can also help avoid certain types of errors caused by
exceeding set limits or running out of allocated resources. The following sections describe
several areas where you may be able to change configuration settings or resource
allocation to improve Vista Plus behavior and performance. They include:

Scalability configuration options

Kernel parameter settings on Sun Solaris and HP-UX

Adjusting MySQL memory limits

Preventing lost connections to an Oracle database

Improving search performance

Scalability Configuration Options

Scalability refers to Vista Pluss ability to maintain good performance and reliability as the
number of connected users and captured reports increases, without adversely affecting
other applications. Correctly setting certain configuration optionsduring installation, in
the server.cfg file, or in the warehouse configurationcan help ensure the best
performance in your installation, and can minimize the network and server resources
Vista Plus uses.
There are two areas you can configure which affect scalability: the way in which
connections between clients and the Vista Plus server are handled and how many server
processes may run at one time. Both of these areas are discussed below.

Client Server Connections

Two types of connections are possible between the Vista Plus server and any of its client
programs: persistent and short-lived. A persistent connection exists for as long as the client
program is connected to the server, whether or not the client and server are exchanging
information. A short-lived connection is opened whenever the client requests information
from the server and closed after the server sends the data back to the client. A persistent
connection can give slightly better performance to the client using it, since it does not have
the overhead of re-opening the connection for each information request. However, since
persistent connections remain open longer, they use more resources on the Vista Plus
server.

42

Vista Plus Technical Addendum

Tuning Vista Plus Server Configuration

The type of connection used is set for each server by the vadmin
ModifyWarehouseConfig commands ctype parameter. You can set ctype to one of three
values:

p for persistent connections

s for short-lived connections

u for user-selectable

The default is for persistent connections for compatibility with previous Vista Plus
releases. However, short-lived connections are generally more efficient and result in the
best overall client-server performance. If conserving server resources is not an issue, you
can use persistent connections, which may reduce response time for individual clients. If
ctype is set to user-selectable, individual users can select short-lived or persistent
connections when they log on using the Windows Client or Web View. Other clients
vadmin, Server Admin, and capture clientsuse persistent connections.
When using short-lived connections you can set the amount of time before a connection is
shut down using the ModifyWarehouseConfig ctimeout option. ctimeout specifies a
number of seconds from 1-30; if a short-lived connection has no activity for this long, it is
shut down and has to be re-opened the next time the client communicates with the server.
This means that connections that are in constant communication with the serverfor
example, when a file is being sent to the server to be capturedare not shut down and reopened between every packet of data. Instead, they stay open until there is a period of
time with no data being sent. For best performance, we recommend setting ctimeout to
one or two seconds; client performance can degrade if it is much higher than that,
especially at large installations. Setting a longer time-out value reduces the benefits of
short-lived connections, as it leaves more connections open for longer periods of time; if
you have a high time-out value you may need to increase the
MAX_WORKER_PROCESSES setting to allow more concurrent client connections (see
the next section).
Note

Important! Regardless of server host resources, most UNIX servers cannot

support more than 200-300 persistent connections; Windows servers cannot
support more than about 450. We highly recommend short-lived connections
for installations which approach or exceed these limits.

Server Processes
Note

This is a greatly simplified description of the internal structure of the Vista

Plus server software. It is included only to give some background for the
server.cfg settings discussed later in the section.

The Vista Plus server is made up of many different pieces of software. When the server
is started, what actually starts is a process called the Server Pool Monitor. This Monitor
spawns child tasks as needed to handle communication between the Vista Plus server and
clients. These child tasks are called Worker Processes. The Worker Processes accept
information and requests from the clients and handle all communication with the clients.

Vista Plus Technical Addendum

43

Tuning Vista Plus Server Configuration

As needed, they pass on requests to Service Processes, which perform server functions such
as report capture, indexing, and so on.
Each Worker Process handles communication with one client process at one time. If you
are using persistent client connections, the maximum number of Worker Processes limits
the number of users who can connect to the server at one time. If you use short-lived
connections (see the previous section), more users can be active, since not all users will
actually have a connection open with the server at any one time.
When a client requests a connection with a Service Process, the request is placed in the
connection queue. If there is no Worker Process available, the request waits until one is
available. If no Worker Process becomes available before a certain time has elapsedyou
can set the time periodthe server tells the client it cannot make the connection.
In the server.cfg file, you can set these values that affect how many connections are
available and how they are handled:

The INITIAL_WORKER_PROCESSES statement sets the number of Worker

Processes that the Server Pool Monitor starts when it begins. The default is three. This
is also the minimum number of Worker Processes that can be running; the server will
leave this many processes active even if there are no client connections.

The MAX_WORKER_PROCESSES statement limits the number of Worker Processes

that can exist at the same time. The default is 100. As stated above, if you use
persistent connections, this also sets the maximum number of users who can log on to
the server at once; sites using persistent connections will need to set this value higher
than similar sites using short-lived connections. The time-out period for short-lived
connections also affects the number of Worker Processes needed.
Large installations will need to increase MAX_WORKER_PROCESSES. You will
want to increase it if users receive cannot communicate with server error messages
more than once in a great while. Increasing the CONNECTION_QUEUE and/or
CONNECT_TIMEOUT settings (described below) can also reduce the number of
unsuccessful attempts to connect to the server.

The MAX_SERVICE_PROCESSES statement sets the maximum number of Service

Processes which can exist at one time. The default is three per service type. There are
three types of services: action processing, indexing, and report parsing. Again, larger
installations will want to increase this value; the maximum setting is 100. Since at any
given time many connected clients will not be using a Service Process, this setting
does not need to be as large as MAX_WORKER_PROCESSES.

The CONNECTION_QUEUE statement sets the maximum number of client requests

that can be waiting for a Worker Process at one time. The default value is 100; if you
set this to 0, there is no connection queuing and the server tells the client immediately
if there is no Worker Process available when the client requests one. The maximum
setting is 1000.
While the CONNECTION_QUEUE does not increase the number of people who can
connect to Vista Plus, it can keep users from receiving errors by holding the
connection request until a Worker Process is available. In general, if your

44

Vista Plus Technical Addendum

Tuning Vista Plus Server Configuration

MAX_WORKER_PROCESSES is large enough for everyone who may want to use

Vista Plus, you can have a very small CONNECTION_QUEUE setting. If your system
resources dictate a small MAX_WORKER_PROCESSES, you can reduce the
frequency of refused connections by increasing the CONNECTION_QUEUE setting.

The CONNECT_TIMEOUT statement determines how long a client connection

request will wait in the connection queue before the server tells the client it could not
make the connection. The value is in seconds; the default is 15. If you have users
receiving cannot communicate with server error messages, you may want to
increase this number as well as MAX_WORKER_PROCESSES. However, while this
can decrease the number of failed connections, it also increases the length of time
users wait before they see the error message and can try again. The highest allowed
setting is 600 (ten minutes).
Note

Before being placed in the Vista Plus connection queue, the client request is
processed by an operating system queue. This queue has a fixed size which is
out of the control of Vista Plus. On the AIX operating system, this initial
queue can contain only ten requests. Therefore, you may want to make the
Vista Plus connection queue larger on AIX systems, to prevent the operating
system queue from filling up due to a lack of space in the Vista Plus queue.

Obviously, the interactions between these settings and exactly how they affect your
system performance can be quite complex and depend on many factors: how much
memory and other system resources are available on the server, the number of Vista Plus
users you have, how often and for how long those users use Vista Plus and exactly what
they do while connected, and so on. In general, larger installations will want to allow
larger numbers of worker and service processes while using short-lived connections with
a low time-out setting. However, as you increase the number of simultaneous connections
accessing the Vista Plus database, the limiting factor for performance can switch from how
many users can connect to the server to how fast the server accesses information in the
database. Allowing too many connections at once may result in occasional database error
messages.
For help in adjusting these settings to increase your Vista Plus performance, please contact
customer support.

Vista Plus Technical Addendum

45

Tuning Vista Plus Server Configuration

Recommended Kernel Parameters on the Server Host

The following sections discuss three types of operating system kernel parameters which
can affect Vista Plus server operation: shared memory parameters, semaphore parameters,
and segment size parameters. We discuss what these kernel settings do and how that can
affect Vista Plus. Most of these settings exist on both Sun Solaris and HP-UX server hosts:

All of the shared memory and semaphore settings are found on Solaris. They are in
the etc/system file.

A few of these settings do not exist on HP; they are noted in the tables (with ------- )
and descriptions. The settings that do exist are found in usr/conf/master.d/core-hpux, but
are easiest to check and modify using the sam tool.

The segment size parameters are found only on HP.

If you change any of the parameters described in the following sections, you must reboot
the server for the new setting to take effect.
Warning

The sections below discuss how these kernel parameters affect the operation
of Vista Plus. Open Text Corporation does not claim that the recommended
settings and guidelines below will work on all server hosts or in all
configurations. All recommendations and information apply only to Vista
Plus. Other software on the server host may have its own requirements for
these settings; please see the documentation for your operating system and
for any other software installed on the host for more information.

Note

Vista Plus memory and semaphore use is the same on IBM AIX as on other
versions of UNIX. However, AIX handles these areas differently; there is no
need to modify any AIX kernel settings for Vista Plus.

Shared Memory Parameters

The amount of shared memory Vista Plus uses varies depending on the size of the user
license and the number of report warehouses installed. Vista Plus requires four memory
segments, of varying sizes:

46

Type

Size in Bytes

Description

Warehouse

44 + (24 * number
of seats in the
license)

There is one segment for each warehouse on the system.

Vista Plus always requests enough memory for at least 1000
seats, even if fewer users are licensed; the maximum
number of users it will allocate memory for is 10,000.
Therefore, the minimum size of this segment is 24,044 bytes
and the maximum is 240,044 bytes. If the license is for more
than 1000 users but fewer than 10,000, you can use the
formula to determine the size of the segment.

Vista Plus Technical Addendum

Tuning Vista Plus Server Configuration

Type

Size in Bytes

Description

License

424 + (408 *
number of seats)

Again, Vista Plus will request enough memory for at least

1000 seats but not allocate more memory than needed for
10,000, resulting in a segment of from 408,424 bytes to
4,080,424 bytes. Use the formula to find the actual segment
size in your situation based on the number of users. There is
only one of these segments, regardless of the number of
warehouses.

Communicatio
n

4924

Used by Vista Plus to talk with its services; there is one

communication segment per warehouse.

Tracing

40812

There is also one tracing segment per warehouse.

Note

Vista Plus user licenses are based on the total number of users, not the
number of concurrent users. We have found allocating memory for 10,000
concurrent users on a single host to be sufficient even in large installations.

This table shows the operating system shared memory settings (all values are decimal):
etc/system Setting

Sun Maximum
Value

HP Minimum
Value

HP Maximum
Value

set shmsys:shminfo_shmmax

4,294,967,295

2048

1,073,741,824

set shmsys:shminfo_shmmin

4,294,967,295

---------

---------

set shmsys:shminfo_shmseg

32,767

shmmni

set shmsys:shminfo_shmmni

214,783,647

1024

Note

HP also has a shmem parameter. This turns shared memory on and off. It
must be set to 1 to enable shared memory. Vista Plus will not work if shared
memory is turned off.

Here are short descriptions of each setting and how they relate to Vista Plus:

shmmax Is the largest allowed size of a single memory segment. As discussed

above, depending on the number of licensed users, the largest single segment Vista
Plus requires is between 408,424 and 4,080,424 bytes. shmmax must be large enough
to allow a memory segment this size.

shmmin The opposite of shmmax, this is the smallest allowed size of a shared
memory segment. For example, if shmmin is ten and you request a two-byte segment,
you will have problems. This is most commonly set to one; it must be one to ensure
proper Vista Plus operation. This parameter does not exist on HP.

shmseg Is the largest number of memory segments any one process can have at
once. As discussed above, each Vista Plus server needs only four memory segments.

shmmni Sets the total number of memory segments that can be allocated at once,
for all processes. Each segment has a unique identifier; shmmni tracks the total
number of identifiers. For most installations, 100 segments (this is the default for
many Solaris systems) should be more than enough.

Vista Plus Technical Addendum

47

Tuning Vista Plus Server Configuration

Note

On Solaris, the total size of all segments cannot be more then 25% of the total
system memory. This means if you have 1 GB of RAM in your system and you
request six segments of 50 MB each, you would get no more than five of the
requested segments. This is true even if the shmmni and shmmax settings
allow more segments and segments of 50MB or larger.

Semaphore Parameters
Semaphores are basically used to make sure only one action is happening to a piece of
data at a time. This ensures two processes do not try to update the same item at the same
time. Each piece of data for a process can have a semaphore associated with it. To make
the maintenance of the semaphores easier, they are often grouped in semaphore sets. This
allows a process to carry out an action on the set, which applies it to each semaphore in
the set, instead of carrying out the action separately on each semaphore.
Vista Plus uses eleven semaphores for each warehouse on a server. Vista Plus allocates
semaphores in sets of one.
This table shows the operating system semaphore settings:
etc/system Setting

Sun Maximum
Value

HP Minimum
Value

HP Maximum Value

set semsys:seminfo_semmap

2147483647

32767 or (semmni + 2),

whichever is less

set semsys:seminfo_semmni

65535

semmns

set semsys:seminfo_semmns

2147483647

32767

set semsys:seminfo_semmnu

2147483647

nproc 4

set semsys:seminfo_semmsl

2147483647

---------

---------

set semsys:seminfo_semopm

2147483647

---------

---------

set semsys:seminfo_semume

2147483647

smmmns

set semsys:seminfo_semusz

8 * (semume + 2)
See notes below

---------

---------

set semsys:seminfo_semvmx

2147483647

65535

set semsys:seminfo_semaem

2147483647

32767 or semvmx,
whichever is less

Here are short descriptions of each parameter and how they can affect Vista Plus
operation.

semmap When the system is first booted, one big chunk of memory is used for
semaphores. As processes request memory for semaphores, this chunk gets broken
into many smaller pieces. semmap keeps track of the available parts of this memory.
For example, we start with one big chunk:
AAAAAAAAAAAAAAAAAAAAAAAAAAA

48

Vista Plus Technical Addendum

Tuning Vista Plus Server Configuration

Two processes request semaphores. Each process keeps track of the memory area it
has and semmap keeps track of what is free.
AAAAAAAAAAAAAAAAAAAAAAAAAAA
|
|
|
|
semmap
PID1
PID2

If PID2 releases its semaphores back to the pool it looks like this:
AAAAAAAAAAAAAAAAAAAAAAAAAAA
|
|
|
|
semmap
PID1
semmap

Since semmap is tracking two locations that are not contiguous (they dont touch each
other), it must maintain two entries, one for the first location and another for the second.
The number specified in the semmap setting is the maximum number of separate
locations semmap can keep track of. If the memory gets fragmented into more chunks
than semmap can handle, it loses track of some chunks and they are unusable until the
system is rebooted and the memory gets put back into one big chunk.
Some Solaris systems use a default of 10 for semmap. This will probably need to be
increased for Vista Plus. Remember, this is the maximum number of semaphore chunks
the system can keep track of: as processes request and release semaphores, the number of
chunks can be higher than the total number of semaphores used. With a 4.3 or later server,
especially with multiple warehouses, there can easily be more then ten chunks. (The
system is pretty good about making two neighboring chunks into one big chunk, but it
may not always be able to do this.) semmap will then not be able to track all the chunks
and will essentially lose them until the system is rebooted. This can make Vista Plus seem
to work for a while and then stop. The higher the value, the more fragmentation can be
handled.

semmni Defines how many semaphore sets can be used on the system at one time.
For example, if it is set to ten and two processes each request five semaphore sets, no
other processes will be able to get a semaphore set until one of the processes is
finished with its. Note that semmni limits the number of semaphore sets, not the
number of semaphores. Since Vista Plus allocates its semaphores in sets of one, it
needs as many semaphore sets as it does semaphores, so this setting should be the
same as semmns.

semmns Defines the total number of semaphores on the system, which determines
the amount of memory available for semaphores. Increasing this number increases the
semaphores available in two ways: not only are more semaphores allowed, but
because the size of the memory chunk is increased, the fragmentation of the chunk is
decreased and, in general, bigger segments are available. This makes it more likely for
semmap to be able to keep track of the available chunks and allocate the needed
semaphore memory when it is requested. This is the setting most likely to need
adjustment for Vista Plus, especially on servers with several warehouses installed. If
you see server.debug.log messages saying the service_daemon could not start, it could be
because semmns is too low. If you do need to raise this setting, dont set it too high as

Vista Plus Technical Addendum

49

Tuning Vista Plus Server Configuration

this increases the system overhead needed to keep track of semaphore memory
chunks.

semmnu Defines the number of semaphore undo structures. Basically, if you are
using a semaphore to change something, but want to be able to undo the change if
something goes wrong, the undo structure records what you did and allows you to
undo it. Vista Plus does not use undo structures, so this setting has no effect on Vista
Plus.

semmsl Limits the number of semaphores one process can have. It should be less
then semmns to keep one process from taking all the semaphores on the system. No
single Vista Plus process uses more than three semaphores, so you are unlikely to
need to change this setting for Vista Plus. This parameter does not exist on HP.

semopm When a process has a semaphore, it wants to do operations on that

structure. It can do one command at a time or several at once. semopm limits the
number of operations that can be done to a semaphore at once during a semop call.
Vista Plus performs only one operation per semop call so this setting has no effect on
Vista Plus. This parameter does not exist on HP.

semume Is the maximum number of records any single process can have in an
undo structure. Vista Plus does not use undo structures, so this setting has no effect on
Vista Plus.

semmvx Limits the maximum value of a semaphore. Never set this above 32767,
even though it is possible to do so. Lower is usually not a problem unless it is
extremely low. In most cases you dont need to adjust this value for Vista Plus.

semaem Sets the maximum value of an "adjust on exit" undo element. Vista Plus
does not use undo structures, so this setting has no effect on Vista Plus.

semusz Is the size in bytes for undo structures. This is set to 8 * (semume + 2) when
the system is booted. Vista Plus does not use undo structures, so this setting has no
effect on Vista Plus. This parameter does not exist on HP.

Segment Size Parameters

On HP-UX, the segment size parameters can affect performance of Vista Plus and the
database software it uses. There are three parameters you may need to adjust: maxdsize
(maximum data segment size), maxssize (maximum stack segment size), and maxtsize
(maximum text segment size). We recommend setting all of these parameters to at least
the recommended size for your operating system version.

50

Vista Plus Technical Addendum

Tuning Vista Plus Server Configuration

MySQL Memory Limits

In some cases, displaying an extremely long list of items from a UNIX server host in the
Server Admin client causes a Server unknown error message. This happens when the
MySQL user reaches its defined operating system memory limit while displaying the list.
This symptom has been seen when listing page security definitions when there are tens of
thousands of page securities defined.
When this happens, increase the allowed data size and virtual memory size use for the
operating system user that MySQL runs under. Setting these limits to a larger value, or to
unlimited, can resolve the problem.
The exact commands to use to check and change the data and virtual memory allocations
vary depending on your operating system and shell. In most cases, you can use the ulimit
command to do this. There are also ways other than ulimit to change the data and
memory limits. Refer to your operating system documentation.

Preventing Lost Oracle Database Connections

In rare cases, some Vista Plus customers have had problems where Vista Plus loses its
connection to the Oracle database for no apparent reason. The server.debug.log file shows
the error message ORA-03114: not connected to ORACLE.
The solution to this error and the dropped connection is outside Vista Plus. Implementing
one or more of the recommendations in the DataDirect knowledge base article: 3032 :
ORA-03114 not connected to ORACLE should correct the problem.

Vista Plus Technical Addendum

51

Tuning Vista Plus Server Configuration

52

Vista Plus Technical Addendum

Index

Symbols

files in the report warehouse

epurposing
folder structure for the report warehouse
font support in Web View

.lis files in migration

.locks directory

24
12, 13

B
bundles, temporary files

12

C
capturing reports
see report capture
check_gens utility
29
CIRCULAR_LOGGING
17
client-requested archiving, temporary files
12
connection type
42
timeout period
43
CONNECTION_QUEUE
44
CONNECTION_TIMEOUT
45
convert_gens utility
34

D
database size and index values
del_gens utility
dircapture temporary files

35
31
12

E
emulating printer settings using -x
encryption
and Web View
of passwords
epurposing files
error logging levels
etc/system file on Solaris
external password authorization
process
sample module
warning

Vista Plus Technical Addendum

15
15
14
28

8
11
11
11
15
17
46
19
19
20
19

generations
checking database for
29
deleting
31
epurposing files
15
files for each
15
group link record for ImportFileInfo
3
group record for ImportFileInfo
3
groups, adding or deleting with ImportFileInfo 2
gsr temporary files
13

H
HP-UX
kernel parameters
sam tool
usr/conf/master.d/core-hpux

46
46
46

I
ImportFileInfo
data file format
group link record
group record
user record
password parameter
ptype parameter
setting passwords
index values
adding to database
space considerations
removing from database
reclaiming space
indexing, temporary files
INITIAL_WORKER_PROCESSES

2
2
3
3
3
4
4
4
34
35
38
40
12
44

53

Index

K
kernel parameters
segment size
semaphores
shared memory

R
46
50
48
46

L
lock manager temporary files
LOGGING

12
17

M
MAX_SERVICE_PROCESSES
MAX_WORKER_PROCESSES
message logging levels
migration
.lis files
offline volume structure
VMIdentify temporary files
VMTransport
temporary files
VMTransport temporary files

44
43, 44
17
24
24
25
27
26
27

O
offline volume structure
operating system kernel parameters

25
46

P
PARSE_RENDER_IMAGE
passwords
encryption
external authorization
setting with ImportFileInfo
PCL codes, prepending with -x
performance
connection type
worker processes
persistent connections
prepending PCL codes with -x
printer settings, emulating with -x
printing, temporary files

54

15
11
19
4
8
42
42
43
42
8
8
13

report capture
temporary files
-x option
file location
samples
report warehouse
epurposing files
files
folder structure
repository search, temporary files
requirements
segment size
semaphores
shared memory
revert_gens utility
reclaiming database space

12
8
8
9
15
15
14
13
50
48
46
38
40

S
scalability
connection type
worker processes
searching, temporary files
segment size parameters
semaphore parameters
server pool monitor
server.cfg file
CIRCULAR_LOGGING
CONNECTION_QUEUE
CONNECTION_TIMEOUT
encryption flags
INITIAL_WORKER_PROCESSES
LOGGING
MAX_SERVICE_PROCESSES
MAX_WORKER_PROCESSES
PARSE_RENDER_IMAGE
temporary directory setting
temporary directory settings
TMP
TMPDIR
server.debug.log file
circular logging
service processes
shared memory parameters
short-lived connections

42
42
43
12, 13
46, 50
46, 48
43
17
44
45
11
44
17
44
43, 44
15
12
12
26
26
17
17
43, 44
46
42

Vista Plus Technical Addendum

Index

Solaris
etc/system file
kernel parameters

46
46

T
temporary files
.locks directory
TMP directory
TMPDIR directory
VMIdentify
VMTransport
warehouse parameters setting
TMP directory
TMPDIR directory

12
12, 13
12, 13
12, 13
27
26, 27
12
12, 13, 26
12, 13, 26

U
users
adding or deleting with ImportFileInfo
external password authorization

2
19

V
vadmin
gsr command temporary files
13
ImportFileInfo
see ImportFileInfo

Vista Plus Technical Addendum

ModifyWarehouseConfig
ctimeout parameter
ctype parameter
vconvert temporary files
virtual X server and Web View
VMIdentify
.lis files
temporary files
VMITsync.lck
VMTransport
offline volume structure
temporary files
VMTrsync.lck

43
43
12
28
24
24
27
27
24
25
13, 26, 27
27

W
warehouse parameters
temporary directory
Web View font support
worker processes

12
28
43, 44

X
-x option for report capture
file location
samples
Xvfb and Web View

8
8
9
28

55