Front cover

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher
Configuration of WebSphere Edge Server Scenarios for Web Traffic Express and Network Dispatcher Details of latest enhancements

Cristiane Ferreira Ana Mostardinha Byron Braswell

ibm.com/redbooks

International Technical Support Organization WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher July 2001

SG24-6172-00

Take Note! Before using this information and the product it supports, be sure to read the general information in Special notices on page 483.

First Edition (July 2001) This edition applies to Version 1.0.3 of WebSphere Edge Server for Multiplatforms for use with Windows NT, Windows 2000, AIX, Linux and Solaris operating systems. This document created or updated on August 1, 2001. Comments may be addressed to: IBM Corporation, International Technical Support Organization Dept. HZ8 Building 662 P.O. Box 12195 Research Triangle Park, NC 27709-2195 When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.
Copyright International Business Machines Corporation 2001. All rights reserved. Note to U.S Government Users Documentation related to restricted rights Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp.

Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix The team that wrote this redbook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Special notice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi IBM trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Comments welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Part 1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1. WebSphere Edge Server for Multiplatforms concepts . . . . . . . 3 1.1 WebSphere Edge Server for Multiplatforms . . . . . . . . . . . . . . . . . . . . . 4 1.2 Caching and filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Load balancing and server monitoring capabilities . . . . . . . . . . . . . . . . 6 1.4 What is new in WebSphere Edge Server . . . . . . . . . . . . . . . . . . . . . . . 8 Chapter 2. What WebSphere Edge Server can do for you. . . . . . . . . . . . . 15 2.1 Building high performance Web sites . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2 Who can benefit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.2.1 Content hosting Internet Service Providers. . . . . . . . . . . . . . . . . . . . 18 2.2.2 Corporate Web sites and content aggregators . . . . . . . . . . . . . . . . . 19 2.2.3 Backbone Internet Service Providers . . . . . . . . . . . . . . . . . . . . . . . . 21 2.2.4 Access Internet Service Providers . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Part 2. Network Dispatcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Chapter 3. Network Dispatcher installation . . . . . . . . . . . . . . . . . . . . . . . . 27 3.1 Installing Network Dispatcher on AIX . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.1.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.1.2 Choosing the components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.1.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.1.4 Starting the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.1.5 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.2 Installing Network Dispatcher on Windows NT . . . . . . . . . . . . . . . . . . 40 3.2.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.2.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.2.3 Starting the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.2.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.3 Installing Network Dispatcher on Linux . . . . . . . . . . . . . . . . . . . . . . . . 55 3.3.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.3.2 Choosing the components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Copyright IBM Corp. 2001

iii

3.3.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.3.4 Starting the software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.3.5 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Chapter 4. Network Dispatcher basic configuration . . . . . . . . . . . . . . . . . 69 4.1 Load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.1.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.1.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.1.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 4.2 High availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.2.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 4.2.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 4.2.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 4.3 Using Interactive Session Support . . . . . . . . . . . . . . . . . . . . . . . . . . 127 4.3.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 4.3.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Chapter 5. Advanced features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 5.1 Remote configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 5.2 Collocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 5.2.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 5.2.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 5.2.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 5.3 Load balancing with fixed weights . . . . . . . . . . . . . . . . . . . . . . . . . . 167 5.3.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 5.3.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 5.3.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 5.4 Rules-based load balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 5.4.1 Types of rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 5.4.2 How rules are evaluated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 5.4.3 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 5.4.4 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 5.4.5 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 5.5 Wide area Network Dispatcher support . . . . . . . . . . . . . . . . . . . . . . 191 5.5.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 5.5.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 5.5.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 5.6 Mutual high availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206 5.6.1 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 5.6.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 5.6.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 5.7 Affinity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 5.7.1 Server Directed Affinity API to control client-server affinity . . . . . . . 232

iv

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

5.7.2 Cross port affinity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 5.7.3 Affinity address mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 5.7.4 Rule affinity override . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 5.7.5 Cookie affinity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 5.8 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 5.8.1 Basic logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 5.8.2 Binary logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 Part 3. Web Traffic Express . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Chapter 6. Web Traffic Express installation . . . . . . . . . . . . . . . . . . . . . . . 247 6.1 Installing Web Traffic Express on AIX. . . . . . . . . . . . . . . . . . . . . . . . 248 6.1.1 System hardware and software requirements. . . . . . . . . . . . . . . . . 248 6.1.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 6.1.3 Starting and stopping Web Traffic Express . . . . . . . . . . . . . . . . . . . 260 6.1.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 6.2 Installing Web Traffic Express on Windows NT . . . . . . . . . . . . . . . . 262 6.2.1 System hardware and software requirements. . . . . . . . . . . . . . . . . 262 6.2.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 6.2.3 Starting and stopping Web Traffic Express . . . . . . . . . . . . . . . . . . . 272 6.2.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 6.3 Installing Web Traffic Express on Linux . . . . . . . . . . . . . . . . . . . . . . 274 6.3.1 System hardware and software requirements. . . . . . . . . . . . . . . . . 275 6.3.2 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 6.3.3 Starting and stopping Web Traffic Express . . . . . . . . . . . . . . . . . . . 284 6.3.4 Uninstallation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 Chapter 7. Web Traffic Express basic configuration . . . . . . . . . . . . . . . . 287 7.1 Basic configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 7.1.1 Configuration environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 7.1.2 Using the Configuration and Administration forms . . . . . . . . . . . . . 289 7.1.3 Web Traffic Express Administration settings. . . . . . . . . . . . . . . . . . 291 7.1.4 Configuring the basic settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 7.1.5 Configuration of the ibmproxy.conf file . . . . . . . . . . . . . . . . . . . . . . 307 7.2 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 7.2.1 Configuring the browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 7.2.2 Testing the access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310 Chapter 8. Advanced features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 Security enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Handling header information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.2 Basic user authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.3 User authentication using LDAP . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.4 Blocking URLs using the CyberPatrol plug-in . . . . . . . . . . . . . . . . . 313 . 314 . 314 . 320 . 326 . 336

Contents

8.2 Cache content management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351 8.2.1 Controlling which documents are cached . . . . . . . . . . . . . . . . . . . . 352 8.2.2 Cache freshness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354 8.2.3 Automatic cache refreshing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 8.2.4 Caching of dynamic content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359 8.3 SOCKS client support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367 8.3.1 How to set up flexible-client SOCKS . . . . . . . . . . . . . . . . . . . . . . . . 368 8.3.2 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372 8.4 Proxy auto-configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 8.4.1 How to write a PAC file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374 8.4.2 How to set up an automatic proxy . . . . . . . . . . . . . . . . . . . . . . . . . . 376 8.4.3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 8.5 Proxy chaining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384 8.5.1 How to set up proxy chaining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386 8.5.2 Testing the configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388 8.5.3 Configuration of the ibmproxy.conf file . . . . . . . . . . . . . . . . . . . . . . 390 8.6 Logging and performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 8.6.1 Activity Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391 8.6.2 Configuration for performance improvements . . . . . . . . . . . . . . . . . 397 8.6.3 WTE logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 Part 4. Combining Network Dispatcher and Web Traffic Express . . . . . . . . . . . . . . . 413 Chapter 9. Multiple Web Traffic Express proxy servers. . . . . . . . . . . . . . 415 9.1 Using Autoproxy or Network Dispatcher . . . . . . . . . . . . . . . . . . . . . . 416 9.2 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 9.3 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418 9.3.1 Configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422 9.4 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy . . . . . . . . . . . . . . . . . . . . . . . . . 425 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 10.2 Scenario description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 10.3 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428 10.4 Network Dispatcher configuration file . . . . . . . . . . . . . . . . . . . . . . . 435 10.5 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436 Chapter 11. Content Based Routing (CBR). . . . . . . . . . . . . . . . . . . . . . . . 443 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444 11.1.1 Installation of the CBR function . . . . . . . . . . . . . . . . . . . . . . . . . . . 444 11.2 CBR for HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 11.2.1 CBR scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445 11.2.2 WTE CacheByIncomingUrl directive . . . . . . . . . . . . . . . . . . . . . . . 461

vi

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

11.3 CBR proxy for IMAP and POP3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 11.3.1 Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462 11.3.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463 11.3.3 Configuration file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 11.3.4 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 Part 5. Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473 Appendix A. Java source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 TimeServlet Java source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476 CalcServlet Java source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476 Related publications . . . . . . . . . . . . . . . . . . . . . . IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other resources . . . . . . . . . . . . . . . . . . . . . . . . Referenced Web sites . . . . . . . . . . . . . . . . . . . . . . How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . IBM Redbooks collections . . . . . . . . . . . . . . . . . ...... ...... ...... ...... ...... ...... ....... ....... ....... ....... ....... ....... ...... ...... ...... ...... ...... ...... . . . . . . 479 479 479 479 480 481

Special notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487

Contents

vii

viii

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Preface
As the Internet economy evolves, it is no longer enough to simply establish an online presence to sell your products and services. To compete in this fast-paced, global marketplace, you need to respond rapidly and intelligently to changing application and network demands while simplifying the implementation of new e-business solutions. Availability, scalability and performance are critical to ensuring that your e-business runs smoothly and cost-effectively. WebSphere Edge Server for Multiplatforms allows Internet service providers (ISPs) and enterprise customers to confidently host mission-critical Web sites for intranet and Internet e-business applications. WebSphere Edge Server brings together innovative caching, local- and wide-area load balancing and application-aware quality-of-service routing capabilities. The integration of these functions and their wide array of features can help reduce cost for the site owner while improving customer satisfaction. This redbook is intended for networking personnel and information technology administrators who plan to use WebSphere Edge Server to provide Internet access to end users and distribute Web-accessible content more efficiently. This redbook will help you install, tailor and configure the features and enhancements included in WebSphere Edge Server for Multiplatforms Version 1.0.3. WebSphere Edge Server has two main components: 1. Caching proxy component (also known as Web Traffic Express 3.6) 2. Load balancing component (also known as Network Dispatcher 3.6) Throughout this redbook, we will refer to the WebSphere Edge Server components as Web Traffic Express (WTE) and Network Dispatcher (ND).

The team that wrote this redbook

This redbook was produced by a team of specialists from around the world working at the International Technical Support Organization, Raleigh Center. Cristiane Ferreira is a Network Specialist in the RS/6000 Support and Services team at IBM Brazil. She has nine years of experience in UNIX platforms and has been working with AIX support for the past five years. Her areas of expertise include AIX and Linux systems, TCP/IP networking, firewalls, and networking security.

Copyright IBM Corp. 2001

ix

Ana Mostardinha is a Solution Architect in Brazil. She has four years of experience in AIX and Windows. She holds a degree in Meteorology from the Federal University of Rio de Janeiro and a System Analyst degree from the State University of Rio de Janeiro. Her areas of expertise include AIX, MQSeries and Tivoli Storage Manager. Byron Braswell is a networking professional at the International Technical Support Organization, Raleigh Center. He received his B.S. degree in Physics and M.S. degree in Computer Sciences from Texas A&M University. He writes extensively and teaches IBM classes worldwide on all areas of networking. Before joining the ITSO, Byron worked in IBM Learning Services Development in networking education development. Thanks to the following people for their contributions to this project: Margaret Ticknor Bill Moore Carla Sadtler Gail Christensen International Technical Support Organization, Raleigh Center Craig Lanzen Bob Delima IBM Raleigh Manu Gugnani Wangming Ye Shih-pai Lee Jeff Kaminski Jim Li Jeff Carlson Doug Slade Falguni Shah Dave Holder Chris Nolin Maria Wadlow IBM Pittsburgh Shendong Chen IBM Austin

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Special notice
This publication is intended to help networking personnel and information technology administrators to install and customize WebSphere Edge Server for Multiplatforms. The information in this publication is not intended as the specification of any programming interfaces that are provided by WebSphere Edge Server. See the PUBLICATIONS section of the IBM Programming Announcement for WebSphere Edge Server for more information about what publications are considered to be product documentation.

IBM trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States and/or other countries:
AIX APPN e (logo) IBM MQSeries Operating System/2 OS/2 Redbooks Redbooks Logo RS/6000 SecureWay WebSphere

Comments welcome
Your comments are important to us! We want our IBM Redbooks to be as helpful as possible. Please send us your comments about this or other Redbooks in one of the following ways: Use the online Contact us review redbook form found at:
ibm.com/redbooks

Send your comments in an Internet note to:

[email protected]

Mail your comments to the address on page ii.

Preface

xi

xii

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Part 1

Part

Introduction
In this part, we introduce WebSphere Edge Server for Multiplatforms and its component load balancing and caching proxy functions.

Copyright IBM Corp. 2001

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 1.

WebSphere Edge Server for Multiplatforms concepts

This chapter provides an overview of WebSphere Edge Server and a description of its main components.

Copyright IBM Corp. 2001

1.1 WebSphere Edge Server for Multiplatforms

WebSphere Edge Server for Multiplatforms is Web infrastructure software that addresses the scalability, reliability and performance needs of e-business applications in both local and geographically distributed environments. Its functions incorporate robust, leading-edge caching and load balancing that together compensate for the inherent weakness of the Internet to support critical business applications and expectations. WebSphere Edge Server has been developed using IBMs extensive experience with very demanding Web sites. WebSphere Edge Server for Multiplatforms V1 replaces IBM WebSphere Performance Pack V3.0. The new WebSphere Edge Server is the latest IBM integrated solution for local and wide-area load balancing, content-based quality of service routing, and Web content filtering and caching for multi-vendor Web server environments. WebSphere Edge Server is made up of two main components which allow you to reduce Web server congestion, increase content availability and improve Web server performance: 1. Caching and filtering The caching and filtering component, known as Web Traffic Express (WTE), is a caching proxy server that provides highly scalable caching and filtering functions used in receiving requests and serving URLs. Since tunable caching is capable of supporting high cache hit rates, this component can reduce bandwidth costs and provide more consistent rapid customer response times. 2. Load balancing The load balancing component, known as Network Dispatcher (ND), is a server that is able to dynamically monitor and balance TCP servers and applications in real time. The main advantage of the load balancing component is that it allows heavily accessed Web sites to increase capacity, since multiple TCP servers can be dynamically linked in a single entity that appears in the network as a single logical server. These components can increase the scalability, availability and reliability of your Web site while reducing infrastructure costs. Installation procedures allow the user to select which components to install, and on which machine(s) the selected component(s) should be located. Subject to installation needs and operating platforms, components can coexist on a single machine or can be distributed among multiple machines.

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Refer to Chapter 3, Network Dispatcher installation on page 27 and Chapter 6, Web Traffic Express installation on page 247 for specific hardware and software requirements for WebSphere Edge Server for Multiplatforms components.

1.2 Caching and filtering

Web Traffic Express, the caching and filtering component of WebSphere Edge Server, is both a caching proxy server and a content filter. The advanced caching of this component minimizes network bandwidth utilization and ensures that end users spend less time when retrieving the same content multiple times. This component acts as a gateway for multiple clients and performs basic Web server duties, such as receiving requests and serving URLs. A traditional proxy server receives a request for a URL from a client and forwards it to the destination content server. Web Traffic Express does something more; it can save or cache the Web documents it retrieves, and serve subsequent requests for those documents from its local cache. The client gets the requested information faster and network bandwidth utilization is reduced. This component of WebSphere Edge Server also offers other key features of advanced caching, such as: The ability to handle very large caches An option to automatically refresh the cache of the most frequently accessed pages The possibility to cache even those pages where the header information says to fetch them every time A configurable daily garbage collection, to improve server performance and ensure cache maintenance Remote Cache Access (RCA), a function that allows multiple Web Traffic Express machines to share the same cache, thereby reducing the redundancy of cached content

Chapter 1. WebSphere Edge Server for Multiplatforms concepts

Moreover, Web Traffic Express allows you to set content filtering at the proxy server level, rather than or in addition to the browser level, where content filtering could be easily compromised or overridden. This way, depending on the parameters used in the configuration, offensive contents will not be displayed on the clients browser. Content filtering in Web Traffic Express can use: Platform for Internet Content Selection (PICS) rules guiding use of rating labels (such as the Recreational Software Advisory Council on the Internet (RSACI) criteria for inappropriate language, nudity or violence) placed in HTML or HTTP headers, or third-party content rating label distributions. A CyberPatrol filtering plug-in which allows Web content filtering based on the SurfControl database of acceptable and unacceptable Web sites. A lists of URLs and/or sites for which access is to be blocked.

1.3 Load balancing and server monitoring capabilities

The Internet has grown so rapidly over the last few years that you are probably looking for a way to handle your company's share of that traffic. If this growth is not properly handled, users get slow responses or refused connections; this may result in an unsatisfactory user experience, which may in turn cause the user never to visit your site again. Internet sites can become unstable or even fail under critical load conditions. What is needed is a solution that balances the load effectively and protects the user from these bad experiences. Network Dispatcher, the load balancing component of WebSphere Edge Server, has been developed to address these limitations and provide customers with advanced functions to meet their sites scalability and availability needs. It consists of three functions: the Dispatcher, which provides an advanced IP-level load-balancing mechanism for any TCP or UDP protocol. Interactive Session Support (ISS), an intelligent DNS server that can address the limitations of Round Robin DNS while still providing the same DNS interface as before for your clients. the Content Based Routing (CBR) function, which provides full-function load balancing based on information in the HTTP data stream (such as URLs, paths, cookies and so on) or in the POP3 and IMAP data streams. These three functions can be deployed separately or together in various configurations to suit a wide variety of customer application requirements:

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

You can use the Dispatcher function to balance the load on servers within a local area network or a wide area network using a number of weights and measurements that are set dynamically. ISS is a DNS-based load monitoring component (daemon) that can be installed on each of your servers. A group of daemons is called an ISS cell. One of the members of the cell becomes a spokesman for the load monitoring service. You can use the ISS function to balance the load on servers within a local area network or a wide area network using a domain name server round-robin approach or a more advanced user-specified approach. ISS periodically monitors the level of activity on a group of servers and detects which server is the least heavily loaded. ISS provides an observer interface to enable other applications to use the load monitoring service. Observers watch the cell and initiate actions based on the load. Application servers with the ISS load monitor daemon installed can pass periodic load reports to the Dispatcher using the Dispatcher observer. The results of these reports can be factored into the load balancing performed by the Dispatcher. You can use the Content Based Routing function (which must be installed and configured to work together with Web Traffic Express) to load balance traffic based on the content of a clients URL request. CBR also offers a Cookie Affinity feature that allows requests from a particular HTTP client session to be load balanced to the same server for a specified time period. The client session will maintain affinity for a particular server without relying on the IP address of the client. These three components each offer a high availability feature: The Dispatchers high availability feature involves the use of a secondary machine that monitors the main, or primary, machine and stands by to take over the task of load balancing should the primary machine fail at any time. This feature is available on all the platforms where WebSphere Edge Server is supported, without using High Availability Cluster Multi-Processing (HACMP). In ISS high availability, all the nodes in a site work together to eliminate any single point of failure. Multiple CBR machines can be load balanced in turn using Network Dispatcher, therefore granting CBR high availability. The high availability feature provided by the Dispatcher function can be successfully used even in other configurations, for example to guarantee firewall high availability.

Chapter 1. WebSphere Edge Server for Multiplatforms concepts

1.4 What is new in WebSphere Edge Server

In addition to the functions already available in WebSphere Edge Server V1, WebSphere Edge Server for Multiplatforms V1.0.3 enhancements include: Network Dispatcher V3.6 Gigabit Ethernet support Network Dispatcher now supports 1 Gb (gigabit) Ethernet NICs (network interface cards) on all supported platforms: AIX, Red Hat Linux, Solaris, Windows NT and Windows 2000. Gigabit Ethernet is defined by the IEEE 802.3z standard. 802.3z is an enhancement of the IEEE 802.3 standard. 802.3z extends the 802.3 protocol and Media Access Control (MAC) specifications to allow for an operating speed of 1,000 Mb per second. Gigabit Ethernet uses the existing 802.3 frame format and the CSMA/CD access method. On Solaris systems, only Ultra 60 servers support 1 Gb Ethernet NICs. SPARC does not support 1 Gb Ethernet NICs. Quad-port Ethernet support Quad-port Ethernet NICs are supported on all platforms: AIX, Red Hat Linux, Solaris, Windows NT and Windows 2000. Quad-port NICs can operate in one of three modes. These modes are dependent upon the network switch to which the card will connect, the underlying operating system on which the NIC is installed, and the device driver provided by the vendor. Mode 1 1 Port/1 MAC mode. Each port is assigned a unique MAC address. The port, when plugged into the switch, becomes the endpoint of one physical link. Mode 2 Fault tolerant mode. Two or more ports are grouped into a team. One port is assigned a primary role, while remaining ports are given a backup role. The entire team has access to one MAC address, but only the primary port will route traffic over an active link with the switch. If the NIC fault tolerant device driver determines that the primary port is inoperable, the secondary port, which has been inactive, takes over for the primary port, and begins routing traffic to the switch. In this way, protocols that use the interface exposed by a fault tolerant device driver experience no interruption in service if a link fails between a port on the NIC and a port on the switch.

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

 Mode 3 Port aggregation mode, or trunking mode. Two or more ports are grouped into a team. Each team is assigned a MAC address. Protocols then access the team via a logical device driver which controls how the data is multiplexed between the ports in the team. The theoretical bandwidth of the team then becomes the number of ports multiplied by the bandwidth of a single port. This mode of operation requires special switch hardware and software to operate correctly. Network Dispatcher supports quad-port NICs in Mode 1 only. Neither Mode 2 (fault tolerant mode) nor Mode 3 (port aggregation mode, or trunking mode) are supported. Solaris Version 8 support In addition to supporting Solaris V2.6 and Solaris V7 (32-bit mode), Network Dispatcher now also supports Solaris V8 in 32-bit mode. Solaris V8 is 64-bit enabled; however, Network Dispatcher is supported in the 32-bit operating environment only. Red Hat Linux V6.2 support The previous release of Network Dispatcher supported Red Hat Linux V6.1 (Linux kernel version 2.2.12-20, as well as any additional fixes to 2.2.12). This new release of Network Dispatcher adds support for Red Hat Linux V6.2 (Linux kernel version 2.2.16-3, as well as any additional fixes to 2.2.16). Collocated server support for Red Hat Linux Collocation describes the environment where Network Dispatcher does not run on a dedicated machine. Instead, it is installed and runs on one of the server machines that is being load balanced. With this new release of Network Dispatcher, collocation is supported along with high availability at the same time on Red Hat Linux V6.1 and V6.2. Capacity utilization and bandwidth rules The new capacity utilization feature allows Network Dispatcher to measure the amount of data delivered by each of its load balanced servers. It tracks capacity at the server, rule, port, cluster and executor levels by maintaining a new byte counter value for each level: kilobytes transferred per second. New bandwidth rules allow you to balance loads based on the number of kilobytes per second being delivered by a set of servers. The reserved bandwidth rule allows you to allocate a specified bandwidth to sets of servers within your configuration. The shared bandwidth rule allows you to share a specified amount of bandwidth at the cluster or executor level.

Chapter 1. WebSphere Edge Server for Multiplatforms concepts

Server evaluation option The server evaluation option is a new rule option of the ndcontrol rule command. It is used to evaluate the rules condition across only the servers within the rule (new support) or across all the servers within the port (previous support). Generic Routing Encapsulation (GRE) support Generic Routing Encapsulation is an Internet protocol specified in RFCs 1701 and 1702. GRE provides a lightweight encapsulation method for IP and other protocols. Using GRE support, Network Dispatcher encapsulates client IP packets inside IP/GRE packets and forwards them to a server platform that supports GRE. For example, many OS/390 and IBM ^zSeries customers use an OSA Express Network card to provide Internet-intranet connectivity to their mainframe hosts. The OSA Express configuration allows multiple LPARs to share one MAC address, but each has a unique TCP/IP address. GRE support allows Network Dispatcher to load balance packets to multiple server addresses associated with one MAC address. Earlier versions of ND required a one-to-one correspondence between the MAC address and server address. Network Dispatcher implements GRE as part of its Wide Area Network Dispatcher (WAND) feature, thus providing wide area load balancing directly to any server system that can unwrap GRE packets. ND does not need to be installed at the remote site if the remote servers support encapsulated GRE packets. See 5.5, Wide area Network Dispatcher support on page 191 for an example of configuring Wide Area Network Dispatcher support. Advisor fast-failure detection The Advisor is a lightweight client that runs as part of Network Dispatcher. It periodically executes a command to determine the status of the servers. If the command is successful, a server is considered up. If it fails, a server is considered down. In either case, the Advisor informs the Network Dispatcher Manager function. The Manager sets weights for the servers to determine which server should receive new session requests. A down server is given a weight of zero and packets will not be forwarded to it until the server responds to the Advisor. In previous releases of Network Dispatcher, an advisor looped through all the servers configured on a cluster or port. It gathered information on each server and then informed the Manager with a complete update of all servers. With this enhancement, the Manager is updated incrementally after each server is evaluated.

10

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In addition, the timeout values used to determine how long an advisor will wait before declaring a server down are no longer hard coded. The user is now able to configure two timeout values: connecttimeout and receivetimeout. These timers allow the user to configure how generously or aggressively the Advisor marks a server as down. Quiesce enhancement for sticky connections The quiesce command is used to remove a server from the Network Dispatcher configuration for any reason. The quiesce command allows all existing connections to be completed, but prevents any new ones from being sent to the quiesced server. The quiesce command in previous versions of Network Dispatcher did not recognize existing connections that used the affinity/sticky feature. When a server was quiesced, all connections within the stickytime would go to a new server. The quiesce enhancement extends the server quiesce function to recognize existing connections that have the affinity/sticky feature. For example, if a server is quiesced and an existing connection has affinity to the server, Network Dispatcher forwards subsequent new connections from that client to the quiesced server as long as the subsequent new connections arrive before the stickytime expires. This enhancement provides a graceful, less abrupt handling of sticky connections when quiescing servers. You can quiesce a server and wait for the time when there is the least amount of traffic to completely remove the server from the configuration. Enhanced Interactive Session Support (ISS) load balancing of Network Dispatchers In a two-tiered ISS configuration, an ISS DNS monitor determines how to load balance across a lower tier of Network Dispatcher machines. With the previous version of Network Dispatcher, load information provided by scripts running on the ND machines furnished load and CPU information on the ND machine, not the pool of servers behind it. This new version provides a Network Dispatcher self advisor that collects load status information on the load balanced back end servers. This self advisor specifically measures the rate of connections per second on the back end servers and writes the results to the ndloadstat file. Network Dispatcher also provides an external metric called ndload. The ISS agent on each Network Dispatcher machine calls the ndload script which extracts a string from the ndloadstat file and returns it to the ISS agent. Each ISS agent on each of the lower tier Network Dispatcher machines returns the load status value to the ISS monitor for use in determining which Network Dispatcher to forward client requests to.

Chapter 1. WebSphere Edge Server for Multiplatforms concepts

11

SSL proxy-to-server support In previous releases of Network Dispatcher when implementing Content Based Routing (CBR), Secure Sockets Layer (SSL) connections were supported only between the client and the proxy host (WTE). Only HTTP traffic was supported between the proxy and the server. This release of Network Dispatcher extends CBR to support SSL from the proxy to the server, which allows for complete SSL connections from the client to the server. CBR continues to support client-to-proxy SSL and proxy-to-server HTTP along with proxy-to-server SSL. Java 1.3 support The previous release of Network Dispatcher ran on Java 1.1.8. This release of Network Dispatcher requires Java 1.3 on all supported platforms. The requirements for each platform are documented in Chapter 3, Network Dispatcher installation on page 27. Java 1.1.x is no longer supported. Web Traffic Express V3.6 Additional supported system types This release of WTE includes support for Red Hat Linux 6.2 (kernel 2.2.16) and SuSE Linux 6.4 (kernel 2.2.14). In addition, this release of WTE adds support for the Solaris 8 operating system running in 32-bit mode. Only SPARC and Ultra-SPARC chipsets are supported. Solaris 8 on an x86 chipset is not supported. Client-side certificate authentication Previous versions of WTE support using SSL to authenticate a server and use encryption for data passed between the client and the server. This release of WTE includes support for client authentication. WTE will retrieve a client certificate through an SSL session and use it to authenticate the client. Disabling listener threads when SSL is enabled When SSL is enabled in WTE, a listener thread is enabled for the SSL port (typically port 443). However, in previous releases of WTE, enabling SSL did not disable the listener threads for standard HTTP requests (typically ports 80 and 8080); running active HTTP listener threads can possibly make a site insecure on a server performing secure transactions. This release of WTE includes a new directive to disable listener threads for standard HTTP requests. The ability to turn off these listener threads enhances server security.

12

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Use of IP ranges in protection masks This enhancement allows administrators to more easily configure WTE to authenticate requests from specified IP addresses using IP ranges. This avoids the need to create very large protection rule files that could adversely impact proxy performance. Requests over persistent connections WTE can be configured to support persistent client-server connections. With a persistent connection, the WTE server accepts multiple requests from the client and sends responses over the same TCP/IP connection. Use of persistent connections reduces latency for clients and lowers CPU load on the WTE proxy server, but it requires more resources in terms of server memory and process threads. Previous releases of WTE support persistent connections for GET requests only. This release of WTE adds support for POST and PUT requests to be handled over persistent connections. JavaServer Pages (JSP) and servlet response caching This new function enables WTE to cache JavaServer Pages (JSP) and servlet-generated HTML from a WebSphere Application Server (WAS) that is configured with a WTE plug-in module. This function makes use of the WAS dynamic caching (dynacache) capabilities introduced in WebSphere Application Server V3.5. WTE retrieves JSP-created HTML from the WAS dynamic cache. Dynamic content can then be cached on the edge of the network, freeing the content host from repeatedly retrieving files from the application server to satisfy requests for those files from different clients. Lightweight Directory Access Protocol (LDAP) authorization plug-in This release of WTE includes an LDAP plug-in which enables WTE to authorize clients by using an LDAP server and user-defined policy information. This is done by directing the authorization routine within WTE to retrieve data from an LDAP server rather than from the WTE registry. The authorization routine, which is called first, verifies access to WTE objects granted by user-defined security policy information. The authentication routine, which is called second, uses the LDAP server database as a WTE registry instead of the original WTE registry. A consistent pop-up window prompting for user name and password is provided using either the LDAP server or the WTE registry. Access through WTE can be further refined by using a security policy, which allows system administrators to grant or deny access according to defined IP addresses, host names, or group or organizational units within an LDAP database.

Chapter 1. WebSphere Edge Server for Multiplatforms concepts

13

Internet Caching Protocol (ICP) plug-in The Internet Caching Protocol allows for the sharing of caches in a multiple proxy configuration. The ICP plug-in enables WTE to query ICP compliant caches in search of HTML pages and other cacheable resources. When the WTE server receives an HTTP request, it searches its own cache for the resource. If the resource is not found in the local cache and the ICP plug-in is enabled, WTE wraps the URL within an ICP query packet and delivers this packet to all identified ICP peer caches (including other WTE servers with the enabled ICP plug-in). If no peers respond with hits, the WTE server that received the original request continues to process the request by using other search methods, or simply retrieves the requested URL itself. CyberPatrol filtering plug-in The CyberPatrol filtering plug-in allows Web content filtering based on the SurfControl database of acceptable and unacceptable Web sites. SurfControl maintains a database of Web sites, the content of which can be subject to filtering. The database is assembled by SurfControl staff who decides whether a Web site should be included in any of their filtering categories. Filters can be selected to block access to Web sites that are listed in negative categories, or they can be set to allow access only to Web sites included in the positive categories. WTE allows you to specify additional Web sites to block or to allow.

14

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 2.

What WebSphere Edge Server can do for you

In this chapter, we review several configurations demonstrating how WebSphere Edge Server components can fit into a network configuration to provide the benefits of load balancing and traffic reduction.

Copyright IBM Corp. 2001

15

2.1 Building high performance Web sites

WebSphere Edge Server for Multiplatforms allows you to design several architectures to enhance the performance of your Web site. Figure 2-1 offers an idea of the multiple configurations that can be obtained combining the WebSphere Edge Server components:
WTE

ND Primary

ND Primary

ND Backup

Internet
Client

ND Backup

WTE

Web Servers

Figure 2-1 A sample WebSphere Edge Server environment

The Web Traffic Express (WTE) component minimizes response time and network bandwidth utilization by providing Web content caching. It also ensures reliable content filtering at the proxy server level. Multiple Web Traffic Express servers can be load balanced. The Network Dispatcher (ND) component distributes the load between multiple clustered Web servers and Web Traffic Express servers. As soon as a client request arrives, the load balancing machine uses sophisticated monitoring tools and forwards the request to the least loaded server. High availability is provided by a backup Dispatcher machine, which monitors the state of the primary machine and takes over if it fails. The Web server selected by the Network Dispatcher machine can respond directly to the client without any further involvement of the Network Dispatcher machine. Since there is no need for the server response to go back through the same physical path, a separate high-bandwidth connection can be used.

16

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

IBM used the WebSphere Edge Server technology to create a scalable and reliable system that efficiently handled unprecedented traffic volumes. On February 17th, 1998, at 12:41 (Japan Standard Time), the official Web site of the Olympic Winter Games in Nagano logged 98,226 hits per minute. Less than a week later, a peak load of more than 103,400 hits per minute was supported while still providing normal response time. By using WebSphere Edge Server, you will have an efficient Web site, capable of providing fast responses and able to handle very large amounts of simultaneous requests without any major delay. Furthermore, the high availability features built into WebSphere Edge Server will make the Web services available even if one or more of your server machines should unexpectedly fail. The following figure shows an example of what your Web customers do not see if your Web site uses WebSphere Edge Server:

Figure 2-2 A window that can be prevented

2.2 Who can benefit

WebSphere Edge Server allows you to design and use multiple architectures. The scenarios described in this section provide specific examples of how various ISP implementations can benefit from the use of WebSphere Edge Server.

Chapter 2. What WebSphere Edge Server can do for you

17

2.2.1 Content hosting Internet Service Providers

Content-hosting ISPs can use WebSphere Edge Server to support and distribute the content from their own Web sites more efficiently, and to provide more response- and cost-effective access to other sites (see figure below).

HTTP Server A HTTP Server D HTTP Server B HTTP Server E HTTP Server C

Network Dispatcher Load-Balancing Backup

Network Dispatcher Load-Balancing Primary

Web Client

Internet

Gateway or Firewall

Figure 2-3 Content-hosting Internet Service Providers

Within a content-hosting server farm, the WebSphere Edge Server components can be configured to provide high availability and accessibility as follows: The scalability of Web sites can be achieved by adding multiple HTTP servers and by using efficient load balancing to dispatch requests to the HTTP server with the best capacity to handle the requests. Network Dispatcher, the load balancing component of WebSphere Edge Server, guarantees improved availability and scalability by allowing a farm of Web servers to provide a single Web site image to clients. High availability can be maintained by configuring a hot standby Network Dispatcher component. Proxy caching can be used to provide quicker access to content from other sites, as well as to optimize the backbone network traffic capacity. Both availability and performance may be further enhanced by geographically distributing HTTP servers, together with proxy caching for other Internet content closer to user access points, such as points of presence (POPs) and network access points (NAPs).

18

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Content-hosting service providers or corporate webmasters can therefore benefit from local and wide area load balancing, and proxy caching. The flexible configuration of these components can ensure that requests are directed to the most appropriate local or remote location, and can allow location outages or routine maintenance schedules to be handled without disrupting the customers activity. The WebSphere Edge Server components can be used in conjunction with firewalls and authentication gateways to provide secure access where desired, and the load balancing function of WebSphere Edge Server can also be used to scale these capabilities.

2.2.2 Corporate Web sites and content aggregators

Use of WebSphere Edge Server by corporate Web sites and content aggregators is similar to that of content hosting ISPs.

Internet

Gateway or Firewall

Network Dispatcher Load-Balancing Primary

Network Dispatcher Load-Balancing Backup

Web Client Web Traffic Express Caching Reverse Proxy Web Traffic Express Caching Reverse Proxy

HTTP Server A

HTTP Server B

HTTP Server C

HTTP Server D

Figure 2-4 Corporate Web sites and content aggregators

In many cases, corporate Web sites and content aggregators need to maintain a demilitarized zone (DMZ) to ensure that access to Web content by employees, business partners, or customers does not expose internal computer resources to unauthorized users or hackers. Here, the load balancing Network Dispatcher function can be used to provide the degree of scalability necessary to satisfy users. In addition, Web Traffic Express caching and filtering proxy servers may be deployed on the same machines or on separate systems to filter or optimize access from within the corporation to external Web sites.

Chapter 2. What WebSphere Edge Server can do for you

19

Many corporate Web sites are located at the head office or in regional locations, but branch offices or business partners may need frequent access to the content. Most offices of this type have relatively low line speed (somewhat less than 1.5 Mbps) network access to the regional or corporate sites. Relatively few users can concurrently use all the available bandwidth, and the response times become erratic. At these locations, HTTP servers with general purpose caching can provide more consistent user response time.

HTTP Server A Web Client HTTP Server C HTTP Server B Gateway or Firewall

Web Traffic Express Caching & Filtering Proxy Web Traffic Express Caching & Filtering Proxy
HTTP Server

Network Dispatcher Load-Balancing Backup Network Dispatcher Load-Balancing Primary

Network Dispatcher Load-Balancing Backup Network Dispatcher Load-Balancing Primary

Web Client

Gateway or Firewall

Branch Office

Gateway or Firewall

Internet or Intranet

Corporate Head Office

Figure 2-5 Home office-branch office configuration

Some industries also have small branch offices in addition to larger branch or regional offices. A simple deployment of Web Traffic Express caching and filtering on a single platform, eventually combined with a firewall, is probably sufficient to provide more consistent response time for employees in the small branch offices. In the larger branches or regional offices, it may be desirable to have more than one Web Traffic Express proxy server and to use the load balancing Network Dispatcher functionality to provide better resource management. Two approaches can be used to reduce redundant page storage and to address the effectiveness of caching in these locations: 1. Hierarchical caching

20

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Optimize a primary (initial) cache for higher page hit rate by favoring more files of a smaller size, and a hierarchical cache for higher hit byte rate by favoring fewer files of a larger size. 2. Remote Cache Access (RCA) Configure RCA together with the load balancing function of Network Dispatcher to reduce the index size of each proxy and to increase the scalability of the caching storage.

2.2.3 Backbone Internet Service Providers

Backbone ISPs typically provide co-location and/or peering services for other ISPs in addition to content hosting for large national or international corporations. In many cases, they may provide virtual ISP services for other service providers such as content-hosting ISPs. Backbone ISP customers are increasingly demanding both high availability and differentiated service levels, and backbone ISPs are responding by enhancing their Internet infrastructures. Features demanded from backbone ISPs include: Load balancing for a variety of traffic (for example, mail and FTP in addition to Web traffic) In addition, requests are made for traffic balancing management and high availability for authentication servers and management systems. Proxy caching To minimize the effects of hot potato routing and Web traffic surges, backbone ISPs are typically installing highly scalable caches at major peering points and network interconnections points, such as network access points (NAPs).

Chapter 2. What WebSphere Edge Server can do for you

21

Web Client

Network Dispatcher Load-Balancing Primary

Network Dispatcher Load-Balancing Backup

Gateway or Firewall

Web Traffic Express Caching & Filtering Proxy

Web Traffic Express Caching & Filtering Proxy

Web Traffic Express Caching & Filtering Proxy

Gateway or Firewall

Internet Web Servers

Figure 2-6 Backbone ISPs

International ISPs in particular need caching to reduce the costs associated with transoceanic links. Such installations can make very effective use of the RCA feature of Web Traffic Express, a variation of the traditional caching function that, when used together with load balancing and shared file storage, can dramatically reduce page storage redundancy. Typical configurations for backbone ISPs include load balancing for authentication gateways (such as authentication servers and subscriber management application servers) as well as for WTE caching proxy servers. Because of the amount of traffic and the desire for high availability, such caching servers would likely use the remote cache access (RCA) feature of Web Traffic Express. This feature enables all of the WTE servers in a defined group to share the contents of their caches with one another.

2.2.4 Access Internet Service Providers

Access Internet Service Providers (Access ISPs) need to provide more consistent response time to their customers and to conserve their backbone network link and access charges. Caching at POPs addresses these needs. These configurations are similar to those for backbone ISP solutions (see figure below).

22

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Because access service providers target many of the small- to medium-size business customers, there is also an opportunity for them to create revenue producing services by deploying smaller caching devices on customer premises, but configuring and managing them as an ISP service. For this scenario, the caching would look much as it does for corporate branch offices.
Network Dispatcher Load-Balancing Primary Network Dispatcher Load-Balancing Backup

Web Client

Gateway or Firewall

Web Traffic Express Caching & Filtering Proxy

Web Traffic Express Caching & Filtering Proxy

Web Traffic Express Caching & Filtering Proxy

Gateway or Firewall ISP's Intranet

Web Client

Network Dispatcher Load-Balancing Backup

Network Dispatcher Load-Balancing Primary

Gateway or Firewall

Gateway or Firewall

Web Traffic Express Caching & Filtering Proxy

Web Traffic Express Caching & Filtering Proxy

Web Traffic Express Caching & Filtering Proxy

Internet Web Servers

Gateway or Firewall

Figure 2-7 Access Internet Service Providers

Chapter 2. What WebSphere Edge Server can do for you

23

24

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Part 2

Part

Network Dispatcher
In this part, we describe the Network Dispatcher component of WebSphere Edge Server for Multiplatforms, which is also referred to in many publications as the load balancer.

Copyright IBM Corp. 2001

25

26

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 3.

Network Dispatcher installation

This chapter covers the installation steps for Network Dispatcher in the following environments: AIX: see 3.1, Installing Network Dispatcher on AIX on page 28 Windows NT: see 3.2, Installing Network Dispatcher on Windows NT on page 40 Linux: see 3.3, Installing Network Dispatcher on Linux on page 55 In addition, procedures for uninstallation and starting and stopping services are also covered.

Copyright IBM Corp. 2001

27

3.1 Installing Network Dispatcher on AIX

In this section, we describe the requirements for installing Network Dispatcher on a machine running AIX. We explain the procedure for installation and uninstallation, and how to start using Network Dispatcher.

3.1.1 Requirements
These are the required components for installing and running Network Dispatcher on AIX: Any RS/6000-based machine IBM AIX Version 4.3.3.10 or higher, plus the Java required PTFs IBM AIX Developer Kit, Java 2 Technology Edition, Version 1.3.0 for the Java Runtime Environment. You must download both the Developer Kit installable package and the Runtime Environment installable package prior to running the InstallShield program.1 50 MB of available disk space for installation Note: Additional disk space will be needed for logs Any of the following Network Interface Cards (NICs) supported by the TCP/IP protocol stack to make network connections: 16 Mb Token ring 10 Mb Ethernet 100 Mb Ethernet 1 Gb Ethernet Quad port Ethernet Fiber distributed data interface (FDDI)

Web Traffic Express Version 2.0 or higher if you are using the CBR component for load balancing HTTP traffic Netscape Navigator Version 4.07 (or higher) or Netscape Communicator Version 4.61 (or higher) for viewing online help Note: The installation process for AIX does not prompt you for the path to install the software. It is automatically installed in the directory /usr/lpp/nd, so make sure that there is enough free space on the proper file system before installing the product.

Go to http://www-105.ibm.com/developerworks/tools.nsf/dw/java-devkits-byname?OpenDocument&Count=100 (the IBM developerWorks tools and products page) to download the appropriate Java technology runtime environment.

28

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

3.1.2 Choosing the components

Before beginning the installation, we need to discuss the Network Dispatcher components that you can choose to install. During installation, you must choose one of the three installation methods listed below (refer to Figure 3-3 on page 34). Depending on the installation method you choose, you will have a different set of components to select for installation. The installation methods are: 1. Express: installs only the Dispatcher component, and does not configure it. 2. Guided: installs one of the following components: Remote administration client This is the complete Network Dispatcher graphical user interface (GUI), which, when installed, allows you to remotely manage a Network Dispatcher server. Dispatcher component This is the main component for load balancing and high availability. CBR for IMAP/POP3 This is the Content Based Routing component for IMAP and POP3. CBR for HTTP This is the Content Based Routing component for HTTP (this component needs Web Traffic Express). You can also install those same components using the custom install method (see below), but the Guided install uses a wizard that prompts you for information to create a basic configuration during installation. 3. Custom: when selecting the custom method, you must select which of the following components you want to install: Dispatcher runtime This is the base component for load balancing and high availability. By selecting this option, the next two options are automatically selected too. Dispatcher administration This is the graphical user interface (GUI) which allows you to configure a server running the Dispatcher runtime (locally or remotely). Dispatcher license This option allows you to copy the Dispatcher component license into the server.

Chapter 3. Network Dispatcher installation

29

 Interactive Session Support (ISS) runtime This is the base component for Interactive Session Support. If this option is selected, the next two options are automatically selected also. Interactive Session Support administration This is the graphical user interface (GUI) which allows you to configure a server running the ISS runtime (locally or remotely). Interactive Session Support license This option allows you to copy the ISS component license into the server. Content Based Routing (CBR) runtime This is the base component for Content Based Routing. If this option is selected, the next two options are automatically selected also. Content Based Routing administration This is the graphical user interface (GUI) which allows you to configure a server running the CBR runtime (locally or remotely). Content Based Routing license This option allows you to copy the CBR component license into the server. Documentation This option is used to install the product manuals.

3.1.3 Installation
AIX 4.3.3 is shipped with Java Development Kit and Java runtime V1.1.8. As noted previously (refer to Java 1.3 support on page 12), this version is no longer supported by Network Dispatcher. You must install IBM Java Development Kit (JDK) V1.3.0, which is available for download on the developerWorks site, at the following URL:
http://www.ibm.com/developerworks/java/jdk/index.html

JDK 1.3.0 requires the following filesets to be installed on the machine at the levels listed below: X11.fnt.fontServer.4.3.3.12.bff X11.base.lib.4.3.3.15.bff X11.base.rte.4.3.3.14.bff X11.base.rte.4.3.3.10.bff X11.motif.lib.4.3.3.15.bff

30

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

bos.rte.libc.4.3.3.15.bff devices.isa_sio.baud.rte.4.3.2.1.bff These fixes are also available from the Web site mentioned above. After you uninstall JDK 1.1.8 and install the AIX fixes, you can install JDK 1.3.0. The file we downloaded from the developerWorks site is Java130.rte. You can use SMIT to install JDK. Run smit install_latest, then type in the path where you downloaded the Java130.rte file, click Enter, and on the next screen click Enter again. When software installation is complete, update the PATH variable to include the JDK path. You can edit the /etc/environment file, locate the definition of the PATH variable, and add /usr/java130/bin to it, as shown in Example 3-1.
Example 3-1 Updating the PATH variable PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java130/bin export PATH

Log out and log in again so that your user has the updated PATH. Use the following command to test JDK.
Example 3-2 Testing the Java runtime # java -version java version "1.3.0" Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0) Classic VM (build 1.3.0, J2RE 1.3.0 IBM build ca130-20000622 (JIT enabled: jitc))

Once you have JDK running, you can start the Network Dispatcher installation. Mount the WebSphere Edge Server CD-ROM in the CD-ROM drive (in case it is currently unmounted) and run the setup script to start the installation wizard as follows:
Example 3-3 Starting the installation wizard mkdir /cdrom mount -v cdrfs -p -r /dev/cd0 /cdrom cd /cdrom ./setup

Important: The installation wizard requires an X-Windows server. If you do not have a graphics adapter on your machine, make sure you can use Telnet to access it and export the display to another machine which is running an X-Windows server.

Chapter 3. Network Dispatcher installation

31

Once the installation wizard is started, the first thing you need to do is to select the language of the software. We selected English, as shown in Figure 3-1.

Figure 3-1 Selecting the language

32

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Next, you need to select which component of WebSphere Edge Server you are installing. Select Network Dispatcher, and click Next (see Figure 3-2).

Figure 3-2 Selecting the component to be installed

Chapter 3. Network Dispatcher installation

33

On the next window, you need to select the installation method; we selected the Custom method, as shown in Figure 3-3. For more details about the installation methods refer to 3.1.2, Choosing the components on page 29.

Figure 3-3 Selecting the installation method

34

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In the next window (shown in Figure 3-4), select all the components to be installed.

Figure 3-4 Selecting the software components to install

Chapter 3. Network Dispatcher installation

35

The next window shows a summary of the selected options to install. If the listing is correct, click Next to proceed and start the installation.

Figure 3-5 Summary of the selected options

36

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

After the software is copied to the machine, you are presented with a summary of the installation, as shown in Figure 3-6.

Figure 3-6 Installation summary

Chapter 3. Network Dispatcher installation

37

The last window will display a reference to the readme file, which is shipped with the installation media. We recommend that you read this file carefully, since it contains important, last-minute changes and information about the product.

Figure 3-7 Reference to the readme file

3.1.4 Starting the software

Before using the Network Dispatcher GUI, you need to start the Network Dispatcher server. You can do it by running the following command:
# ndserver

If you want to stop the server, run the following command:

# ndserver stop

Use the ndadmin command to start the Network Dispatcher GUI:

# ndadmin

38

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The Network Dispatcher GUI is shown in Figure 3-8.

Figure 3-8 Network Dispatcher GUI

If you want to begin configuring Network Dispatcher, refer to Chapter 4, Network Dispatcher basic configuration on page 69 for a basic scenario configuration.

3.1.5 Uninstallation
The recommended procedure to uninstall the software is to use the same setup script you used to install it. When uninstalling, you need to use the -nduninstall option, as shown below:

Chapter 3. Network Dispatcher installation

39

Example 3-4 Starting the uninstallation # mkdir /cdrom # mount -v cdrfs -p -r /dev/cd0 /cdrom # cd /cdrom # ./setup -nduninstall Running ND Uninstall... Details on the uninstall are located in the log file: /usr/lpp/nd/uninstall.log ND Uninstall complete.

All the steps performed in the uninstallation processes are recorded on the log file /usr/lpp/nd/uninstall.log. The uninstall process does not completely remove the software. If you want to completely remove it, you need to delete the installation directory (in AIX it is /usr/lpp/nd). Do not remove this directory if you plan to install Network Dispatcher on this machine again, since it keeps the configuration files even after you uninstall the software.

3.2 Installing Network Dispatcher on Windows NT

In this section, we cover the steps to install Network Dispatcher on Windows NT, we discuss some post-installation checks you need to perform and the steps to uninstall the software.

3.2.1 Requirements
These are the components required to install and run Network Dispatcher on Windows NT or Windows 2000: Any Intel x86 PC supported by Microsoft Windows 2000 or Microsoft Windows NT Windows 2000 Professional, Server or Advanced Server; or Windows NT Workstation or Server, Version 4.0 with Service Pack 5 or higher Note: Windows Server is required for machines running the NameServer Observer mode of ISS. IBM Cross Platform Technologies for Windows V2.0 (Java SDK 1.3). You must download both the Developer Kit installable package and the Java Runtime Environment installable package prior to running the InstallShield program.2 50 MB of available disk space for installation
2

Go to http://www.ibm.com/developerworks/java/jdk/index.html to download the appropriate Java SDK.

40

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Note: Additional disk space will be needed for logs. Any of the following Network Interface Cards (NICs) supported by the TCP/IP protocol stack to make network connections: 16 Mb Token ring 10 Mb Ethernet 100 Mb Ethernet 1 Gb Ethernet Quad port Ethernet

Web Traffic Express Version 2.0 or higher if you are using the CBR component for load balancing HTTP traffic Netscape Navigator Version 4.07 (or higher), Netscape Communicator Version 4.61 (or higher) or Internet Explorer 4.0 (or higher) for viewing online help

3.2.2 Installation
Before installing Network Dispatcher, you must install the Java Development Kit (JDK) V1.3.0, which is available for download on the developerWorks site, at the following URL:
http://www.ibm.com/developerworks/java/jdk/index.html

The downloaded package contains an install.exe executable file. Run this file to start the installation, as shown in Figure 3-9. Make sure to use the path in which you downloaded the files from the Internet.

Figure 3-9 Installing JDK

After installing JDK, make sure that the Path environment variable was updated to include the JDK path. To do so, click Start->Settings->Control Panel->System, select the folder Environment and locate the variable path. Make sure that the path to the bin directory of JDK was added to the value of the variable path (in our example, it is C:\Program Files\Ibm\Java13\jre\bin), as shown in Figure 3-10.

Chapter 3. Network Dispatcher installation

41

Figure 3-10 Path after installing JDK

Once JDK is installed, you can begin the installation of Network Dispatcher. Click Start->Run... and select the setup.exe file from the WebSphere Edge Server installation media, as seen below.

Figure 3-11 Starting the installation

42

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Once the installation wizard is started, the first thing you must do is to select the language. We selected English, as shown in Figure 3-12.

Figure 3-12 Selecting the language

Chapter 3. Network Dispatcher installation

43

Next, you must select which component of WebSphere Edge Server you are installing. Select Network Dispatcher, and click Next.

Figure 3-13 Selecting the component to be installed

44

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In the next step, you must select a path where Network Dispatcher should be installed. We recommend that you use the default path.

Figure 3-14 Selecting the path where the product will be installed

Chapter 3. Network Dispatcher installation

45

In the next window, you select the installation method; we selected the custom method, as shown in Figure 3-15. For more details on the installation methods, refer to 3.1.2, Choosing the components on page 29.

Figure 3-15 Choosing the installation method

46

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In the next window (shown in Figure 3-16), select all the components to be installed.

Figure 3-16 Selecting the software components to install

Chapter 3. Network Dispatcher installation

47

The next window shows a summary of the selected options to install. If the listing is correct, click Next to proceed and start the installation.

Figure 3-17 Summary of the selected options

48

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

After installing the software, you are presented with a summary of the installation, as shown in Figure 3-18.

Figure 3-18 Summary of the installation

Chapter 3. Network Dispatcher installation

49

The last window displays a reference to the readme file, which is shipped with the installation media. We recommend that you read this file carefully since it contains important, last-minute changes and information about the product.

Figure 3-19 Reference to the readme file

Note that you need to reboot the machine after the installation process is completed.

50

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

3.2.3 Starting the software

Once the installation is finished, we recommend that you check a few items in the environment before beginning to use the graphical user interface (GUI) to configure Network Dispatcher. After rebooting the machine, make sure that the Network Dispatcher service is started. Click Start->Settings->Control Panel->Services, and locate the IBM Network Dispatcher service. The status of this service must be Started, as shown in Figure 3-20.

Figure 3-20 IBM Network Dispatcher service

Chapter 3. Network Dispatcher installation

51

The second thing you need to check is the IBMNDPATH environment variable. Click Start->Settings->Control Panel->System, choose the Environment tab and locate the variable IBMNDPATH. Make sure that its value corresponds to the path where the product was installed (see Figure 3-21).

Figure 3-21 The IBMNDPATH variable

52

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Next, you can use the GUI to configure the software. Click Start->Programs->IBM WebSphere->Edge Server->IBM Network Dispatcher to open the GUI. The Network Dispatcher GUI main window is shown in Figure 3-22.

Figure 3-22 Network Dispatcher GUI main window

If you are ready to begin configuring Network Dispatcher, refer to Chapter 4, Network Dispatcher basic configuration on page 69 for a basic scenario configuration.

Chapter 3. Network Dispatcher installation

53

3.2.4 Uninstallation
If you want to uninstall Network Dispatcher, click Start->Settings->Control Panel->Add/Remove Programs and select IBM Network Dispatcher from the Install/Uninstall tab, as shown in Figure 3-23. Click the Add/Remove button, and the software uninstallation process will be started.

Figure 3-23 Uninstalling the software

The system shows the progress of the uninstallation, and when it finishes, it prompts you for a reboot. The uninstall process does not completely remove the software. If you want to completely remove the software, you must delete the installation directory (in our example, C:\Program Files\Ibm\nd) and the following registry entries: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Enum\Root\LEGACY_IBM NETDISP HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\Enum\Root\LEGACY_IBM NETDISP HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\Root\LEGACY_I BMNETDISP

54

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The files kept in the installation directory are the log files and the configuration files for each component. Do not remove either the directory or the registry entries if you plan to reinstall Network Dispatcher again on this machine.

3.3 Installing Network Dispatcher on Linux

In this section, we describe the requirements for Network Dispatcher to be installed on a machine running Linux. We explain the steps involved in installing and uninstalling Network Dispatcher and how to start the software.

3.3.1 Requirements
These are the required components to install and run Network Dispatcher on Linux: Any IBM-compatible PC or PS/2 (including SMP models) supported by one of the following Linux distributions: Red Hat Linux Version 6.1 (Linux kernel Version 2.2.12-20) Red Hat Linux Version 6.2 (Linux kernel Version 2.2.16-3) 50 MB of available disk space for installation Note: Additional disk space will be needed for logs The following Network Interface Cards (NICs) supported by the TCP/IP protocol stack to make network connections: 10 Mb Ethernet 100 Mb Ethernet 1 Gb Ethernet Quad port Ethernet

A version of the Korn Shell (ksh), which must be installed IBM Runtime Environment for Linux, Java 2 Technology Edition, V1.3.3 Web Traffic Express Version 2.0 or higher if you are using the Content Based Routing (CBR) component for load balancing HTTP traffic Note: When using the CBR component, you must use Network Dispatcher V3.6.0.1 along with IBMRuntime Environment for Linux, Java 2 Technology Edition V1.3-1.0.

Go to http://www.ibm.com/developerworks/java/jdk/index.html to download the appropriate Linux Java technology runtime environment.

Chapter 3. Network Dispatcher installation

55

Netscape Navigator Version 4.07 (or higher) or Netscape Communicator Version 4.61 (or higher) for viewing online help Note: The installation process for Linux does not prompt you for the path to install the software. It is automatically installed in the directory /opt/nd, so make sure that there is enough free space on the proper file system before installing the product.

3.3.2 Choosing the components

Before beginning the installation, we need to discuss the Network Dispatcher components which you can choose to install. During the installation, you must choose one of the three installation methods listed below (refer to Figure 3-26 on page 61). Depending on the method you choose, you will have a different set of components to select for installation. The installation methods are: 1. Express: installs only the Dispatcher component, and does not configure it. 2. Guided: installs one of the following components: Remote administration client This is the complete Network Dispatcher graphical user interface (GUI), which, once installed, allows you to remotely manage a Network Dispatcher server. Dispatcher component This is the main component for load balancing and high availability. CBR for IMAP/POP3 This is the Content Based Routing component for IMAP and POP3. CBR for HTTP This is the Content Based Routing component for HTTP (this component needs Web Traffic Express). You can also install those same components using the custom install, but the guided install uses a wizard that prompts you for information to create a basic configuration.

56

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

3. Custom: if you select the custom method, you must select which of the following components you want to install: Dispatcher runtime This is the base component for load balancing and high availability. If this option is selected, the next two options are automatically selected also. Dispatcher administration This is the graphical user interface (GUI) which allows you to configure a server running the Dispatcher runtime (locally or remotely). Dispatcher license This option allows you to copy the Dispatcher component license into the server. Content Based Routing runtime This is the base component for Content Based Routing (CBR). If this option is selected, the next two options are automatically selected also. Content Based Routing administration This is the graphical user interface (GUI) which allows you to configure a server running the CBR runtime (locally or remotely). Content Based Routing license This option allows you to copy the CBR component license into the server. System Monitor Agent This is the component for System Monitor Agent (SMA). For more information about SMA, refer to IBM Network Dispatcher Users Guide Version 3.0 for Multiplatforms, GC31-8496-05. Documentation This option is used to install the product manuals.

3.3.3 Installation
Before installing Network Dispatcher, you must install the Java Development Kit (JDK) or Java Runtime Environment V1.3.0, which are both available for download on the developerWorks site, at the following URL:
http://www.ibm.com/developerworks/java/jdk/index.html

From the site above, we downloaded the following files: Java Development Kit: IBMJava2-SDK-1.3-6.0.i386.rpm Java Runtime: IBMJava2-JRE-1.3-6.0.i386.rpm

Chapter 3. Network Dispatcher installation

57

You can choose which one you prefer to install (do not install both!). We installed JDK, using the following command:
# rpm -ivh IBMJava2-SDK-1.3-6.0.i386.rpm

This package is installed in the directory /opt/IBMJava2-13. If you install Java Runtime, the install path is the same, but the binary path is /opt/IBMJava2-13/jre/bin. You can test if the Java installation was successful by running the following command (see example below):
Example 3-5 Testing Java Runtime # /opt/IBMJava2-13/bin/java -version java version "1.3.0" Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.0) Classic VM (build 1.3.0, J2RE 1.3.0 IBM build cx130-20010207 (JIT enabled: jitc)

Before proceeding to the Network Dispatcher installation, you need to set the environment variable JAVA_HOME to the path where JDK was installed and update the path to include the JDK bin directory, as shown in Example 3-6.
Example 3-6 Updating the environment variables export JAVA_HOME=/opt/IBMJava2-13 export PATH=$JAVA_HOME/bin:$PATH

We recommend that you add the commands shown in Figure 3-6 to the /etc/profile file to make sure that those environment variables will be set every time you log in. The next step is to install Network Dispatcher. First, you must mount the CD-ROM drive (in case it is unmounted) and run the setup script, as shown in Example 3-7.
Example 3-7 Starting the installation mount /cdrom cd /mnt/cdrom/ ./setup

Important: The installation wizard requires a X-Windows server. If you do not have a graphics adapter on your machine, make sure you can use Telnet to access it and export the display to another machine which is running a X-Windows server.

58

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Once the installation wizard is started, the first thing you need to do is to select the language of the software. We selected English, as shown in Figure 3-24.

Figure 3-24 Selecting the language

Chapter 3. Network Dispatcher installation

59

Next, you must select which component of WebSphere Edge Server you are installing. Select Network Dispatcher, and click Next (see Figure 3-25).

Figure 3-25 Selecting the component to be installed

60

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In the next window, you need to select the installation method; we selected the custom method, as shown in Figure 3-26. For more details on the installation methods, refer to 3.3.2, Choosing the components on page 56.

Figure 3-26 Selecting the installation method

Chapter 3. Network Dispatcher installation

61

In the next window (shown in Figure 3-27), select all the components to be installed.

Figure 3-27 Selecting the software components to install

62

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The next window shows a summary of the selected options to install. If the listing is correct, click Next to proceed and start the installation.

Figure 3-28 Summary of the selected options

Chapter 3. Network Dispatcher installation

63

After installing the software, you are presented with a summary of the installation, as shown in Figure 3-29.

Figure 3-29 Summary of the installation

64

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The last window will display a reference to the readme file, which is shipped with the installation media. We recommend you read this file carefully since it contains important, last-minute changes and information about the product.

Figure 3-30 Reference to the readme file

3.3.4 Starting the software

Before using the Network Dispatcher GUI, you must start the Network Dispatcher server by running the following command:
# ndserver

If you want to stop it, run the following command:

# ndserver stop

Use the ndadmin command to start the Network Dispatcher GUI:

# ndadmin

Chapter 3. Network Dispatcher installation

65

The Network Dispatcher GUI is shown in Figure 3-31.

Figure 3-31 Network Dispatcher GUI

3.3.5 Uninstallation
The recommended procedure to uninstall the software is to use the same setup script you used to install it. When uninstalling, you need to use the -nduninstall option, as shown below:
Example 3-8 Starting the uninstallation # mount /cdrom # cd /mnt/cdrom/ # ./setup -nduninstall Running ND Uninstall... Details on the uninstall are located in the log file: /opt/nd/uninstall.log ND Uninstall complete.

All the steps performed in the uninstallation processes are recorded in the log file /opt/nd/uninstall.log.

66

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The uninstall process does not completely remove the software. If you want to completely remove the software, you must delete the installation directory (in Linux it is /opt/nd). Do not remove this directory if you plan to install Network Dispatcher on this machine again, since it keeps the configuration files even after you uninstall the software.

Chapter 3. Network Dispatcher installation

67

68

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 4.

Network Dispatcher basic configuration

In this chapter, we explain the implementation of basic scenarios using Network Dispatcher. First, we describe a basic scenario using Network Dispatcher to load balance two WebSphere Application Servers. Then, we improve this scenario by adding one more Network Dispatcher machine for high availability. Finally, we add Interactive Session Support (ISS) to the scenario to improve the load balancing of the Web servers.

Copyright IBM Corp. 2001

69

4.1 Load balancing

In this section, we will describe a simple load balancing scenario. Later on, we will add some extra functions to this same scenario.

4.1.1 Scenario description

We created a simple scenario, involving one Network Dispatcher load balancing the requests to two Web servers. All requests sent to the cluster address go to the Network Dispatcher machine, which will forward these requests to the Web servers. In this scenario, we installed Network Dispatcher on a Windows NT machine, and WebSphere Application Server on two other machines, one running Windows NT and the other running AIX, as shown in Figure 4-1.

9.24.105.39 m23kk904

9.24.105.59 rs60008

Web servers

Cluster: 9.24.105.87 wses1

Network Dispatcher
9.24.105.82 m238p4yl

Figure 4-1 Basic load balancing scenario

70

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The following table shows more information on the machines we used in this scenario.
Table 4-1 Machines used in the load balancing scenario Host Name m238p4yl IP Address 9.24.105.82 Operating System and Software Windows NT 4.0 Service Pack 5 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Service Network Dispatcher Web server

rs60008

9.24.105.59

m23kk904

9.24.105.39

Web server

These machines were all connected to the same Ethernet network.

4.1.2 Configuration
First, we explain how to use the GUI. Click Start->Programs->IBM Network Dispatcher. If you are using Network Dispatcher on a UNIX machine (AIX, Linux or Solaris), run the ndadmin command to start it:
# ndadmin

During the installation, we chose to install all the administration components (Dispatcher, Interactive Session Support and Content Based Routing), so the GUI will show all of them under the group Network Dispatcher (for more information on installing those components, refer to Chapter 3, Network Dispatcher installation on page 27). If you click any item under Network Dispatcher, it will insert a menu option on the menu bar. This new option will be placed beside the File menu option (see Figure 4-2). You can also access this context menu by selecting the item (in this case, Dispatcher) and right-clicking it.

Chapter 4. Network Dispatcher basic configuration

71

Figure 4-2 Network Dispatcher GUI

Now we can make a connection to our local Dispatcher server. If you are running the GUI in a separate machine, refer to 5.1, Remote configuration on page 156. We will describe the steps to perform this basic configuration using both the wizard and the GUI. At the end of this section, we will show the configuration file that was produced. If you prefer to use the command line interface, you can use the commands generated by this configuration, shown in Basic configuration using the command line interface on page 89.

Basic configuration using the configuration wizard

Open the GUI as explained above. Click Dispatcher (under Network Dispatcher), and the context menu will appear on the menu bar. Click Dispatcher on the menu bar and it will show you two menu items: Connect to Host... and Start Wizard, as shown in Figure 4-3.

72

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-3 Dispatcher menu

Click Start Configuration Wizard, and the Welcome window will be displayed. Click Next, and a window explaining the purpose of the configuration wizard will be displayed. Click Next again.

Chapter 4. Network Dispatcher basic configuration

73

The next window will begin prompting for information on your environment. First, you must type in the host name (or IP address) of the machine where the Dispatcher is running (see Figure 4-4).

Figure 4-4 Entering the host name of the Dispatcher

Note: We strongly recommend that you use a DNS server. Make sure it is resolving forward and reverse lookups correctly. In this chapters examples, we often use the host name and domain, which were previously added to the DNS data files. In this scenario, the Dispatcher machine we are using is m238p4yl.itso.ral.ibm.com (IP address 9.24.105.82). Click Update Configuration & Continue to proceed.

74

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Next, you must add the host name which resolves to the IP address of your cluster (see Figure 4-5).

Figure 4-5 Adding the host name of the cluster

We added the host name and domain of our cluster, which is wses1.itso.ral.ibm.com (IP address 9.24.105.87). Click Update Configuration & Continue; this will display a window confirming the creation of the new cluster.

Chapter 4. Network Dispatcher basic configuration

75

Next, you need to add a port to the cluster. Since we are creating a cluster to load balance HTTP traffic, we selected port 80, as shown in Figure 4-6.

Figure 4-6 Selecting a port

Again, we received a confirmation that the port was successfully added to the cluster.

76

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In the next window, add all server host names that are part of this cluster, that is, all Web servers that would respond to HTTP requests sent to the host name of the cluster (see Figure 4-7).

Figure 4-7 Creating a list of servers for a cluster

Chapter 4. Network Dispatcher basic configuration

77

To add the servers, click the Add a server button, and type in the host name and domain of the first server. We added the host name m23kk904.itso.ral.ibm.com.

Figure 4-8 Adding a server to the cluster

Click Next, and the window shown in Figure 4-7 will be updated with this new server. Click Add a server again to add the next server, and continue until you have finished adding all your servers to this list.

78

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

When we finished adding our two servers, both were listed as shown in Figure 4-9.

Figure 4-9 Complete list of servers for the new cluster

Once the list is complete, click Update Configuration & Continue to proceed with the configuration.

Chapter 4. Network Dispatcher basic configuration

79

Next, we chose to start an advisor for the port we added: port 80 (see Figure 4-10).

Figure 4-10 Starting an advisor

A window is displayed to confirm that the advisor was started. The next window has information about setting up the servers that are part of the cluster to listen to requests sent to them using the cluster IP address. The servers need to recognize this cluster IP address as one of their valid local IP addresses in order to process the packet; otherwise, they will just forward the packet back to the Dispatcher machine (we cover this configuration in Configuration of the back-end servers on page 95). Once you have read the information about the operating system you are using, click Exit to finish the configuration.

80

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

A window will be displayed confirming the end of the configuration for the cluster. If you want to add another cluster, click New Cluster (see Figure 4-11). If you do not have any more clusters to add, click Exit.

Figure 4-11 Finishing the cluster configuration

Go to Configuring the Dispatcher and the back-end servers on page 90 for the next steps in the configuration of our basic scenario.

Basic configuration using the GUI

Open the window as described in 4.1.2, Configuration on page 71, expand the group Network Dispatcher and click Dispatcher. On the menu bar, click Dispatcher->Connect to Host..., as shown in Figure 4-3 on page 73. On the pop-up window, type the host name of the Dispatcher server as shown in Figure 4-12 (if you are using the GUI locally, just confirm the local address by clicking OK).

Chapter 4. Network Dispatcher basic configuration

81

Figure 4-12 Connecting to the Dispatcher server

Note: If you receive an error while trying to connect to the Network Dispatcher server, make sure that ndserver is started. After connecting to the Network Dispatcher server, one item will be added in the left pane of the window, which will be called Host:<hostname.<domain> as shown on Figure 4-13.

Figure 4-13 Dispatcher host name

82

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Click the newly added item Host in the window, and a new Host item will be added to the menu bar. Select Host on the menu bar, then choose Start Executor, as shown in Figure 4-14.

Figure 4-14 Context menu for the Host item

Chapter 4. Network Dispatcher basic configuration

83

One more item will be added to your GUI, named Executor <Dispatchers IP address>, as shown in Figure 4-15.

Figure 4-15 GUI showing the Executor status

Once the Executor is started, the next step is to add the cluster. In our scenario, our cluster address is 9.24.105.87 (see Figure 4-1 on page 70). To add this cluster, click Executor on the left pane of the window, which will add Executor to the menu bar. Click Executor on the menu bar, and click Add Cluster.... A pop-up window will be displayed, prompting you for information about this new cluster, as shown in Figure 4-16.

84

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-16 Adding a cluster

Fill in the IP address of your cluster in the Cluster address field, and the IP address of your Network Dispatcher server in the Primary host for the cluster field (we will discuss primary and backup hosts in 4.2, High availability on page 104). Check the option Configure this cluster?, which will enable the Dispatcher to receive packets sent to the cluster address (this works as an IP alias of the local interface). When you select this checkbox, another pop-up window is displayed, as shown in Figure 4-17.

Figure 4-17 Configuring the cluster address

You must add the network interface that is configured on the same network as the cluster address. Also add the network mask.

Chapter 4. Network Dispatcher basic configuration

85

If you are using Windows NT, you must specify the interface by using en for Ethernet and tr for Token Ring, plus the sequence number of the adapter, beginning with 0. For example, click Start->Settings->Control Panel->Network, and select the Adapters tab. It will show you the list of network adapters on your machine, and their sequence number. In Figure 4-18 there is only one adapter, and the sequence number is 1. This will be specified as en0. If there were a second adapter, it would be en1, and so forth. The same logic applies to Token Ring.

Figure 4-18 List of network adapters

86

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

If you are using AIX or Linux, use the ifconfig command to list all interfaces on the machine, as shown below:
Example 4-1 Output of ifconfig command # ifconfig -a lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT, PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255

Use the interface identification that the ifconfig command output shows (for example en0, en1). Once the cluster is added, you need to add the port that our back-end servers are listening to. Click Cluster on the left pane of the window, and on the context menu click Cluster->Add port. In our scenario, we used port 80, as shown in Figure 4-19.

Figure 4-19 Adding the port

The next step is to add the servers that are listening to port 80 and will receive the HTTP requests for that cluster. Click Port:80 on the left pane of the window, and on the context menu click Port:80->Add server. A pop-up window will be displayed prompting for the IP address of the first server. We typed in the address of our first server, which was 9.24.105.39, as shown in Figure 4-20.

Chapter 4. Network Dispatcher basic configuration

87

Figure 4-20 Adding a new server

Make sure that the Network router address checkbox is not checked (we will discuss this option in more detail in 5.5, Wide area Network Dispatcher support on page 191). Click OK to add this server. For our scenario, we repeated the same process to add the second server, the IP address of which was 9.24.105.59. Once you have finished including the servers, you need to start the Manager. Click Host on the left pane of the window (it is right below Dispatcher, at the top of the tree). In the context menu, click Host->Start Manager. A pop-up window will be displayed, as shown in Figure 4-21. Click OK to confirm the default values.

Figure 4-21 Starting the manager

Right after the Manager is started, another pop-up window will automatically be displayed, allowing you to start an advisor. Since we are working with HTTP on port 80 in our back-end servers, we selected the advisor Http (in the Advisor name field) and port 80 (in the Port number field), as shown in Figure 4-22.

88

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-22 Adding an advisor

You do not need to add the file name of the log file. The default file name is Http_80.log. Go to Configuring the Dispatcher and the back-end servers on page 90 for the next steps in the configuration of our basic scenario.

Basic configuration using the command line interface

You can perform the same configuration we did in the previous sections using the command line interface. This works for all platforms. Example 4-2 shows the sequence of commands that will create a cluster, add the port and the servers, and start the advisor for port 80.
Example 4-2 Configuration using the command line interface ndcontrol executor start ndcontrol executor set nfa 9.24.105.82 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.82 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.104.237 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol cluster configure 9.24.105.87 en0 255.255.255.0 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log

Chapter 4. Network Dispatcher basic configuration

89

In the following section, we explain how to save your configuration so it will be run automatically after a system reboot. Important: The non-forwarding address (NFA) used in the second command in Example 4-2 should be the address that the machine is using to listen to network requests (the local address of the machine, not the cluster IP address).

Configuring the Dispatcher and the back-end servers

Once the cluster is configured (whether using the configuration wizard, the GUI, or the command line interface), you still need to go through a few more steps:

Checking the IP aliases on the Dispatcher

During the configuration, you are supposed to add an IP alias for the cluster IP address to the network interface of the Network Dispatcher machine. During the configuration, we showed you how to add the IP alias through the Dispatcher (see Figure 4-16 and Figure 4-17 on page 85). You may want to check the status of the IP alias in your configuration. You can do that using the GUI or command line. If you want to use the GUI, click Cluster:9.24.105.87, then click the Current Statistics tab in the right pane of the window. Locate the Current alias on NIC field. If it shows configured, then the IP alias was added; if it shows unconfigured, then the IP alias was not added or it was removed. See Figure 4-23.

90

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-23 Checking the status of the cluster alias

You can also use the command line to check the IP aliases. If you are using a UNIX system, use the ifconfig command, as shown in Example 4-3. When using the ifconfig command, check the proper command syntax for your operating system.
Example 4-3 Checking for IP aliases in AIX # ifconfig -a lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT > inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT, PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

Chapter 4. Network Dispatcher basic configuration

91

If you are using a Windows system, use the ndconfig command, as shown in Example 4-4. Note that the ndconfig command does not show all IP addresses configured on the machine; it only shows the ones that were configured by the Dispatcher on a specified network interface.
Example 4-4 Checking for IP aliases in Windows NT C:\>ndconfig en0 en0: flags=00000063<UP,BROADCAST,NOTRAILERS,RUNNING> inet 9.24.105.87 netmask 0xFFFFFF00 broadcast 9.24.105.255

Patching the Linux kernel

A requirement of using Linux as a back-end server (and in collocation scenarios) is that the Linux kernel must not ARP on the loopback device. There is a patch available at the developerWorks Web site mentioned later in this section, but you still must identify whether or not you need to install this patch on your Linux system. Log in to your Linux system, then check if the following file exists:
/proc/sys/net/ipv4/conf/lo/hidden

If it exists, you do not need to install the patch, but you do need to run the following commands:
echo 1 > /proc/sys/net/ipv4/conf/lo/hidden echo 1 > /proc/sys/net/ipv4/conf/all/hidden

These commands will prevent the machine from ARPing on the loopback device. This command must be run after each reboot. You can add it to /etc/rc.d/rc.local so that it will be automatically run at boot time. If the file does not exist, you need to install the patch. You can get it from the following URL:
http://www.ibm.com/developerworks/linux/

Click the following links on the page: Projects and Patches->Patches->IBM Network Dispatcher->Network Dispatcher/patch arp 2.2.12-13->download. For instructions on installing this patch, refer to IBM Network Dispatcher Users Guide Version 3.0 for Multiplatforms, GC31-8496-05, and refer to the product readme file as well. After you install the patch, you also need to run the following command:
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_invisible

92

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Note that the path is the same as the built-in version on later kernels, but the file name is different. You need to run this command after each reboot, as we mentioned before.

Setting the manager proportions

The weight of each server is calculated by the Manager using four values: 1. number of active connections 2. number of new connections 3. port specific (provided by the advisors, which inform the load on each server and which servers are up) 4. system metrics (provided by ISS) You can change the proportions used by the Manager to calculate the weight. The default values are 49 (active connections), 49 (new connections), 2 (port specific) and 0 (system metrics). If you configure the ISS (refer to 4.3, Using Interactive Session Support on page 127 for more information), the Manager automatically sets the proportions to 48, 48, 2, and 2. To change the default proportions, you can use the GUI. Click Manager in the left pane of the window, and in the right pane click the Configuration Settings tab, as shown in Figure 4-24.

Chapter 4. Network Dispatcher basic configuration

93

Figure 4-24 Setting the Manager proportions

On this tab, the first fields are the proportion fields. If you want to set 40% for active connections, 40% for new connections, 20% for the advisor and 0% for ISS (we are not using these values in this scenario), type in the values as shown in Figure 4-24. Click Update Configuration at the bottom of the window to activate the changes. If you are using the command line interface, you can run the following command to apply these proportions, instead of using the GUI:
# ndcontrol manager proportions 40 40 20 0

We recommend that you use the default values of 49, 49, 2, and 0 if you are using any advisors.

94

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Saving the configuration

The first thing you need to do after the configuration is ready is to save it. Click Host:m238p4yl.itso.ral.ibm.com (locate the corresponding item on the left pane of the GUI window). In the context menu, click Save configuration file as... and add the file name. We used default.cfg. Important: If you choose to save your configuration in a file called default.cfg, Network Dispatcher will automatically read it every time ndserver is started.

Starting Network Dispatcher after a reboot

If you are running Network Dispatcher in Windows NT, the service IBM Network Dispatcher is configured to be automatically started after each reboot, so you can proceed to Configuration of the back-end servers on page 95. If you are running AIX, Linux or Solaris, you must configure the system to run the ndserver command after each reboot. In our AIX environment, we enabled the automatic startup of ndserver. We added it to the inittab by running the following command:
# mkitab nd:2:wait:/usr/bin/ndserver > /dev/console 2>&1

Configuration of the back-end servers

Although the Network Dispatcher server is ready to begin load balancing the traffic, there is still one more configuration step to perform: preparing the back-end servers to accept packets sent to the cluster IP address. We do this by adding an IP alias to the loopback interface of these servers. This alias is the IP address of the cluster. Important: The cluster address must be added to the back-end servers as a non-advertising address (the server must not respond to ARP requests for that address). That is why we use it as an alias to the loopback interface.

Chapter 4. Network Dispatcher basic configuration

95

We describe below how we added this alias to our Windows NT and AIX machines: Windows NT First, you must install the MS Loopback Adapter (if you have not already done so). Click Start->Settings->Control Panel->Network. In the Network window, select the Adapters tab, as shown in Figure 4-25.

Figure 4-25 List of adapters

96

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Click the Add... button, and select the MS Loopback Adapter, as shown in Figure 4-26.

Figure 4-26 Adding the MS Loopback Adapter

Click OK; another pop-up window will be displayed, prompting for the frame type you are using. The options are: 802.3 (Ethernet) 802.5 (Token Ring) FDDI

Figure 4-27 Frame type

Click OK; on the next pop-up window, you need to type in the path to the Windows NT CD-ROM.

Chapter 4. Network Dispatcher basic configuration

97

Once the system finishes copying the files, the Network window will display the newly added MS Loopback Adapter. The process is not yet completed: click Close, and the system will install the drivers it needs. After that, it will prompt you to add the address you want to put in the loopback adapter. Type in the IP address of the cluster, as shown in Figure 4-28.

Figure 4-28 Applying an IP address to the loopback adapter

Click OK. You will need to reboot the machine.

98

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

After rebooting, check the routing table. Compare it to the one in Example 4-5.
Example 4-5 Routing table after adding the loopback adapter C:\>route print =========================================================================== Interface List 0x1 ........................... MS TCP Loopback interface 0x2 ...00 06 29 b0 94 63 ...... Intel 8255x-based Integrated Fast Ethernet 0x3 ...20 4c 4f 4f 50 20 ...... MS LoopBack Driver =========================================================================== =========================================================================== Active Routes: Network Destination Netmask Gateway Interface Metric 0.0.0.0 0.0.0.0 9.24.105.1 9.24.105.39 1 9.24.105.0 255.255.255.0 9.24.105.39 9.24.105.39 1 9.24.105.0 255.255.255.0 9.24.105.87 9.24.105.87 1 9.24.105.82 255.255.255.255 127.0.0.1 127.0.0.1 1 9.24.105.87 255.255.255.255 127.0.0.1 127.0.0.1 1 9.255.255.255 255.255.255.255 9.24.105.39 9.24.105.39 1 127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1 224.0.0.0 224.0.0.0 9.24.105.39 9.24.105.39 1 224.0.0.0 224.0.0.0 9.24.105.87 9.24.105.87 1 255.255.255.255 255.255.255.255 9.24.105.39 9.24.105.39 1 ===========================================================================

Note that after the loopback adapter was added, the system also added one more route to the routing table. Now, there are two routes to the same destination (the local network 9.24.105.0) using two different gateways: first, the Ethernet adapter IP address (9.24.105.39) and second, the cluster IP address that was added to the loopback (9.24.105.87). The second route is incorrect. Since the first one is correct, the second one should not cause your system any trouble, but we recommend that you remove the second route anyway. You can use the following command in a command prompt window:
C:\> route delete 9.24.105.0 9.24.105.87

This command must also be run after a reboot. We created a batch file, C:\routedel.bat, and added the following lines to it:
@echo off route delete 9.24.105.0 9.24.105.87 exit

Chapter 4. Network Dispatcher basic configuration

99

Then we added a shortcut between this batch file and the Startup program group, as shown in Figure 4-29.

Figure 4-29 Shortcut to the batch file

This batch file will be run after a reboot and it will delete that second route. If you need to add more aliases to the loopback, add the route delete for each alias to this same batch file. AIX The only thing you need to do in AIX in order to add the IP alias is to run the ifconfig command, as follows:
# ifconfig lo0 alias 9.24.105.87 netmask 255.255.255.0

Important: Make sure you use the netmask option with the correct netmask of your network. If you use a wrong netmask, or do not use the option at all, a new route will be added to your routing table using the loopback as gateway, causing you to experience several routing problems. Add this command to the end of the /etc/rc.net file so it will be run every time the networking configuration is run (for example, after a system reboot).

100

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

If you want to confirm that the alias was added to the loopback adapter, run the following command:
# ifconfig lo0

You will see output similar to that in Example 4-6.

Example 4-6 Loopback configuration lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

You can test to see if your server is answering requests sent to the IP address of the cluster by using a browser on the local machine and requesting the IP address of the cluster. The following table shows which command you need to use to add an alias on different platforms.
Table 4-2 Adding an IP alias Platform AIX HP-UX Linux OS/2 Solaris Command ifconfig lo0 alias <cluster IP address> netmask <netmask> ifconfig lo0 <cluster IP address> ifconfig lo:1 <cluster IP address> netmask 0.0.0.0 up For the second alias, use lo:2, and so forth. ifconfig lo <cluster IP address> ifconfig lo0:1 <cluster IP address> 127.0.0.1 up

After adding the alias, check for the extra route that may have been created, and remove it according to the correct procedures for each operating system. Some operating systems do not allow adding an alias to the loopback adapter. One solution is to change the loopback IP address, but you will not be able to use this server on more than one cluster. Important: If you test the access with a browser and do not get a response, but other services are answering to the cluster IP address (FTP, for example), you may need to configure your HTTP server to listen to the IP address of cluster (refer to the documentation of your HTTP server for more information on how to do this).

Chapter 4. Network Dispatcher basic configuration

101

4.1.3 Tests
The first thing we did for our scenario was to check if the advisor was getting responses from the back-end servers, and if these were active. We used the command ndcontrol manager report. The output is shown in Example 4-7.
Example 4-7 ndcontrol manager report output
C:\>ndcontrol manager report ---------------------------------| HOST TABLE LIST | STATUS | ---------------------------------| 9.24.105.39| ACTIVE| | 9.24.105.59| ACTIVE| ------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.87| WEIGHT | ACTIVE % 40 | NEW % 40 | PORT % 20 | SYSTEM % 0 | ---------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD | ---------------------------------------------------------------------------------------------------| 9.24.105.39| 8 | 8 | 10 | 0 | 9 | 1 | 5| 531| 0| 0| | 9.24.105.59| 10 | 10 | 10 | 0 | 10 | 1 | 14| 110| 0| 0| ---------------------------------------------------------------------------------------------------| PORT TOTALS: | 18 | 18 | | 0 | | 2 | | 641 | | 0 | ----------------------------------------------------------------------------------------------------------------------------------| ADVISOR | PORT | TIMEOUT | -------------------------------| http | 80 | unlimited | --------------------------------

The first table (HOST TABLE LIST) shows the list of back-end servers and their status. In this scenario, we have two back-end servers (9.24.105.39 and 9.24.105.59) and they are both active. The next table shows a list of servers per port. In this case, we configured port 80 which balances to two servers. This table also shows the weight, number of active and new connections, the load value informed by the advisor and by ISS. This information is shown for each server. The last table shows information about the advisors: the name of the advisor that was started, the port and the timeout value attributed to it.

102

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

To test the load balancing with several connections, we created a script on a Linux machine to send an HTTP request to the cluster address, using the wget command. The script is shown in Example 4-8.
Example 4-8 Script to send HTTP requests #!/bin/bash while true do wget wses1.itso.ral.ibm.com sleep 1 done

You can use different values with the sleep command to increase or decrease the delay between requests sent, or even uncomment it if you want to produce heavier traffic. Since this script will get the index.html file from the Web server and store it locally, we recommend that you run it inside a temporary directory. To interrupt this script, use CTRL+C. For our purposes, we started the script and the output of ndcontrol manager report showed us the traffic that was generated (see Example 4-9).
Example 4-9 ndcontrol manager report output after creating several HTTP requests
C:\>ndcontrol manager report ---------------------------------| HOST TABLE LIST | STATUS | ---------------------------------| 9.24.105.39| ACTIVE| | 9.24.105.59| ACTIVE| ------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.87| WEIGHT | ACTIVE % 40 | NEW % 40 | PORT % 20 | SYSTEM % 0 | ---------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD | ---------------------------------------------------------------------------------------------------| 9.24.105.39| 9 | 9 | 10 | 0 | 10 | 72 | 5| 541| 0| 0| | 9.24.105.59| 10 | 10 | 10 | 0 | 9 | 81 | 14| 120| 0| 0| ---------------------------------------------------------------------------------------------------| PORT TOTALS: | 19 | 19 | | 0 | | 153 | | 661 | | 0 | ----------------------------------------------------------------------------------------------------------------------------------| ADVISOR | PORT | TIMEOUT | -------------------------------| http | 80 | unlimited | --------------------------------

Chapter 4. Network Dispatcher basic configuration

103

You can also see the traffic being balanced using the server monitor on the GUI. Click Port:80 in the left pane of the window, and in the context menu click Port:80->Monitor.... The monitor window showing the traffic produced by our tests is shown in Figure 4-30.

Figure 4-30 Server monitor window showing traffic

4.2 High availability

Network Dispatcher provides a built-in high availability function. It allows you to configure a backup machine for your load balancer server; if case the primary load balancer fails, this backup machine will take over the load balancing for all clusters.

104

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The high availability configuration detects and recovers from network and server failures. Network Dispatcher is smart enough to determine that a server or a network is down. In case of a server failure, clients lose only the current connections, but they can immediately establish a new connection to the remaining servers with no problem. The high availability environment involves two Dispatcher machines with connectivity to the same clients and to the same cluster of servers, as well as connectivity between themselves. Both Network Dispatcher servers must be using the same operating systems, and they must be connected to the same network. The two Dispatcher machines are usually referred to as the primary machine and the backup machine. The primary machine normally works as a Dispatcher, and is in active state while it is balancing the load among the servers of its clusters. The backup machine, configured in a very similar way to the primary machine, stays in standby mode unless the primary machine fails. The two machines are synchronized (which means the active server sends the connection table to the standby server) and only the primary machine routes packets, while the backup machine is continually being updated. Note: You can also configure the backup machine to respond to other clusters addresses while it works as a backup for the clusters on the primary machine. See Chapter 5.6, Mutual high availability on page 206. The two machines establish communication to monitor each others status, referred to as a heartbeat, using a port that you can choose. The standby (backup) machine uses the heartbeat and multiple reachability criteria to decide if a failure has occurred on the active (primary) server. If the primary machine fails, the backup machine detects this failure, switches to active state, and begins taking over the routing of packets. When the primary machine is operational again, but in standby state, you can either decide that it automatically becomes the active machine, or leave it in standby mode. In this last case, you will have to manually change its status if you later want it to become the active machine again. This is known as auto recovery or manual recovery.

Chapter 4. Network Dispatcher basic configuration

105

4.2.1 Scenario description

For this scenario, we used the same layout as we did for load balancing (see 4.1, Load balancing on page 70), and we added just one more machine to act as the backup machine in the high availability environment (see Figure 4-31).

9.24.105.39 m23kk904

9.24.105.59 rs60008

Web servers

Port 80 cluster: 9.24.105.87 wses1 9.24.105.62 rs600037 Primary

Figure 4-31 High availability scenario

Network Dispatchers
High availability 9.24.105.60 rs600010 Backup

The table below describes the machines we used in this scenario.

Table 4-3 Machines used in the high availability scenario Host name rs600037 rs600010 rs60008 IP Address 9.24.105.62 9.24.105.60 9.24.105.59 Operating System and Software AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Service Primary Dispatcher Backup Dispatcher Web server

m23kk904

9.24.105.39

Web server

106

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

4.2.2 Configuration
We performed the configuration of the high availability feature of Network Dispatcher in the following order: 1. Configuration of load balancing on the primary machine, and tests 2. Configuration of high availability on the primary machine 3. Configuration of load balancing on the backup machine, and tests 4. Configuration of high availability on the backup machine 5. Creation of high availability scripts on both machines, and tests We will now explain each of these steps in more detail.

Configuration of load balancing on the primary machine

We used the same configuration shown in 4.1, Load balancing on page 70. However, this time we used Network Dispatcher on an AIX machine, as shown in Figure 4-31 on page 106. We used the same back-end servers and the same cluster IP address (see Figure 4-32).

Chapter 4. Network Dispatcher basic configuration

107

Figure 4-32 Configuration of the primary Network Dispatcher server

Configuration of high availability on the primary machine

After we have the load balancing environment working, we need to make one change before proceeding to the high availability configuration: we need to unconfigure the cluster IP address by removing the IP alias that was added to the network interface of the machine. Open the GUI on the machine that you selected to be the primary machine, and connect to the local server. Click Cluster in the left pane of the window, and click Cluster->Unconfigure cluster address... in the context menu. A pop-up window will be displayed, as shown in Figure 4-33. Click Yes to confirm.

108

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-33 Unconfiguring the cluster address

To start adding the high availability information, click Executor in the left pane of the window, and in the context menu click Executor->Add High Availability Backup.... See Figure 4-34.

Figure 4-34 Configuring high availability on the primary machine

Choose a port number that will be used by both Network Dispatcher servers to exchange the information needed to keep them synchronized, and fill it in the Port number field. You can choose any available port, you just need to make sure that this port is available on both servers. If you want to check which ports are currently in use, run the following command (in both UNIX and Windows environments):
# netstat -an

Chapter 4. Network Dispatcher basic configuration

109

In the Role field, select the role that this machine will have in the high availability scenario (primary, backup, or both). In our scenario, this machine is our primary machine, so we selected primary. In the Recovery strategy field, choose how your primary machine is going to behave in case the backup machine has taken over. If you select Auto, it will take over as soon as it is up (and responding to the network). If you select Manual, you will need to run a command to force it to take over. As long as you do not run the command, the backup remains active. In our scenario, we selected Auto. Note: If you select Auto in the Recovery strategy field, you may have problems when you want to run maintenance procedures on the primary machine, since it will try to take over after every reboot. In this case, we recommend that you select Manual as the Recovery strategy field, or that you not allow the Network Dispatcher to start after the reboot while you are doing the maintenance procedures on the machine. The last things you need to add in this window is the heartbeat source address and the heartbeat destination address. The heartbeat is a ping packet that is sent from the local machine to the other server in the same high availability cluster to make sure it is responding. In the heartbeat source address field, fill in the local IP address of the machine. In the heartbeat destination address field, fill in the IP address of the backup machine. When you are finished, click OK. Back in the GUI, the next thing we need to add is a reach target IP address. This works in a similar way to the heartbeat, but this time we use another machine as the destination for the ping packet. We recommend that you use as the reachability target a machine that is also guaranteed to be up. For example, you can use the IP address of your router (use the IP of the interface that is directly connected to the local network). We used the IP address of our local router, which is 9.24.105.1. To add this IP address as a reach target IP address, click Executor in the left pane of the window, then go to the context menu and click Executor->Add Reach Target. A pop-up window will be displayed, as shown in Figure 4-35.

110

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-35 Adding a reach target on the primary machine

Fill in the IP address of the machine you selected to be the reach target, and click OK. Note: If the Network Dispatcher machine is connected to two or more networks, we recommend that you select a reach target in each network.

Chapter 4. Network Dispatcher basic configuration

111

After we add this information, our primary machine is ready. See Figure 4-36 for the full configuration (look at the pane on the right).

Figure 4-36 High availability configuration for the primary machine

112

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Although the configuration is not complete (we still have to configure the backup machine), you can see the information and status of your primary machine by running the command ndcontrol high status, as shown in Example 4-10.
Example 4-10 ndcontrol high status on the primary machine # ndcontrol high status High Availability Status: ------------------------Role ................. Primary Recovery strategy .... Auto State ................ Active Sub-state ............ Not Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... n/a Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.62/9.24.105.60 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

Note that all the information you have entered is shown, as is the status of the communication between the two servers (the Sub-state) and the status of the heartbeat and reachability. Since we have not yet configured the backup machine, the Sub-state is shown as Not Synchronized.

Chapter 4. Network Dispatcher basic configuration

113

Configuration of load balancing on the backup machine

We will now prepare the backup machine. Follow the same steps as described in 4.1, Load balancing on page 70 to complete the load balancing configuration. The only difference this time is that you must not configure the cluster when you are adding the cluster IP address. See Figure 4-37.

Figure 4-37 Adding a cluster on the backup machine

You must fill in the cluster IP address in the Cluster address field. In the Primary host for the cluster field, you must enter the IP address of the primary machine. Leave the Configure this cluster? checkbox unselected, because we are going to use the high availability scripts to add the IP alias to the proper network interface.

114

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Configuration of high availability on the backup machine

After you have finished the load balancing configuration, you can add the high availability configuration. In the left pane of the window, click Executor, then go to the context menu and click Executor->Add High Availability Backup.... The panel shown in Figure 4-38 will be displayed.

Figure 4-38 Configuring high availability on the backup machine

This is the same window that you see when adding the primary machine (see Figure 4-34 on page 109). In the Port number field, make sure you use the same port number that you selected on the primary machine. In the Role field, select Backup. In the Recovery strategy field, select the same strategy you selected for the primary machine. In our scenario, we selected Auto. For the heartbeat information required, you must provide the source (the local IP address) and the destination (the IP address of the primary machine). Click OK to add the high availability configuration.

Chapter 4. Network Dispatcher basic configuration

115

Next, you need to add a reach target. We recommend that you use the same machine(s) you used on the primary machine. Click Executor in the left pane of the window, then go to the context menu and click Executor->Add Reach Target. A pop-up window will be displayed, as shown in Figure 4-39.

Figure 4-39 Adding a reach target on the backup machine

Fill in the IP address of the target, and click OK.

116

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Look at Figure 4-40 to see the complete high availability configuration on the backup machine (look at the right pane).

Figure 4-40 High availability configuration for the backup machine

As before, we can now run the ndcontrol high status command on both machines, and compare the output.

Chapter 4. Network Dispatcher basic configuration

117

See in Example 4-11 the output of this command on the primary machine after adding the backup machine.
Example 4-11 ndcontrol high status on the primary machine # ndcontrol high status High Availability Status: ------------------------Role ................. Primary Recovery strategy .... Auto State ................ Active Sub-state ............ Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... 9.24.105.60 Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.62/9.24.105.60 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

Note that all information remains unchanged, except for the Sub-state, which now shows Synchronized.

118

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

See the output of the same command on the backup machine in Example 4-12.
Example 4-12 ndcontrol high status on the backup machine # ndcontrol high status High Availability Status: ------------------------Role ................. Backup Recovery strategy .... Auto State ................ Standby Sub-state ............ Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... 9.24.105.62 Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.60/9.24.105.62 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

The output is similar to the output for the primary machine, except for the Role (this one is Backup), the State (it is Standby now, which means that the other server is active) and the remote IP addresses, which point to the IP address of the primary machine. As soon as you finish configuring both servers, we recommend that you make a backup of the Network Dispatcher configuration, as described in Saving the configuration on page 95.

Chapter 4. Network Dispatcher basic configuration

119

Creating the high availability scripts

The following are the four scripts that Network Dispatcher uses: goActive: This script is run every time the Network Dispatcher server goes into active state. This script deletes loopback aliases and adds devices aliases. This script is run every time the Network Dispatcher server goes into standby state. This script deletes devices aliases and adds loopback aliases. This script is run when the Network Dispatcher executor is stopped, or when it is started for the first time. This script deletes all devices aliases and loopback aliases. This script is run when the Network Dispatcher server goes into idle state (when the machine is a stand-alone dispatcher, or when the high availability features have not yet been added). Do not create this script if you are using high availability.

goStandby:

goInOp:

goIdle:

These scripts must be placed in the <product install path>/dispatcher/bin directory. You will find a set of examples in the <product install path>/dispatcher/samples directory. You can start building your own scripts using these examples. If you want to use the examples, copy them to the bin subdirectory, and edit them there. Important: If you are using UNIX, you need to add the execution permission to the scripts after you create them. Make sure to run the following command:
# chmod u+x <product install path>/bin/go*

Our scenario is simple: we use one cluster for load balancing two back-end Web servers. When either of the servers (primary or backup) takes over, it needs to start responding to requests sent to the cluster IP address, so it has to add an IP alias of the cluster IP address to the local network interface. This will be done in the goActive script, on both machines (but it will run only on the machine going into active state).

120

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

A simple goActive script is shown in Example 4-13.

Example 4-13 The goActive script #!/bin/ksh # goActive script INTERFACE=en0 CLUSTER=9.24.105.87 NETMASK=255.255.255.0 ifconfig lo0 delete $CLUSTER ifconfig $INTERFACE alias $CLUSTER netmask $NETMASK

Note: The scripts are usually identical for the primary machine and the backup machine, unless there is some particular command you need to run in each machine. In our scenario, we used the exact same script on both machines. The next script is goStandby, which deletes the cluster IP alias from the device and adds it to the loopback, as shown in Example 4-14.
Example 4-14 The goStandby script #!/bin/ksh # goStandby script INTERFACE=en0 CLUSTER=9.24.105.87 NETMASK=255.255.255.0 ifconfig $INTERFACE delete $CLUSTER ifconfig lo0 alias $CLUSTER netmask $NETMASK

The last script in our customization is goInOp, which is supposed to delete all aliases, as shown in Example 4-15.
Example 4-15 The goInOp script #!/bin/ksh # goStandby script INTERFACE=en0 CLUSTER=9.24.105.87 NETMASK=255.255.255.0 ifconfig $INTERFACE delete $CLUSTER ifconfig lo0 delete $CLUSTER

Once these scripts are created on both machines, you must stop the executor (using the ndcontrol executor stop command) and restart Network Dispatcher before starting the tests, so that it can run the scripts upon starting. Do it first on the primary machine; after you start the primary machine, do it on the backup machine. For more information on starting and stopping the servers, refer to Chapter 3, Network Dispatcher installation on page 27.

Chapter 4. Network Dispatcher basic configuration

121

Configuration files
If you want to use the command line interface, here are the commands that were generated by following the configuration steps above. Example 4-16 lists the statements for the primary machine configuration file.
Example 4-16 primary machine configuration file ndcontrol executor start ndcontrol executor set nfa 9.24.105.62 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.105.39 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log ndcontrol highavailability heartbeat add 9.24.105.62 9.24.105.60 ndcontrol highavailability backup add primary auto 12345 ndcontrol highavailability reach add 9.24.105.1

122

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Example 4-17 lists the statements for the backup machine configuration file.
Example 4-17 backup machine configuration file ndcontrol executor start ndcontrol executor set nfa 9.24.105.60 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.105.39 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log ndcontrol highavailability heartbeat add 9.24.105.60 9.24.105.62 ndcontrol highavailability backup add backup manual 12345 ndcontrol highavailability reach add 9.24.105.1

4.2.3 Tests
First, review what kind of recovery strategy you selected. If you selected Auto, the only way you can test the high availability configuration is by forcing the active machine to fail. If you selected Manual, you can also use a command to force the takeover. To check if your configuration is up and running, run the command ndcontrol high status and locate the State field, as shown in Example 4-11 on page 118 and Example 4-12 on page 119.

Tests with auto recovery

The difference between auto and manual recovery is the behavior of the environment after a failed server has been recovered. If it is an auto recovery and the backup machine became active because the primary machine was not responding, then as soon as the primary machine is available, it will become active and the backup machine will be in standby mode again.

Chapter 4. Network Dispatcher basic configuration

123

If it is a manual recovery, the automatic takeover is only performed in case of failure of the active machine, or by running a command (see also Tests with manual recovery on page 127). So, if the primary machine fails, the backup machine becomes active. When the primary machine is back online again, it does not become active until the administrator explicitly commands it to do so, or unless the backup machine fails. Note: No matter what type of recovery strategy you selected, the backup server will automatically take over in case of failure of the primary server. In testing, when using auto recovery you have to simulate a problem with the active server to force the takeover, since you cannot force it using any command. A simple way to do that is to disable the network interface. In our scenario, rs600037 is the primary machine, and rs600010 is the backup machine. We checked that both machines were working properly, using the ndcontrol high status command. Below is the output for the primary machine:
Example 4-18 Status of the primary machine High Availability Status: ------------------------Role ................. Primary Recovery strategy .... Auto State ................ Active Sub-state ............ Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... 9.24.105.60 Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.62/9.24.105.60 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

Note that the Sub-state is synchronized, which means that both Network Dispatcher servers are currently exchanging information through the network. Also, the Reachability status shows that the router is reachable.

124

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Now look at the same command output for the backup machine.
Example 4-19 Status of the backup machine High Availability Status: ------------------------Role ................. Backup Recovery strategy .... Auto State ................ Standby Sub-state ............ Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... 9.24.105.62 Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.60/9.24.105.62 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

The same information is given about the backup machine as about the primary machine. Once both servers were up and synchronized, we stopped the Ethernet interface on the primary machine, using the command below.
# ifconfig en0 down

That forces the backup machine to take over.

Chapter 4. Network Dispatcher basic configuration

125

A few seconds after running this command, we can see that the backup machine has taken over the primary Dispatcher functions by running the ndcontrol high status command again:
Example 4-20 Status after takeover High Availability Status: ------------------------Role ................. Backup Recovery strategy .... Auto State ................ Active Sub-state ............ Not Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... n/a Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.60/9.24.105.62 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

Note that this machine is active, but the Sub-state is not synchronized (because it cannot communicate with the primary machine). If you look at the IP addresses on the network interfaces of the backup machine, you will notice that this machine now has an IP alias on the Ethernet interface for the cluster IP address.
Example 4-21 IP addresses of the backup machine after takeover # ifconfig -a lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT, PSEG> inet 9.24.105.60 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

126

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

We ran the script shown in Example 4-8 on page 103 to send HTTP requests to the cluster address while we were doing tests with high availability, to make sure that there would be no interruption on the communication between the client and the back-end Web servers.

Tests with manual recovery

If you configured the high availability recovery strategy as manual, you can start your tests by forcing the backup machine to take over. To do so, go to the backup machine, and run the following command:
# ndcontrol high takeover

Restriction: You can only run the ndcontrol high takeover command if the following three conditions are met: 1. the recovery strategy is manual 2. the machine state is Standby 3. the Sub-state is Synchronized You can confirm this information by running the ndcontrol high status command. In case you decided to use the manual recovery, you can also do tests using the ifconfig <interface> down command, as described in Tests with auto recovery on page 123.

4.3 Using Interactive Session Support

By using the monitoring capabilities provided by Interactive Session Support (ISS) on the back-end server machines, you can provide the Dispatcher with server load information. In this case, ISS cooperates with the Dispatcher, but ISS does not make any load balancing decision. The ISS monitor collects specific server information such as CPU usage, memory usage and disk activity from the ISS agents running on the individual servers, and forwards it to the Dispatcher. The Dispatcher uses this load information, along with other sources of information, to determine which is the least loaded server of the cluster, then performs load balancing.

Chapter 4. Network Dispatcher basic configuration

127

4.3.1 Scenario description

We used the same scenario as described in 4.1, Load balancing on page 70, as shown in Figure 4-41.

9.24.105.39 m23kk904

Web servers

9.24.105.59 rs60008

ISS

ISS

ISS

Cluster: 9.24.105.87 wses1

Figure 4-41 ISS scenario

Network Dispatcher
9.24.105.82 m238p4yl

The table below describes the machines we used in this scenario.

Table 4-4 Machines used in the high availability scenario Host name rs600037 rs60008 IP Address 9.24.105.62 9.24.105.59 Operating System and Software AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Service Network Dispatcher Web server

m23kk904

9.24.105.39

Web server

128

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

4.3.2 Configuration
In this section, we discuss all the steps necessary to configure ISS on the back-end servers and to configure them to send the information to the ISS monitor.

Installation
As already mentioned, Network Dispatcher has three subcomponents: Dispatcher, ISS and CBR. In this section, we discuss the installation of the ISS component. Important: ISS requires the installation of Java Development Kit (or Java Runtime) V1.3.0. Make sure that there will be no conflicts with any software you have installed on your back-end servers. ISS is supported on three operating systems: AIX, Windows NT and 2000, and Solaris. Refer to the appropriate platform section of Chapter 3, Network Dispatcher installation on page 27 for details on how to install ISS on AIX and Windows NT. When you reach the point where you must choose which ND component to install, select these three components to install ISS on your back-end server: Interactive Session Support Runtime Interactive Session Support Administration Interactive Session Support License

Chapter 4. Network Dispatcher basic configuration

129

Figure 4-42 shows these components selected during the installation procedure:

Figure 4-42 Installing ISS components

ISS has a special system monitoring function. If you want ISS to be part of your environment, you should select one machine to run as an ISS monitor, while an ISS agent process runs on the back-end servers in the cluster, gathering information about the status of the systems and feeding it back to the ISS monitor. The three ISS components listed above should be installed on the ISS monitor machine as well as on the back-end server machines. On Windows NT, at the end of the ISS component installation, you will be instructed to reboot. After rebooting, notice that the IBM Interactive Session Support service is not automatically started, as shown in the Services folder of the Control Panel (see Figure 4-43).

130

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-43 The ISS server is not automatically started after rebooting

You should click Start to activate the ISS daemon. Refer to Starting the ISS daemon on page 133 for further information on this.

Scenario configuration
In this section, we show you how to create a scenario where both the Dispatcher and ISS components of Network Dispatcher are used. Here, we worked in the same environment that we described in 4.1, Load balancing on page 70, but we enhanced that scenario by adding the ISS function. In the environment shown in Figure 4-41 on page 128, we decided to have the Dispatcher machine run as the ISS monitor, too. ISS is used here only in its capacity to gather server load information from the back-end servers, where the ISS agent process runs. This information is then passed back to the ISS monitor process, running in this case on the same Dispatcher machine. The monitor interacts with the Dispatcher and provides it with information about the loads on the servers. The Dispatcher uses this information along with other sources to perform the load balancing. The steps required to configure ISS in this information-gathering capacity start out essentially the same as if ISS were to perform the load balancing. The key difference between the information gathering function and the load balancing function of ISS is the choice of Observer type, as explained below.

Chapter 4. Network Dispatcher basic configuration

131

Network environment
Figure 4-41 on page 128 shows the environment we used. Note that as soon as you make ISS part of the load balancing process, the Manager configuration must reflect the presence of ISS. To ensure this, change the Manager proportions (see Setting the manager proportions on page 93) to take into account the input coming from ISS. In this case, we set the proportions of importance for the Manager to 48, 48, 2, and 2. The following example demonstrates how easy it is to implement ISS high availability.

ISS configuration
In 4.1, Load balancing on page 70, we described the network environment we had set up using the Dispatcher, but without using ISS. In this section, we show how to add ISS to the already existing environment.

ISS configuration methods

In this section, we show how to make some changes to the sample configuration file shipped with the product, then how to complete the customization with the GUI. Each customization that we perform with the GUI can also be performed on the command line with the isscontrol command. We will supply the syntax of the corresponding isscontrol command that can be used to accomplish several of the tasks for which we used GUI. For further details on the isscontrol command, please refer to IBM Network Dispatcher Users Guide Version 3.0 for Multiplatforms, GC31-8496-05. The Dispatcher sample configuration file is <install path>/iss/iss.cfg, where <install path> varies by operating system. The original Dispatcher sample configuration file for the Windows NT platform, as it is shipped with the product, is shown in Example 4-22.
Example 4-22 ISS shipped sample configuration file # This is a default configuration file which should be located in # the root ISS directory (nd\iss). Replace yourHostName with the # hostname of this machine and uncomment the line. # This is the minimal configuration needed for ISS to # run successfully. Cell issCell local 01

# Node yourHostName

The sample file in Windows NT is C:\Program Files\Ibm\nd\iss\iss.cfg.

132

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Example 4-23 shows our customized version of the Dispatcher sample file. You do not need to rename the file; you can simply edit the sample file. In AIX, you can edit the sample file etc/iss.cfg and you will not need to rename it either. As the comments in the Dispatcher sample file indicated, you must create this minimal configuration file before you use the ISS GUI if you do not have any preexisting configuration files. We replaced issCell with Cary, we removed the command in the Node line and we replaced yourHostName with m238p4yl.
Example 4-23 Customized Dispatcher ISS configuration file # This is a default configuration file which should be located in # the root ISS directory (nd\iss). Replace yourHostName with the # hostname of this machine and uncomment the line. # This is the minimal configuration needed for ISS to # run successfully. Cell Cary local Node m238p4yl 01

Important: When you start using ISS, all the machines in your cell must have the same ISS configuration file. In order to perform the configuration steps listed below, you must be the root user on AIX and Solaris or, if the Dispatcher is installed on Windows NT, you must be a system administrator.

Starting the ISS daemon

ISS operates as either the monitor or the agent, depending on the contents of the configuration file or parameters entered. To start the ISS daemon on AIX and Solaris from a command line, enter the command:
iss -g

The -g flag instructs the ISS daemon to monitor for communication from the ND GUI on port 12099. If your iss.cfg file is not located in /etc or is not called iss.cfg, then you will need to use the -c flag followed by the full path to the configuration file to let ISS know where the file is. The -l flag can be used to specify a log file.

Chapter 4. Network Dispatcher basic configuration

133

Note that on Windows NT, ISS is a Windows NT service and is not automatically started. To start ISS as the monitor, open the Services window of the Windows NT Control Panel (see Figure 4-43 on page 131), select IBM Interactive Session Support and then click Start. In the Startup Parameters text field, you can specify the arguments for the iss command. Notice, however, that if you want to code a backslash (\), you have to type a double backslash (\\). For example, if you want to specify the configuration file C:\Myconf\iss1.cfg and the log file C:\Myconf\iss1.log, you have to type:
-c C:\\Myconf\\iss1.cfg -l C:\\Myconf\\iss1.log

Since we are using Windows NT, and it is also the machine where we are starting the ISS daemon as a monitor, we start it with the parameter -g, as shown in Figure 4-44.

Figure 4-44 Starting the ISS daemon using the parameter -g

134

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Starting the configuration GUI

Now you can start the GUI. To do so, click Start->Programs->IBM WebSphere->Edge Server->IBM Network Dispatcher. If you are using a UNIX operating system, you can start the GUI by running the command ndadmin. Figure 4-45 shows the GUI window.

Figure 4-45 Network Dispatcher GUI window

The left pane displays a tree structure, with Network Dispatcher at the top level, and Dispatcher, Interactive Session Support and Content Based Routing as components if they are installed. You can select elements in the tree structure by left-clicking, and you can display pop-up menus by right-clicking. The pop-up menus for the tree elements are also accessible from the menu bar located at the top of the window. Each item in the tree is marked with a plus sign (+) or a minus sign (-). Click the plus sign (+) to expand the items within your selection and the minus sign (-) to minimize those items.

Chapter 4. Network Dispatcher basic configuration

135

Connecting to a host
The next step in configuring ISS is to make a connection to a host where the ISS daemon is running. In this case, we are performing this configuration on the ISS manager machine itself. If you cannot run the GUI from the same machine, refer to 5.1, Remote configuration on page 156 for information on how to make a remote connection through the GUI. To make the connection, right-click Interactive Session Support in the tree structure. In the pop-up window that appears, select Connect to Host.... We selected the host name of the machine where we are performing the initial ISS configuration, the same machine where we would be running the ISS monitor and the Dispatcher (see Figure 4-46).

Figure 4-46 Selecting a host for the ISS configuration

Important: Make sure that you have started the ISS daemon with the parameter -g before connecting to it; otherwise, you will receive the following error message: The are no more keys available for connecting to a host.

136

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

After clicking OK, the GUI was refreshed to show the ISS host connection and the single Cary cell that we had defined in our iss.cfg file. Selecting Cell: Cary and clicking the Configuration Settings tab shows us the default values (see Figure 4-47).

Figure 4-47 Default cell configuration settings

Chapter 4. Network Dispatcher basic configuration

137

Adding nodes
The next step is to add nodes to your configuration. To do this, right-click Cell: Cary and then select Add Node in the Cell. You will be prompted for the node name and the order number for the node, as shown in Figure 4-48.

Figure 4-48 Adding a node

We configured the Dispatcher machine m238p4yl to be the primary ISS monitor in the iss.cfg configuration file. We added m23kk904, and using the default settings, it is a backup to the monitor node. We added this node as node number 2. This means that m23kk904 will take over and become the ISS monitor, should the primary ISS monitor m238p4yl fail. These simple steps are enough to implement ISS high availability.

138

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

At this point, the GUI is updated to reflect the new node that we added (see Figure 4-49).

Figure 4-49 New node added to the Cary cell

Using the command line, we added the last node, rs60008, and set the CanMonitor to false using the following commands:
C:\> isscontrol add node rs60008 3 C:\> isscontrol set node rs60008 CanMonitor false

This new node will not be able to take over in case of failure of the previous nodes, because we set the CanMonitor to false. Figure 4-50 shows the settings of the rs60008 node.

Chapter 4. Network Dispatcher basic configuration

139

Figure 4-50 Configuration settings for the last node

When values are changed directly in the GUI window, it is important to click the Update Configuration button at the bottom of the window. This procedure writes the changes to the ISS configuration file (iss.cfg). If the nodes you are adding have multiple network interfaces on them, then you can define these individually by right-clicking the Node entry; this opens the Node menu, which can be used to select Add Interface to Node. In our scenario, our nodes had only one network interface, so we did not need this step.

140

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Defining resource types

The next step is to add resources to your configuration. To do this, right-click Cell: Cary and then select Add ResourceType in the Cell. In this case, we would like to add a CPU ResourceType and so we type CPU in the Add ResourceType window, as shown in Figure 4-51:

Figure 4-51 Adding a CPU resource type

Chapter 4. Network Dispatcher basic configuration

141

ResourceType entries are given default values (shown in Figure 4-52), which in this case are appropriate for the CPU ResourceType.

Figure 4-52 Default values for a CPU resource

We then modified the CPU ResourceType Recover limit (the level down to which CPU utilization should be taken before the server is considered for ranking) and Fail limit (the level of CPU utilization beyond which ISS removes the server from ranking) by entering the new values 80 and 95, respectively, directly into the GUI fields. After this, we clicked the Update Configuration button to update the ISS configuration file (iss.cfg).

142

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Defining services
The next step is to add services to our configuration. In general, a service is a group of servers measured by the same ResourceTypes (refer to Defining resource types on page 141). A node can be a member of a single service or of many services. For our configuration, we right-clicked Cell: Cary and then selected Add Service in the Cell. In the Add Service window, we gave the service a name, wwwservice, and also supplied a service DNS name. The Service DNS name is not needed when you are using ISS to update the Dispatcher as in this case. However, you must fill in the blank with a placeholder name; for this reason, we typed in PlaceHolder, as shown in Figure 4-53.

Figure 4-53 Adding a service

After adding the service name, we selected Add ResourceType to Service from the Service menu. In the Add ResourceType to Service window, we selected the resource type CPU that we had defined in Defining resource types on page 141 (see Figure 4-54).

Figure 4-54 Adding a resource type to a service

Chapter 4. Network Dispatcher basic configuration

143

After this, we selected Add Node Interface to Service from the Service menu; we were then presented with the Add Node Interface to Service window, showing the list of nodes we had added (see Adding nodes on page 138): see Figure 4-55.

Figure 4-55 Adding a node to the service selection list

We were able to select only one node at a time from this window. We added rs60008 through the GUI and added the other node to our wwwservice from the command line as follows:
C:\> isscontrol add node m23kk904 to service wwwservice

144

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The Interface List section of the Service information for the wwwservice service was updated to reflect the nodes we had added, as shown in Figure 4-56.

Figure 4-56 Service statistics showing resource type and nodes

Chapter 4. Network Dispatcher basic configuration

145

Defining observers
The next step is to add observers to the configuration. An observer is a load balancer or any other network process that uses information about the status of nodes in the cell. ISS supports three types of observers that can subscribe to reports on a cell and/or perform load balancing for that cell. For our configuration, we right-clicked Cell: Cary and then selected Add Observer in the cell menu. The Add Observer window contains three radio buttons at the top to select which type of Observer you are defining. We selected Dispatcher. ISS will work in conjunction with the Dispatcher to perform the load balancing function. ISS does not make any load balancing decisions itself. The ISS monitor collects server load information from the ISS agents running on the individual servers and forwards it to the Dispatcher. When we selected Dispatcher, the default port number changed to 10004. This is the port that is used to send the metric load information to the Dispatcher. In the selection list at the bottom of the Add Observer window, we selected the host where the Dispatcher server was running (m238p4yl). This host is the machine where the Dispatcher is running and to which ISS provided load statistics (see Figure 4-57).

Figure 4-57 Adding an Observer

After we clicked OK, we went back to the GUI and selected the new Observer, Observer: m238p4yl. The settings for this Observer were immediately displayed. Next, we added a service to this Observer by right-clicking Observer: m238p4yl in the tree and selecting Add Service to Observer.

146

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

A list of the defined Services was displayed in the Add Service to Observer window, and in this case there was only one: wwwservice. We selected it and clicked OK (see Figure 4-58).

Figure 4-58 Adding a service to the Observer window

Chapter 4. Network Dispatcher basic configuration

147

As a result of this, the Service List section for our m238p4yl Observer was updated to reflect the service name we had just added to our Observer (see Figure 4-59).

Figure 4-59 Updated Observer information showing the Service name

148

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

At this point, we examined our ISS configuration file, /etc/iss.cfg, and saw that it had been modified to include the ISS keywords required to create the ISS environment that we had configured through the GUI (see Example 4-24).
Example 4-24 Configuration file iss.cfg # ISS Configuration file # Automatically generated at 2001/04/02 17:33:39 Cell PortNumber LogLevel HeartbeatInterval HeartbeatsPerUpdate HeartbeatsToNetFail HeartbeatsToNodeFail Node Node Node NotMonitor ResourceType Metric Policy MetricWeight MetricNormalization MetricLimits Service SelectionMethod ResourceList NodeList Cary 7139 INFO 10 2 4 6 m238p4yl m23kk904 rs60008 LOCAL

1 2 3

CPU INTERNAL MIN 1.0 0.0 80.0 wwwservice BEST CPU rs60008 m23kk904 m238p4yl wwwservice

CPULoad

100.0 95.0 PlaceHolder 0

Dispatcher ServiceList

10004

Once the configuration of our environment was complete, we placed a copy of this configuration file on the two other nodes, then started the ISS daemon on them as well. On both Web server nodes, the ISS daemon started in its capacity as an agent as a result of the format of its own node entry in the configuration file.

Managing ISS
In this section, we explain how to stop and start both ISS and its configuration GUI. See Starting the ISS daemon on page 133 for details on how to start ISS, and Starting the configuration GUI on page 135 for instructions on how to start the configuration GUI.

Chapter 4. Network Dispatcher basic configuration

149

In the following section, we see how ISS can be controlled and reconfigured during run time.

Dynamically reconfiguring ISS

You can control and reconfigure ISS while it is running by using the isscontrol command or the GUI. In our scenario, Web server node m23kk904 was configured to be capable of taking over the function of ISS monitor if required. Web server node rs60008 was configured to not take over the function of ISS monitor. Changing the ISS configuration was done in two steps: 1. We changed the designation of Web server node rs60008 so that it could be selected as an ISS monitor. 2. We checked to see whether this change was reflected in the configuration file of one of our other nodes, for example m23kk904. Prior to the change being made, we used FTP to propagate the iss.cfg file to each of the nodes in our cell. We then started ISS on each of these nodes. Next, we verified (both in the ISS configuration file and via the GUI) that m23kk904 had been designated as a potential monitor. We did this verification on both m238p4yl and m23kk904. Figure 4-60 shows the m23kk904 node status as it appeared on m238p4yl.

150

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 4-60 m23kk904 node status

Chapter 4. Network Dispatcher basic configuration

151

We then used the GUI on m238p4yl to change the status of rs60008, by selecting Can Monitor on the Monitor pull-down menu (see Figure 4-61).

Figure 4-61 Changing the Monitor value for server rs60008

We then clicked the Update Configuration button at the bottom of the GUI. We also confirmed that the ISS configuration files were updated on all of the machines by looking at the C:\Program Files\Ibm\nd\iss\iss.cfg file on both m238p4yl (where the change was made) and m23kk904, and looking at the /etc/iss.cfg file on rs60008 (one of the nodes participating in the cell). We confirmed by the timestamp on the ISS configuration file on m23kk904 that the change had been automatically propagated from m238p4yl.

152

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Saving the host connections

The GUI host connections can be saved by selecting File on the menu bar of the configuration window and then clicking Save Host Connections As.... In the pop-up window that appears, enter the name of the configuration file where you want to save the Host Connection information (see Figure 4-62). This file is saved in the directory <install path>/admin/configurations, where <install path> varies by operating system.

Figure 4-62 Saving the host connection

As a result of saving this information, on subsequent invocations of the GUI this host connection will be listed as one of the choices in the Connect to Host selection window.

Disconnecting from the host

To disconnect from the host, right-click the Host item, either in the tree or on the menu bar, then select Disconnect from Host.

Terminating the GUI

To exit the GUI, select File->Exit. Your ISS configuration file should automatically be updated at this time.

Stopping ISS
On Windows NT, as a system administrator, from the Services folder of the Control Panel, select IBM Interactive Session Support, then click Stop. On AIX and Solaris, as root, enter:
# isscontrol shutdown <nodename>

Chapter 4. Network Dispatcher basic configuration

153

154

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 5.

Advanced features
This chapter provides details and scenarios describing the advanced features of Network Dispatcher.

Copyright IBM Corp. 2001

155

5.1 Remote configuration

Situations may arise where you cannot run the Graphical User Interface (GUI) on the same machine where you installed the Dispatcher, ISS or CBR components. You can still use the GUI remotely by installing it on a client machine which will communicate to Network Dispatcher through Java Remote Method Invocation (RMI) calls. In this case, you will need to create a set of keys (public and private keys) for authentication. Using the scenario shown in Figure 5-1, we set up a remote configuration client on the m238p4yl machine, and we connected to the Dispatcher component on the rs600037 machine.

9.24.105.39 m23kk904

9.24.105.59 rs60008

Web servers

Port 80 cluster: 9.24.105.87 wses1

Network 9.24.105.62 Dispatcher rs600037

Figure 5-1 Remote configuration scenario

9.24.105.82 m238p4yl

156

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The following table shows more information on the machines we used.

Table 5-1 Machines used in the remote configuration scenario Host Name m238p4yl rs600037 rs60008 IP Address 9.24.105.82 9.24.105.62 9.24.105.59 Operating System and Software Windows NT 4.0 Service Pack 5 AIX 4.3.3 WebSphere Edge Server V1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Service Configuration client Network Dispatcher Web server

m23kk904

9.24.105.39

Web server

These machines were all connected to the same Ethernet network. First, we installed the administration components (see 3.1.2, Choosing the components on page 29) on the Windows NT machine, which will work as a configuration client. Since this communication is authenticated, we need to create the public and private key on the Network Dispatcher machine. You can use the following commands, according to which components you need to manage remotely: ndkeys: creates keys for the Dispatcher component cbrkeys: creates keys for the Content Based Routing component isskeys: creates keys for the Interactive Session Support component These commands must be run on the machine on which you installed the components you want to configure (Dispatcher, CBR or ISS). In our scenario, we run the following command on the machine rs600037:
Example 5-1 Creating keys # ndkeys create Key files have been created successfully

This command creates a pair of keys: the public key is created in the key directory, and the private key is created in the admin directory. Note that each component has its own key subdirectory, and inside the admin directory you will find separate subdirectories for each component.

Chapter 5. Advanced features

157

The public key file is authorization.key. The private key file is named with the IP address of the server and the port number on which RMI is listening. In our scenario, the following files were created. Note that our server is running on AIX. public key: private key: /usr/lpp/nd/dispatcher/key/authorization.key /usr/lpp/nd/admin/keys/dispatcher/9.24.105.62-10099.key

After you locate the files, you need to copy the private key file to the remote configuration client machine, and place it under the directory <install path>/admin/keys/dispatcher. Since we are using a Windows NT machine as our client, we copied the 9.24.105.62-10099.key file to the C:\Program Files\Ibm\nd\admin\keys\dispatcher directory. After the file is copied, you just need to start the GUI and connect to the remote server, as described in 4.1.2, Configuration on page 71. If you want to authorize other machines to be used as remote configuration clients, you only need to copy the same key file to each one of them as described above. If you create a new pair of keys, the machines will not be able to access this server anymore unless you copy the new private key to them. You can also delete the keys on the server, using the delete parameter with the ndkeys, isskeys and cbrkeys commands. If you delete the keys, the remote clients will no longer be able to access this server.

5.2 Collocation
When you do not have a dedicated machine to run Network Dispatcher, and you install the software on one of the back-end servers, this is called collocation. In this case, the machine where Network Dispatcher is installed is also one of the balanced servers. This is useful for small environments, where you do not want to use one machine exclusively for Network Dispatcher. Collocation is supported on AIX, Red Hat Linux and Solaris. It is not supported on Windows NT and Windows 2000. In Linux platform, it is only supported in the absence of high availability. Important: Make sure you have the patch for the Linux loopback device as described in Patching the Linux kernel on page 92.

158

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

5.2.1 Scenario description

We created a simple scenario to illustrate this feature. We installed Network Dispatcher on a Linux machine, which also has a HTTP server running on it; it balances the load to itself and two other HTTP servers (see Figure 5-2).

9.24.105.39 m23kk904

9.24.105.59 rs60008

Web servers

Network Dispatcher
9.24.105.63 rhlinux1
Figure 5-2 Collocation scenario

Cluster: 9.24.105.89 wses3

In this scenario, all requests sent to the wses3.itso.ral.ibm.com cluster address will be received by the Network Dispatcher server (rhlinux1) which will load balance between the three HTTP Servers: rhlinux1, m23kk904 and rs60008.

Chapter 5. Advanced features

159

Table 5-2 shows more information on the machines we used in this scenario.
Table 5-2 Machines used in the collocation scenario Host Name rhlinux1 rs60008 IP Address 9.24.105.63 9.24.105.59 Operating System and Software Red Hat Linux 6.2 WebSphere Edge Server V1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Service Dispatcher Web server

m23kk904

9.24.105.39

Web server

These machines were all connected to the same Ethernet network.

5.2.2 Configuration
You can follow the steps in Chapter 4.1, Load balancing on page 70 to create a basic load balancing scenario without using collocation (we will add the collocation option after this). In this scenario, we are using the wses3 cluster (IP address 9.24.105.89). In this first step, we added two Web servers: m23kk904 (IP address 9.24.105.39) and rs60008 (IP address 9.24.105.59) to port 80 (see Figure 5-3).

160

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 5-3 Basic scenario without collocation

We then added a new server for the 9.24.105.89 cluster, and this new server is the local IP of the machine where we installed Network Dispatcher. We need to use the same IP address that we defined as NFA. If you are using the GUI, the system selects the NFA automatically. It is the IP address that appears associated with the Executor in the left pane of the GUI window (see Figure 5-3). In our scenario, the IP address is 9.24.105.63.

Chapter 5. Advanced features

161

To add this server, click Port:80 in the left pane of the window, then click Port:80->Add Server... in the context menu. In the pop-up window, type the IP address of the new server, as shown in Figure 5-4.

Figure 5-4 Adding a server

After adding the server, click the IP address of the server in the left pane of the window (in our scenario, click 9.24.105.63), and click the tab Configuration Settings in the right pane, as shown in Figure 5-5.

162

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 5-5 Configuring the collocated server

Locate the Collocated field, and change it to yes. Click the Update Configuration button at the bottom of this pane.

Chapter 5. Advanced features

163

After configuring the collocated server, make sure that an IP alias was added to the local adapter (for more information refer to Checking the IP aliases on the Dispatcher on page 90). Make sure that the IP alias was created with the cluster IP address, as shown in Example 5-2.
Example 5-2 IP addresses associated with the local server # ifconfig -a eth0 Link encap:Ethernet HWaddr 00:60:94:31:05:94 inet addr:9.24.105.63 Bcast:9.24.105.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1921246 errors:0 dropped:0 overruns:0 frame:0 TX packets:336399 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 Interrupt:14 Base address:0xc000 eth0:1 Link encap:Ethernet HWaddr 00:60:94:31:05:94 inet addr:9.24.105.89 Bcast:9.255.255.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 Interrupt:14 Base address:0xc000 Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:3924 Metric:1 RX packets:6331 errors:0 dropped:0 overruns:0 frame:0 TX packets:6331 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0

lo

If this IP alias does not exist, you need to configure the cluster address. Click Cluster in the left pane of the window, and click Cluster->Configure Cluster Address... in the context menu. A pop-up window will be displayed, as shown in Figure 5-6.

Figure 5-6 Configuring the cluster address

164

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

You need to type in the interface name that should be associated with the cluster (if you are not sure which name you should use, refer to Basic configuration using the GUI on page 81). Since we are using Linux, we used eth0 (the first Ethernet interface). We also recommend that you type in the correct netmask; do not leave the Netmask field blank. Typing in the wrong netmask or leaving the field blank will cause the system to create a route in the routing table, which may cause problems in your environment. Click OK, and check the ifconfig -a command again to make sure that the IP alias was added successfully.

5.2.3 Tests
First, we can take a look at the output of ndcontrol manager report, to make sure that all servers are active, and the advisor is receiving responses from all of them (see Example 5-3).
Example 5-3 ndcontrol manager report output

---------------------------------| HOST TABLE LIST | STATUS | ---------------------------------| 9.24.105.39| ACTIVE| | 9.24.105.59| ACTIVE| | 9.24.105.63| ACTIVE| ------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.89| WEIGHT | ACTIVE % 49 | NEW % 49 | PORT % 2 | SYSTEM % 0 | ---------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD | ---------------------------------------------------------------------------------------------------| 9.24.105.39| 9 | 9 | 10 | 0 | 10 | 0 | 3| 304| 0| 0| | 9.24.105.63| 9 | 10 | 10 | 0 | 10 | 0 | 10| 156| 0| 0| | 9.24.105.59| 10 | 10 | 10 | 0 | 10 | 0 | 15| 29| 0| 0| ---------------------------------------------------------------------------------------------------| PORT TOTALS: | 28 | 29 | | 0 | | 0 | | 489 | | 0 | ----------------------------------------------------------------------------------------------------------------------------------| ADVISOR | PORT | TIMEOUT | -------------------------------| http | 80 | unlimited | --------------------------------

Chapter 5. Advanced features

165

We used the same script we mentioned in 4.1.3, Tests on page 102. This time, we used it to send HTTP requests to wses3 (see Example 5-4).
Example 5-4 New script to generate HTTP requests #!/bin/bash while true do wget wses3.itso.ral.ibm.com sleep 1 done

After starting this script, we can see through ndcontrol manager report that the load is being balanced among all three servers, as shown in Example 5-5.
Example 5-5 ndcontrol manager report
---------------------------------| HOST TABLE LIST | STATUS | ---------------------------------| 9.24.105.39| ACTIVE| | 9.24.105.59| ACTIVE| | 9.24.105.63| ACTIVE| ------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.89| WEIGHT | ACTIVE % 49 | NEW % 49 | PORT % 2 | SYSTEM % 0 | ---------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD | ---------------------------------------------------------------------------------------------------| 9.24.105.39| 7 | 7 | 5 | 65 | 10 | 17 | 2| 304| 0| 0| | 9.24.105.63| 15 | 15 | 20 | 0 | 11 | 42 | 14| 156| 0| 0| | 9.24.105.59| 5 | 5 | 3 | 83 | 7 | 34 | 13| 31| 0| 0| ---------------------------------------------------------------------------------------------------| PORT TOTALS: | 27 | 27 | | 148 | | 93 | | 491 | | 0 | ----------------------------------------------------------------------------------------------------------------------------------| ADVISOR | PORT | TIMEOUT | -------------------------------| http | 80 | unlimited | --------------------------------

We also monitored the load being balanced through the Monitor tool. To start it, click Port:80 in the left pane of the window, and click Port:80->Monitor... in the context menu (see Figure 5-7).

166

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 5-7 Load balancing monitor

5.3 Load balancing with fixed weights

Weights are values applied to each back-end server to determine how Network Dispatcher will distribute the load between all the servers in a cluster. For example, if one server is set to a weight of 20, and the other server is set to a weight of 10, in every 30 connections, the first server will receive 20 and the second server will receive 10. When you use the Manager, it sets the weight automatically based on information it gets from the Executor, the advisors and ISS. In the previous versions, if you wanted to work with fixed weights, you needed to stop the Manager. In that case, the advisors also are stopped, so the system can no longer determine whether the Web server is responding or not.

Chapter 5. Advanced features

167

Now, you can set a fixed weight for your back-end servers and still have the Manager and the advisors working.

5.3.1 Scenario description

We used a simple load balancing scenario to illustrate this feature (see Figure 5-8).

9.24.105.39 m23kk904 weight:10

Web servers

9.24.105.59 rs60008 weight:20

Cluster: 9.24.105.87 wses1

Network Dispatcher
9.24.105.62 rs600037

Figure 5-8 Fixed weights scenario

We have one Dispatcher load balancing to two back-end servers. We will set one server to a fixed weight of 20, and the other server to a fixed weight of 10.

168

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Table 5-3 shows more information on the machines we used.

Table 5-3 Machines used in the fixed weight scenario Host Name rs600037 rs60008 IP Address 9.24.105.62 9.24.105.59 Operating System and Software AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Service Dispatcher Web server

m23kk904

9.24.105.39

Web server

These machines were all connected to the same Ethernet network.

5.3.2 Configuration
We used the basic configuration described in 4.1, Load balancing on page 70. We created a cluster (9.24.105.87), added port 80 to this cluster, and added two servers to port 80, 9.24.105.39 and 9.24.105.59 (see Figure 5-9).

Chapter 5. Advanced features

169

Figure 5-9 Basic load balancing configuration

To set a fixed weight for the 9.24.105.39 server, click Server:9.24.105.39 in the left pane of the window, and click the Configuration Settings tab in the right pane of the window (see Figure 5-10).

170

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 5-10 Configuring fixed weights

Locate the Fixed weight field, change it to yes and click the Update Configuration button at the bottom of this pane. Now that the server is set to work with a fixed weight, you can change the value of the Server weight field. Locate the Server weight field and type in the value you want for this server. We typed in the value 10 for the 9.24.105.39 server. Click the Update Configuration button to update the configuration.

Chapter 5. Advanced features

171

Important: When setting a server to work with a fixed weight, do it in two steps as we described. You first set Fixed weight to yes, then you must update the configuration before setting the value of the weight you want. We ran tests on our machines and had several problems when setting both fields at the same time. Repeat the same procedure for all servers that will work with fixed weights. In our environment, we followed the same procedure for the second server, 9.24.105.59, and set its weight to 20. You can use the ndcontrol manager report command to check the weight values for all the servers in a cluster (see Example 5-6).
Example 5-6 Checking the weight values for the back-end servers
# ndcontrol manager report ---------------------------------| HOST TABLE LIST | STATUS | ---------------------------------| 9.24.105.39| ACTIVE| | 9.24.105.59| ACTIVE| ------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.87| WEIGHT | ACTIVE % 40 | NEW % 40 | PORT % 20 | SYSTEM % 0 | ---------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD | ---------------------------------------------------------------------------------------------------| 9.24.105.39| 10 | 10 | 10 | 0 | 10 | 0 | 3| 310| 0| 0| | 9.24.105.59| 20 | 20 | 10 | 0 | 10 | 0 | 16| 82| 0| 0| ---------------------------------------------------------------------------------------------------| PORT TOTALS: | 30 | 19 | | 0 | | 0 | | 392 | | 0 | ----------------------------------------------------------------------------------------------------------------------------------| ADVISOR | PORT | TIMEOUT | -------------------------------| http | 80 | unlimited | --------------------------------

172

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Configuration files
Example 5-7 shows the configuration file that you can use to reproduce the configuration described above.
Example 5-7 Fixed weight configuration file ndcontrol executor start ndcontrol executor set nfa 9.24.105.62 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.105.39 ndcontrol server set 9.24.105.87:80:9.24.105.39 fixedweight y ndcontrol server set 9.24.105.87:80:9.24.105.39 weight 10 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol server set 9.24.105.87:80:9.24.105.59 fixedweight y ndcontrol server set 9.24.105.87:80:9.24.105.59 weight 20 ndcontrol cluster configure 9.24.105.87 en0 255.255.255.0 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log

Chapter 5. Advanced features

173

5.3.3 Tests
After we set the fixed weight for both servers, we created traffic to simulate several simultaneous connections to this cluster using the shell script shown in Example 4-8 on page 103. By running this shell script, we were able to create traffic to the cluster address and check the behavior of Network Dispatcher after we set the fixed weights for the back-end servers (see Example 5-8).
Example 5-8 Servers using fixed weights
# ndcontrol manager report ---------------------------------| HOST TABLE LIST | STATUS | ---------------------------------| 9.24.105.39| ACTIVE| | 9.24.105.59| ACTIVE| ------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.87| WEIGHT | ACTIVE % 40 | NEW % 40 | PORT % 20 | SYSTEM % 0 | ---------------------------------------------------------------------------------------------------| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD | ---------------------------------------------------------------------------------------------------| 9.24.105.39| 10 | 10 | 14 | 0 | 10 | 16 | 3| 305| 0| 0| | 9.24.105.59| 20 | 20 | 5 | 1 | 10 | 32 | 16| 67| 0| 0| ---------------------------------------------------------------------------------------------------| PORT TOTALS: | 30 | 19 | | 1 | | 48 | | 372 | | 0 | ----------------------------------------------------------------------------------------------------------------------------------| ADVISOR | PORT | TIMEOUT | -------------------------------| http | 80 | unlimited | --------------------------------

Notice that the Dispatcher balances the load according to the weight we specified: server 9.24.105.59 gets twice the traffic that server 9.24.105.39 gets.

174

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

You can see the same information graphically using the server monitor, as shown in Figure 5-11.

Figure 5-11 Server monitor showing servers using fixed weights

5.4 Rules-based load balancing

You can use rules-based load balancing to fine-tune when and why packets are sent to which servers. Rules-based load balancing is a feature available only for the Dispatcher and Content Based Routing (CBR) components of Network Dispatcher. These components review any rules you add from first priority to last priority, stopping at the first rule that they find to be true; they then balance the load between any servers associated with the rule.

Chapter 5. Advanced features

175

Using rules expands your ability to distribute connections and offers another possible way to manage the load distribution in a cluster. Rules are particularly useful when you want to distribute the load to a subset of your servers for any reason. You must always use rules with the CBR component.

5.4.1 Types of rules

You can use the following types of rules: Client IP address You might want to use rules based on the client IP address if you want to screen the customers and allocate resources based on where they are coming from. For example, you have noticed that your network is getting a lot of unpaid and therefore unwanted traffic from clients coming from a specific set of IP addresses. You can add a rule that instructs the Dispatcher not to serve those requests, or to distribute them among a limited subset of your servers. Time of day You may want to use rules based on the time of day for capacity planning reasons. For example, if your Web site gets accessed most during the same group of hours every day, you might want to dedicate five servers to HTTP full-time, then add another five during the peak time period. Another reason you might use a rule based on the time of day could be, for example, that you want to bring some of the servers down for maintenance every night at midnight. In this case, you would set up a rule that excludes those servers during the necessary maintenance period. Connections per second on a port You might want to use rules based on connections per second on a port if you need to share some of your servers with other applications. For example, you could set two rules: a. If the number of connections per second on port 80 is greater than 100, then the load should be distributed on two specific Web servers. b. If the number of connections per second on port 80 is greater than 2000, then the load should be distributed on 10 specific Web servers.

176

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Or you might be using Telnet and want to reserve two of your five servers for Telnet use, except when the connections per second increase above a certain level. Using this rule, the Dispatcher would balance the load across all five servers at peak times. Note that the Manager must be running for the above scenario to work. Total active connections for a port You might want to use rules based on the total number of active connections on a specific port. This is very useful for when your servers get overloaded and start throwing away packets. In fact, certain Web servers will continue accepting connections even though they do not have enough threads to respond to the requests. As a result, the client requests time out and the customer coming to your Web site is not served. You can set rules based on the total number of active connections to balance capacity within a pool of servers. For example, you know from experience that your servers will stop serving requests after they have accepted 250 connections. In that case, you can create a rule that instructs the Dispatcher to use your current servers, but some additional servers are automatically added when the total number of active connections becomes greater than 250. Those additional servers will otherwise be used for other processing. Note that the Manager must be running for the above scenario to work. This type of rule is available for both the Dispatcher and CBR components. Client port You may want to use rules based on the client port if your clients are running software that uses a specific client port when making requests. For example, you could create a rule that says that any requests with a client port of 10002 will have to use a set of special fast servers because you know that any client requests with that port is coming from an elite group of customers. The client port rule type is available only for the Dispatcher component. Content of a request Request content type rules are used to send requests to sets of servers specifically set up to handle some subset of the traffic of your site. For example, you may want to use one set of servers to handle all CGI requests, another set to handle all streaming audio requests, and a third set to handle all other requests. To do this, you will add one rule with a pattern that matches the path to your CGI directory, another that matches the file type for your streaming audio files, and a third rule, which will always be true, to handle the rest of the traffic. You will then add the appropriate servers to each of the rules.

Chapter 5. Advanced features

177

This rule type is available only for the CBR component. For further information about configuring content type rules, see Chapter 11, Content Based Routing (CBR) on page 443. Type of service Type of service rules allow you to load balance across servers based on the content of the Type of Service field in the IP header. For example, if a client request comes in with one TOS value indicating normal service, it can be routed to one set of servers. If a client request comes in with a different TOS value that indicates a higher priority of service, it can be routed to a different set of servers. This rule type is only available for the Dispatcher component. Capacity and bandwidth rules (in bytes per second) The capacity rules function provides the ability to allocate bandwidth capacity to given servers. This rule is based on the number of bytes served by each server machine. This information will be gathered on a packet-by-packet basis. This rule is configured by setting an upper and a lower boundary. When this rule is tested, if the current traffic load is within the higher and lower boundaries interval, the rule will fire and a server will be chosen from that rules server set. When the traffic exceeds the reserved bandwidth, you can do either of the following: using a rule that is always true, send the traffic to another server that responds with a site busy type of response; share a specific amount of bandwidth at the cluster level or the executor level, using the shared bandwidth rule; when the overall shared bandwidth threshold is neared, using an rule that is always true, you can then direct the traffic to another server that responds with a site busy type of response. Always true A rule can be created that is always true. Such a rule will always be selected, unless all the servers associated with it are down. For this reason, it should ordinarily be at a lower priority than other rules. You can even have multiple always-true rules, each with a set of servers associated with it. The first rule with an available server will be chosen. For example, assume you have six servers. You want two of them to handle your traffic under all circumstances, unless they are both down. If the first two servers are down, you want a second set of servers to handle the traffic. If all four of these servers are down, then you will use the final two servers to

178

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

handle the traffic. To do this, you can set up three always-true rules. The first set of servers will then always be chosen, as long as at least one server is up. If they are both down, one server from the second set will be chosen, and so forth. You can define more than one always-true rule, and thereafter adjust which one gets executed by changing their priority levels. We recommend that you make a plan of the logic that you want Dispatcher to follow before you start adding rules to your configuration.

5.4.2 How rules are evaluated

All rules have a name, type, priority, begin range and end range, and may have a set of servers. Rules are evaluated according to the priority order, with lower priority rules evaluated first. In other words, a rule with a priority of 1 will be evaluated before a rule with a priority of 2. The first rule that is satisfied will be used. Once a rule has been satisfied, no further rules are evaluated. For a rule to be satisfied, it must meet two conditions: 1. The predicate of the rule must be true. That is, the value it is evaluating must be between the begin and end ranges. For always-true rules, the predicate is always satisfied, regardless of the begin and end ranges. 2. If there are servers associated with the rule, at least one of them must be available to which to forward packets. If a rule has no servers associated with it, it needs to meet only the first condition in order to be satisfied. If no rules are satisfied, a server will be selected from the full set of servers available on the port.

Chapter 5. Advanced features

179

5.4.3 Scenario description

To illustrate the use of rules, we created a scenario with Network Dispatcher and three back-end servers, as shown in Figure 5-12.

9.24.105.39 m23kk904

9.24.105.59 rs60008

9.24.105.20 rs600036

Web servers

Cluster: 9.24.105.87 wses1

Network Dispatcher
9.24.105.62 rs600037

Figure 5-12 Rules-based load balancing scenario

We created a rule stating that when the Dispatcher is receiving low traffic for this cluster, it will balance the traffic between two servers: 9.24.105.39 and 9.24.105.59. When the number of connections on this port gets higher than 5 connections per second, it will include a third server, which is 9.24.105.20.

180

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Table 5-4 shows more information on the machines we used.

Table 5-4 Machines used in the rules-based load balancing scenario Host Name rs600037 rs60008 IP Address 9.24.105.62 9.24.105.59 Operating System and Software AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Service Dispatcher Web server

rs600036

9.24.105.20

Web server

m23kk904

9.24.105.39

Web server

These machines were all connected to the same Ethernet network.

5.4.4 Configuration
First, we configured the regular load balancing for these three servers by following the steps in 4.1, Load balancing on page 70. Figure 5-13 shows this configuration.

Chapter 5. Advanced features

181

Figure 5-13 Basic configuration before creating rules

We tested the environment without using any rules to make sure that the three servers were working as expected (see Example 5-9).
Example 5-9 Configuration without rules
# ndcontrol manager report ---------------------------------| HOST TABLE LIST | STATUS | ---------------------------------| 9.24.105.39| ACTIVE| | 9.24.105.59| ACTIVE| | 9.24.105.20| ACTIVE| ------------------------------------------------------------------------------------------------------------------------------------| 9.24.105.87| WEIGHT | ACTIVE % 40 | NEW % 40 | PORT % 20 | SYSTEM % 0 | ----------------------------------------------------------------------------------------------------

182

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

| PORT: 80 | NOW | NEW | WT | CONNECT | WT | CONNECT | WT | LOAD | WT | LOAD | ---------------------------------------------------------------------------------------------------| 9.24.105.39| 8 | 8 | 10 | 0 | 9 | 1 | 2| 309| 0| 0| | 9.24.105.59| 10 | 10 | 10 | 0 | 10 | 1 | 13| 84| 0| 0| | 9.24.105.20| 10 | 10 | 10 | 0 | 10 | 1 | 13| 78| 0| 0| ---------------------------------------------------------------------------------------------------| PORT TOTALS: | 28 | 28 | | 0 | | 3 | | 471 | | 0 | ----------------------------------------------------------------------------------------------------------------------------------| ADVISOR | PORT | TIMEOUT | -------------------------------| http | 80 | unlimited | --------------------------------

Once you confirm that the regular load balancing is working as expected for all back-end servers, you can start adding the rules. Using the GUI, click Port:80 in the left pane of the window, then click Port:80->Add Rule->Total connections (per second) in the context menu. The window shown in Figure 5-14 is displayed.

Figure 5-14 Creating the first rule

Chapter 5. Advanced features

183

We filled in the fields in this window as follows: Rule name The Rule name field can contain any alphanumeric character, underscore, hyphen or period, but it cannot contain any blanks. It can be up to 20 characters long. We chose Conn5 to be the name of this rule. Priority This field is optional. It contains an integer which represents the priority according to which this rule will be evaluated compared to the other rules. The order of evaluation is ascending (from the lowest to the highest). If you do not fill in any value in this field, Network Dispatcher will do it automatically. The first rule you create receives priority 1; the next one receives priority 11 (which is 1 + 10); the next one receives priority 21 (last priority value + 10), and so forth. We filled in the value 10. Begin range This is the lower value in the range that will determine whether or not the rule is true. Depending on the type of rule you are creating, you can use different values for this field: IP address: it is the address of the client. The default is 0.0.0.0. Time: it is an integer, and the default is 0, which represents midnight. Total connections: it is an integer, and the default is 0. Active connections: it is an integer, and the default is 0. Client port: it is an integer, and the default is 0. Reserved bandwidth: it is an integer, and the default is 0.

You do not use this field when creating an always true, shared bandwidth or type of service rule. We filled in the value 5, which means that this rule will be true when the number of connections per second is greater than 5 (and less than the value you specify in the end range field). End range This is the upper value in the range that will determine whether or not the rule is true. Depending on the type of rule you are creating, you can use different values for this field: IP address: it is the address of the client. The default is 255.255.255.255. Time: it is an integer, and the default is 24, which represents midnight. Total connections: it is an integer, and the default is 2 to the 32nd power minus 1. Active connections: it is an integer, and the default is 2 to the 32nd power minus 1.

184

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

 Client port: it is an integer, and the default is 65535. Reserved bandwidth: it is an integer, and the default is 2 to the 32nd power minus 1. You do not specify an end range for an always true, shared bandwidth or type of service (TOS) rule. We left this field blank so that it would work with the default value mentioned above. Level to evaluate This field is valid only for Total connections, Active connections and Shared bandwidth rules. You can choose between evaluating all of the servers on the port or only the servers on the rule. We selected All servers on port. One or more server addresses This is the list of server(s) you have running. You can optionally select one or more from the list to be included with the rule. We selected all servers, because when this rule is evaluated as true, it means that Network Dispatcher is receiving more than 5 connections per second for this port, and in this situation we want to balance the load to all three Web servers. There are a couple of fields that were not shown in Figure 5-14 on page 183 because they are only available for specific rules. They are: TOS (valid only for the Type of service rule) This is an 8-bit entry value consisting of 0, 1, or x. Level to share available bandwidth (valid only for the Shared bandwidth rule) Set the level at which you want bandwidth to be shared. Choose either the cluster or executor level.

Chapter 5. Advanced features

185

After filling in all the fields above, click OK to create the rule. Figure 5-15 shows the configuration including the new rule.

Figure 5-15 Configuration after adding the first rule

Now, we need to create an always true rule. This rule will be used when the previous rule is evaluated false, that means, when the number of connections per second is less than 5. In this case, we only want the Dispatcher to use two servers, 9.24.105.39 and 9.24.105.59.

186

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

To add a new rule, click Port:80 in the left pane of the window, then click Port:80->Add rule->Always true in the context menu. The window shown in Figure 5-16 will be displayed.

Figure 5-16 Adding an always true rule

We filled in the fields in this window as follows: Rule name: LowConn. Priority: 20 (we want this rule to be evaluated after the first one). One or more server addresses: we selected the servers 9.24.105.59 and 9.24.105.39.

Chapter 5. Advanced features

187

See Figure 5-17 for the configuration after adding the second rule.

Figure 5-17 Configuration after adding the always true rule

188

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Configuration file
You can use the commands in Example 5-10 to create the environment described above.
Example 5-10 Ruled-based load balancing configuration
ndcontrol executor start ndcontrol executor set nfa 9.24.105.62 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.105.39 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol server add 9.24.105.87:80:9.24.105.20 ndcontrol ndcontrol ndcontrol ndcontrol rule rule rule rule add 9.24.105.87:80:Conn5 type connection priority 10 beginrange 5 endrange 2147483647 useserver 9.24.105.87:80:Conn5 9.24.105.59 useserver 9.24.105.87:80:Conn5 9.24.105.20 useserver 9.24.105.87:80:Conn5 9.24.105.39

ndcontrol rule add 9.24.105.87:80:LowConn type true priority 20 ndcontrol rule useserver 9.24.105.87:80:LowConn 9.24.105.59 ndcontrol rule useserver 9.24.105.87:80:LowConn 9.24.105.39 ndcontrol cluster configure 9.24.105.87 en0 255.255.255.0 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log

5.4.5 Tests
We began our test by creating a low number of connections, to make sure that only the two servers were going to be used for balancing. Figure 5-18 shows that only the servers 9.34.105.39 and 9.24.105.59 are being used for load balancing. Note that there is always one packet being sent to our third server, 9.24.105.20. This packet is sent by the Manager to collect the advisor information. Although this server is not currently being used for load balancing, the Dispatcher still needs to collect information from it.

Chapter 5. Advanced features

189

Figure 5-18 Rules-based load balancing to two servers

Our previous test showed that the first part of our configuration is working. Next, we need to increase the number of connections to fire the rule and start using the third server. We again used a shell script to send a high number of HTTP requests (see Example 4-8 on page 103). Figure 5-19 shows the traffic after we increased the number of HTTP requests. Note that at the beginning of the graphic, when the load was still low, the 9.24.105.20 server was not being used. Then the load increased and this server started being used for load balancing (after time T-8 on the graphic).

190

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 5-19 Rules-based load balancing to three servers

The third server will only be used when the number of connections is higher than 5. If we decrease the load, it will become idle again.

5.5 Wide area Network Dispatcher support

Network Dispatcher provides a wide area network (WAN) Dispatcher enhancement that offers support for remote servers. A remote server consists of a remote Dispatcher machine and its locally attached servers. A clients packet can now go from the Internet to a Dispatcher machine, from there to a geographically remote Dispatcher machine, then to one of its locally attached servers, and from the server directly back to the Internet and the client. See an example in Figure 5-20.

Chapter 5. Advanced features

191

Network Dispatcher

Network Dispatchers

Back end Web servers

Figure 5-20 Example of layout for a wide area Network Dispatcher

This allows one cluster address to support all worldwide client requests while distributing the load to servers around the world. The Dispatcher machine initially receiving the packet can still have local servers attached to it and can distribute the load among its local servers as well as the remote servers. Note that wide area support is currently available only for the Dispatcher component of Network Dispatcher.

5.5.1 Scenario description

We created a simple scenario to illustrate this feature. We have one Network Dispatcher machine, balancing the load to two local Web servers and two remote Web servers. To be able to load balance to remote Web servers, we need to configure another Network Dispatcher machine on the same network as the remote Web servers (see Figure 5-21).

192

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

9.24.105.39 m23kk904

9.24.105.59 rs60008

9.24.105.62 rs600037

Local Web servers

Local Network Dispatcher

Cluster: 9.24.105.87 wses1

9.24.105.14

R
192.168.10.1

Remote Web servers

192.168.10.6 192.168.10.7 was01 was02
Figure 5-21 Wide area Network Dispatcher scenario

Remote Network Dispatcher

192.168.10.14 rs600010

When a client sends an HTTP request, it is received by the local Network Dispatcher, 9.24.105.62. This Dispatcher will load balance between two local Web servers, 9.24.105.39 and 9.24.105.59, and two remote Web servers, 192.168.10.6 and 192.168.10.7. The load balancing of the two remote Web servers will actually be handled by the remote Network Dispatcher, 192.168.10.14. The following table shows more information on the machines we used for this scenario.
Table 5-5 Machines used in the wide area network scenario Host Name rs600037 rs60008 IP Address 9.24.105.62 9.24.105.59 Operating System and Software AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Service Local Dispatcher Local Web server

Chapter 5. Advanced features

193

Host Name m23kk904

IP Address 9.24.105.39

Operating System and Software Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 AIX 4.3.3 WebSphere Edge Server 1.0.3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 IBM Firewall for NT V4.1

Service Local Web server Remote Dispatcher Remote Web server Remote Web server Router between networks

rs600010 was01

192.168.10.14 192.168.10.6

was02

192.168.10.7

fw01

9.24.105.14 192.168.10.1

5.5.2 Configuration
First, we configured the local Dispatcher (9.24.105.62) based on the configuration described in 4.1, Load balancing on page 70. We added only the two local Web servers, as shown in Figure 5-22.

194

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 5-22 Basic configuration of the first Dispatcher

The next thing we did was to set the wide area port number. This port will be used by the local and the remote Dispatchers to communicate. You can select any port number, as long as it is not currently being used in any of the Dispatchers. You can check that by using the command netstat -an. We selected the port number 23456. To add the port number, click Executor:9.24.105.62 in the left pane of the window, click the Configuration Settings tab in the right pane and locate the field Wide port number (see Figure 5-23).

Chapter 5. Advanced features

195

Figure 5-23 Setting the Wide port number field

Fill in the port number you selected, then click the Update Configuration button at the bottom of the window to activate the change. We typed in the port number 23456. In this configuration, we will not add the remote servers directly to the cluster on this Dispatcher. We will add the remote Dispatcher as if it were a server, and both Dispatchers will exchange information about the requests that need to be sent to the remote Web servers.

196

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

To include the remote Dispatcher, click Port:80 in the left pane of the window, then click Port:80->Add server..., and fill in the Server address field with the non-forwarding address of the remote Dispatcher; select the Network router address checkbox, and fill in the IP address of the router that you use to get to the remote network (see Figure 5-24).

Figure 5-24 Adding the remote Dispatcher as a server

The next step is to configure the remote Dispatcher. Follow the steps described in 4.1, Load balancing on page 70, but take note of a few changes: When adding the cluster, use the same cluster IP address that you used for the local Dispatcher, which is 9.24.105.87 (see Figure 5-25).

Figure 5-25 Adding the cluster on the second Dispatcher

The primary host for this cluster is the local Dispatcher we configured (the one that will receive the requests from the Web browsers). In our scenario, the primary host for this cluster is the Dispatcher 9.24.105.62.

Chapter 5. Advanced features

197

Note that this cluster must not be configured, so do not select the Configure this cluster? checkbox. After you have added the cluster, proceed with adding the port you will use (in our scenario, we are using port 80) and the back-end servers. In this scenario, the back-end servers are the servers 192.168.10.6 and 192.168.10.7. See Figure 5-26 for the complete configuration of the remote Dispatcher.

Figure 5-26 Configuration of the remote Dispatcher

198

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The next step is to set the wide area port number for this remote Dispatcher the same way we did for the local Dispatcher. To add the port number, click Executor:192.168.10.14 in the left pane of the window, and click the Configuration Settings tab in the right pane. Locate the Wide port number field and fill in the same number that you used for the local Dispatcher. Since we used 23456 on the local Dispatcher, we also set this port number on the remote Dispatcher (see Figure 5-27).

Figure 5-27 Setting the Wide port number on the second Dispatcher

Working with advisors

Note that we started the Manager and the advisors on both Dispatchers. The first Dispatcher will work with advisors without any further configuration, but the second Dispatcher (and all remote Dispatchers) will not, since it does not have the IP alias on its network interface.

Chapter 5. Advanced features

199

The remote Dispatcher cannot have an IP alias on its network interface (the IP alias can only be added to the local Dispatcher), so we added the IP alias to the loopback adapter. Refer to Configuration of the back-end servers on page 95 for more information about adding an IP alias. Since we are using AIX, we ran the following command to add an IP alias to the loopback adapter:
# ifconfig lo0 alias 9.24.105.87 netmask 255.255.255.0

This command adds an IP alias to the loopback adapter, but it also adds a route to the whole 9.24.105.0 network using the local IP alias as a gateway. This route disables all communication between this machine and the 9.24.105.0 network, so we must remove it, using the following command:
# route delete 9.24.105.0 9.24.105.87

In its place, we must add a host route with destination to the cluster IP address (9.24.105.87) using the loopback as gateway. We used the following command:
# route add -host 9.24.105.87 127.0.0.1

You should also add these commands to the /etc/rc.net file so that they will be run every time the machine is rebooted, or every time the network is reconfigured. Edit the /etc/rc.net file and add the following lines at the bottom of the file:
ifconfig lo0 alias 9.24.105.87 netmask 255.255.255.0 route delete 9.24.105.0 9.24.105.87 route add -host 9.24.105.87 127.0.0.1

200

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Now all advisors will be able to detect whether or not the local and remote Web servers are available. Figure 5-28 shows the advisor status on the local Dispatcher. Note that it gets information from all back-end servers, including the remote ones (it shows the IP address of the remote Dispatcher).

Figure 5-28 Advisor on the local Dispatcher

Chapter 5. Advanced features

201

Figure 5-29 shows the advisor status on the remote Dispatcher. Note that this screen shows the information for each remote Web server individually.

Figure 5-29 Advisor on the remote Dispatcher

Firewall issues
If you are using a firewall or a router with Access Control List, you need to create filters to allow communication between the Dispatchers.

202

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 5-30 shows the communication that we identified between the Dispatchers in our scenario. Use this as a reference to create the filters on your firewall.

9.24.105.62 rs600037

192.168.10.14 rs600010

R
Local Network Dispatcher
Protocol 47 Echo request (ping) Echo reply (ping)
Figure 5-30 Communication between the two Dispatchers

Remote Network Dispatcher

Configuration files
In this section, we show you the configuration files for both Dispatchers.

Chapter 5. Advanced features

203

Example 5-11 shows the configuration file for the local Dispatcher, which is the one that will receive all requests from the clients. In our scenario, the local Dispatcher is 9.24.105.62.
Example 5-11 Configuration file for the local Dispatcher ndcontrol executor start ndcontrol executor set nfa 9.24.105.62 ndcontrol executor set wideportnumber 23456 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.105.39 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol server add 9.24.105.87:80:192.168.10.14 router 9.24.105.14 ndcontrol cluster configure 9.24.105.87 en0 255.255.255.0 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log

Example 5-12 shows the configuration file for the remote Dispatcher, which is the one located in the remote network. You can use the same commands for all your remote Dispatchers (in case you are working with more than one remote network).
Example 5-12 Configuration file for the remote Dispatcher ndcontrol executor start ndcontrol executor set nfa 192.168.10.14 ndcontrol executor set wideportnumber 23456 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol cluster set 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:192.168.10.7 ndcontrol server add 9.24.105.87:80:192.168.10.6 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log

204

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

5.5.3 Tests
We used the script shown in Example 4-8 on page 103 to generate traffic for this test. We sent several HTTP requests to 9.24.105.87, which is the IP address of the cluster. The requests were received by the local Dispatcher, 9.24.105.62, which balanced them between the two local Web servers and the remote Web servers, managed by the remote Dispatcher, 192.168.10.14. Figure 5-31 shows the server monitor on the local Dispatcher, and how the load is being balanced among the servers. Note that the graphic does not show the two remote Web servers, it only shows the IP address of the remote Dispatcher.

Figure 5-31 Server monitor on the local Dispatcher

Chapter 5. Advanced features

205

If you want to monitor the load balancing on the remote Web servers, you need to run the server monitor on the remote Dispatcher. Figure 5-32 shows the server monitor running on the remote Dispatcher. Note that it now shows the load on the remote Web servers individually.

Figure 5-32 Server monitor on the remote Dispatcher

5.6 Mutual high availability

The mutual high availability feature of Network Dispatcher provides the administrator the opportunity to use both Dispatcher machines in a high availability configuration, the primary and the backup machines, to work simultaneously to balance the load for different clusters while also providing backup for each other. In case one server fails, the other one will balance the load for the clusters assigned to it, and also the clusters assigned to the failed Dispatcher.

206

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

5.6.1 Scenario description

We created a scenario based on the scenario we described in 4.2, High availability on page 104. We assigned a new cluster, wses2.itso.ral.ibm.com, to the backup machine (rs600010), as shown in Figure 5-33, and two Web servers to this new cluster.

Web servers
9.24.105.39 9.24.105.59 rs60008 m23kk904 9.24.105.63 rhlinux1 9.24.105.20 rs600036

Port 80 cluster: 9.24.105.87 wses1 Primary: rs600037 Backup: rs600010

9.24.105.62 rs600037

9.24.105.60 rs600010

Network Dispatcher

Port 80 cluster: 9.24.105.88 wses2 Primary: rs600010 Backup: rs600037

Figure 5-33 Mutual high availability scenario

Chapter 5. Advanced features

207

The table below describes the machines we used in this scenario.

Table 5-6 Machines used in the high availability scenario Host name rs600037 rs600010 rs60008 IP Address 9.24.105.62 9.24.105.60 9.24.105.59 Operating System and Software AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Windows NT 4.0 WebSphere Application Server V3.5 with PTF3 Red Hat Linux V6.2 Apache HTTP Server AIX 4.3.3 WebSphere Application Server V3.5 with PTF3 Service Primary Dispatcher for cluster wses1 Primary Dispatcher for cluster wses2 Web server for cluster wses1 Web server for cluster wses1 Web server for cluster wses2 Web server for cluster wses2

m23kk904

9.24.105.39

rhlinux1 rs600036

9.24.105.63 9.24.105.20

5.6.2 Configuration
Configuring mutual high availability is similar to configuring regular high availability. The only difference is that when adding the clusters, we need to inform each server as to which primary machine will handle each cluster. This will allow us to keep some clusters active on one Dispatcher, and other clusters active on the other Dispatcher, each Dispatcher being the backup of the other.

Configuring the clusters on the Dispatchers

We created the load balancing configuration on both Dispatcher servers independently. We used a remote configuration client so we could work on both machines at the same time. First, we configured the 9.24.105.62 server (see 4.1, Load balancing on page 70). We added the 9.24.105.87 cluster with two servers, 9.24.105.39 and 9.24.105.59.

208

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Next, we configured the 9.24.105.60 server. We added the 9.24.105.88 cluster and two servers, 9.24.105.63 and 9.24.105.20. We also started the Manager and the HTTP advisor on both Dispatchers. The basic configuration of both Dispatchers is shown in Figure 5-34.

Figure 5-34 Basic configuration of both Dispatchers

Once we have both Dispatchers working for each cluster, we can start configuring high availability on each of them. First, we remove the IP alias that we had created on each Dispatcher for its corresponding cluster. Since we created the IP alias using the GUI, we also remove it using the GUI. Locate the cluster that you configured, and click Cluster:[IP address]->Unconfigure Cluster Address.

Chapter 5. Advanced features

209

Make sure that all IP aliases are removed before continuing. Refer to Checking the IP aliases on the Dispatcher on page 90 for more information on how to check the IP aliases. The next thing you need to do is to create the configuration of the clusters on each backup machine. In our environment, we first add the 9.24.105.88 cluster to the first Dispatcher (24.105.62). Note that when creating this cluster on 9.24.105.62, we informed the 9.24.105.62 Dispatcher that the primary machine for this cluster is the other Dispatcher, 9.24.105.60 (see Figure 5-35).

Figure 5-35 Configuring the 9.24.105.88 cluster on the 9.24.105.62 Dispatcher

Note that you must not select the Configure this cluster? checkbox. We will configure all IP aliases later using the high availability scripts.

210

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Now, continue the configuration of this cluster by adding the port and the servers as you did on the second Dispatcher. See Figure 5-36 for the complete configuration of this new cluster on the 9.24.105.62 Dispatcher.

Figure 5-36 New cluster on the Dispatcher 9.24.105.62

Now you must do the same thing on the second Dispatcher (9.24.105.60). You must add the cluster for which it will act as a backup. In our scenario, this is the 9.24.105.87 cluster.

Chapter 5. Advanced features

211

When you add the cluster, remember to use 9.24.105.62 as the primary machine for this cluster, as shown in Figure 5-37.

Figure 5-37 Adding the cluster 9.24.105.87 to the Dispatcher 9.24.105.60

212

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Add the port and the servers to this cluster. After you finish, compare this to Figure 5-38, which shows the configuration of the clusters on the Dispatcher 9.24.105.60.

Figure 5-38 Configuration of the clusters on the second Dispatcher

Once both Dispatchers are configured with all the clusters, we can work on the high availability scripts.

Creating the high availability scripts

The high availability scripts work similarly in mutual high availability (see Creating the high availability scripts on page 120 for more information about these scripts). You need to create the goActive, goStandby and goInOp on both machines, and place them in the directory <install path>/dispatcher/bin.

Chapter 5. Advanced features

213

The main difference with the mutual high availability scripts is that Network Dispatcher calls them with a parameter identifying the primary machine IP address. The scripts must query this parameter and perform the ifconfig and ndconfig commands for the cluster IP addresses associated with that primary machine. First, we created the scripts on the 9.24.105.62 Dispatcher. This Dispatcher is primary for the 9.24.105.87 cluster and backup for the 9.24.105.88 cluster (see Figure 5-33 on page 207). The first script, goActive, must add the IP aliases of the active clusters to the network interfaces. This script checks the parameter that is passed by Network Dispatcher, and it will add the IP aliases of the corresponding clusters (se)e Example 5-13.
Example 5-13 goActive script on the 9.24.105.62 Dispatcher #!/bin/ksh # goActive script # LOCAL_NFA=9.24.105.62 REM_NFA=9.24.105.60 CLUSTER1=9.24.105.87 CLUSTER2=9.24.105.88 NETMASK1=255.255.255.0 NETMASK2=255.255.255.0 INTERFACE=en0 if [[ "$1" = $LOCAL_NFA ]]; then # Activating local cluster ifconfig lo0 delete $CLUSTER1 ifconfig $INTERFACE alias $CLUSTER1 netmask $NETMASK1 fi if [[ "$1" = $REM_NFA ]]; then # Activating remote cluster ifconfig lo0 delete $CLUSTER2 ifconfig $INTERFACE alias $CLUSTER2 netmask $NETMASK2 fi

We defined two variables, LOCAL_NFA, which represents the local IP address of the Dispatcher, and REM_NFA, which represents the IP address of the second Dispatcher. Note that both variables will change on the second Dispatcher. CLUSTER1 is the local cluster, and CLUSTER2 is the remote cluster. NETMASK1 is the netmask for CLUSTER1, and NETMASK2 is the netmask for CLUSTER2.

214

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The if clauses check which parameter was passed (the local or the remote IP address), and will add the appropriate IP aliases. The next script, goStandby, uses the same variables and displays the same behavior, but this script will remove the IP aliases from the network interfaces and add them to the loopback interface (see Example 5-14).
Example 5-14 goStandby script on the 9.24.105.62 Dispatcher #!/bin/ksh # goStandby script # LOCAL_NFA=9.24.105.62 REM_NFA=9.24.105.60 CLUSTER1=9.24.105.87 CLUSTER2=9.24.105.88 NETMASK1=255.255.255.0 NETMASK2=255.255.255.0 INTERFACE=en0 if [[ "$1" = $LOCAL_NFA ]]; then # Deactivating local cluster ifconfig $INTERFACE delete $CLUSTER1 ifconfig lo0 alias $CLUSTER1 netmask $NETMASK1 fi if [[ "$1" = $REM_NFA ]]; then # Deactivating remote cluster ifconfig $INTERFACE delete $CLUSTER2 ifconfig lo0 alias $CLUSTER2 netmask $NETMASK2 fi

Note that on goStandby, the variables remain the same. The only difference is in the ifconfig commands.

Chapter 5. Advanced features

215

The last script to be created on this server is goInOp. This script will remove all IP aliases that were added to the local network interface and to the loopback interface, so we do not need to check the parameter this time (see Example 5-15).
Example 5-15 goInOp script on the 9.24.105.62 Dispatcher #!/bin/ksh # goInOp script # CLUSTER1=9.24.105.87 CLUSTER2=9.24.105.88 NETMASK1=255.255.255.0 NETMASK2=255.255.255.0 INTERFACE=en0 # Deactivating local cluster ifconfig $INTERFACE delete $CLUSTER1 ifconfig lo0 delete $CLUSTER1 # Deactivating remote cluster ifconfig $INTERFACE delete $CLUSTER2 ifconfig lo0 delete $CLUSTER2

Now that the scripts are all ready on the first Dispatcher, we need to do the same configuration on the second Dispatcher, 9.24.105.60. The scripts are similar, except for the IP addresses that we associate to the variables.

216

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The first script, goActive, works exactly the same way as it did on the 9.24.105.62 Dispatcher, but we had to change the IP addresses of the variables (see Example 5-16).
Example 5-16 goActive script on the 9.24.105.60 Dispatcher #!/bin/ksh # goActive script # LOCAL_NFA=9.24.105.60 REM_NFA=9.24.105.62 CLUSTER1=9.24.105.88 CLUSTER2=9.24.105.87 NETMASK1=255.255.255.0 NETMASK2=255.255.255.0 INTERFACE=en0 if [[ "$1" = $LOCAL_NFA ]]; then # Activating local cluster ifconfig lo0 delete $CLUSTER1 ifconfig $INTERFACE alias $CLUSTER1 netmask $NETMASK1 fi if [[ "$1" = $REM_NFA ]]; then # Activating remote cluster ifconfig lo0 delete $CLUSTER2 ifconfig $INTERFACE alias $CLUSTER2 netmask $NETMASK2 fi

We associated LOCAL_NFA to the 9.24.105.60 IP address and REM_NFA to the 9.24.105.62 IP address. We also changed CLUSTER1 to 9.24.105.88 (which is the local cluster) and CLUSTER2 to 9.24.105.87 (which is the remote cluster). In our scenario, the netmasks are the same, so check if you need to change them also.

Chapter 5. Advanced features

217

The next script, goStandby, is shown in Example 5-17.

Example 5-17 goStandby script on the 9.24.105.60 Dispatcher #!/bin/ksh # goStandby script # LOCAL_NFA=9.24.105.60 REM_NFA=9.24.105.62 CLUSTER1=9.24.105.88 CLUSTER2=9.24.105.87 NETMASK1=255.255.255.0 NETMASK2=255.255.255.0 INTERFACE=en0 if [[ "$1" = $LOCAL_NFA ]]; then # Deactivating local cluster ifconfig $INTERFACE delete $CLUSTER1 ifconfig lo0 alias $CLUSTER1 netmask $NETMASK1 fi if [[ "$1" = $REM_NFA ]]; then # Deactivating remote cluster ifconfig $INTERFACE delete $CLUSTER2 ifconfig lo0 alias $CLUSTER2 netmask $NETMASK2 fi

Note that the changes we made to goActive were also made to this script.

218

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The last script, goInOp, is shown in Example 5-18.

Example 5-18 goInOp script on the 9.24.105.60 Dispatcher #!/bin/ksh # goInOp script # CLUSTER1=9.24.105.88 CLUSTER2=9.24.105.87 NETMASK1=255.255.255.0 NETMASK2=255.255.255.0 INTERFACE=en0 # Deactivating local cluster ifconfig $INTERFACE delete $CLUSTER1 ifconfig lo0 delete $CLUSTER1 # Deactivating remote cluster ifconfig $INTERFACE delete $CLUSTER2 ifconfig lo0 delete $CLUSTER2

Important: If you are working in a UNIX environment, make sure that the high availability scripts have execute permission. If they do not, run the following command inside the <install path>/dispatcher/bin directory of each Dispatcher: # chmod 700 go*

Chapter 5. Advanced features

219

Configuring mutual high availability

Now that all scripts are ready, you can start adding the high availability configuration to each Dispatcher. In our scenario, we first added the heartbeat on the first Dispatcher, which is 9.24.105.62. Click Executor:9.24.105.62, then click Executor:9.24.105.62->Add Heartbeat in the context menu. The heartbeat is configured using the local IP address (9.24.105.62) as source and the second Dispatcher IP address (9.24.105.60) as destination (see Figure 5-39).

Figure 5-39 Adding the heartbeat on the first Dispatcher

Do the same thing on the second Dispatcher, using the local IP address (9.24.105.60) as source and the remote IP address (9.24.105.62) as destination (see Figure 5-40).

Figure 5-40 Adding the heartbeat on the second Dispatcher

Now add the reach target to both Dispatchers. We chose the reach target in our scenario to be our default gateway router (9.24.105.1).

220

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Click Executor:9.24.105.62, then click Executor:9.24.105.62->Add Reach Target in the context menu, and fill in the IP address you have chosen, as shown in Figure 5-41.

Figure 5-41 Adding the reach target

To add the reach target on the second Dispatcher, click Executor:9.24.105.60, then click Executor:9.24.105.60->Add Reach Target in the context menu, and fill in the same IP address you used on the first Dispatcher. Finally, add the backup information to each Dispatcher as shown below. On the first Dispatcher, click Executor:9.24.105.62, then click Executor:9.24.105.62->Add High Availability Backup in the context menu. The window shown in Figure 5-42 is displayed.

Figure 5-42 Adding backup information

Choose a port number that is not in use in any of the Network Dispatcher machines, and enter it in the Port number field. In the Role field, select Both for mutual high availability. In the Recovery Strategy field, we selected Auto. Refer to 4.3.2, Configuration on page 129 for more details about these fields.

Chapter 5. Advanced features

221

On the second Dispatcher, click Executor:9.24.105.60, then click Executor:9.24.105.60->Add High Availability Backup in the context menu, and add the same information you did for the first server (see Figure 5-42). As soon as you add the backup information, the high availability scripts are also run. You can check the high availability status on the first Dispatcher by clicking Executor:9.24.105.62, and clicking the High Availability tab in the right pane of the window, as shown in Figure 5-43.

Figure 5-43 High availability configuration on the first Dispatcher

222

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Click Refresh Statistics to see the current information. Note that this Dispatcher is in Active state for the 9.24.105.87 cluster and in Standby state for the 9.24.105.88 cluster. You can also see this information by running the command ndcontrol high status, as shown in Example 5-19.
Example 5-19 Status of high availability for the first Dispatcher High Availability Status: ------------------------Role ................. Primary Recovery strategy .... Auto State ................ Active Sub-state ............ Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... 9.24.105.60 High Availability Status: ------------------------Role ................. Backup Recovery strategy .... Auto State ................ Standby Sub-state ............ Synchronized Primary host ......... 9.24.105.60 Port ................. 12345 Preferred target ..... 9.24.105.60 Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.62/9.24.105.60 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

Chapter 5. Advanced features

223

Make sure that the IP aliases were created by the scripts. Use the command ifconfig -a, as shown in Example 5-20.
Example 5-20 IP aliases on the first Dispatcher # ifconfig -a lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 inet 9.24.105.88 netmask 0xffffff00 broadcast 9.24.105.255 en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT, PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255

Note that the local cluster IP alias (9.24.105.87) is in the local network interface, en0, while the remote cluster IP alias (9.24.105.88) is in the loopback interface.

224

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Now check the configuration on the second Dispatcher by clicking Executor:9.24.105.60, then clicking the High Availability tab in the right pane of the window, as shown in Figure 5-44.

Figure 5-44 High availability configuration on the second server

Click Refresh Statistics to see the current information.

Chapter 5. Advanced features

225

Note that this Dispatcher is in Active state for the 9.24.105.88 cluster and in Standby state for the 9.24.105.87 cluster. You can also see this information by running the command ndcontrol high status, as shown in Example 5-21.
Example 5-21 Status of high availability for the second Dispatcher High Availability Status: ------------------------Role ................. Primary Recovery strategy .... Auto State ................ Active Sub-state ............ Synchronized Primary host ......... 9.24.105.60 Port ................. 12345 Preferred target ..... 9.24.105.62 High Availability Status: ------------------------Role ................. Backup Recovery strategy .... Auto State ................ Standby Sub-state ............ Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... 9.24.105.62 Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.60/9.24.105.62 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

226

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Make sure that the IP aliases were also created by the scripts on this machine, as shown in Example 5-22.
Example 5-22 IP aliases on the second Dispatcher # ifconfig -a lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT > inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255 en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64 BIT,PSEG> inet 9.24.105.60 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.88 netmask 0xffffff00 broadcast 9.24.105.255

Note that the local cluster IP alias (9.24.105.88) is in the local network interface, en0, while the remote cluster IP alias (9.24.105.87) is in the loopback interface. Once this configuration is finished, if either of the Dispatchers fails to respond to the network, the other Dispatcher will take over all clusters.

Chapter 5. Advanced features

227

Configuration files
After we saved the configuration on both servers, the following files were generated. The configuration file generated on the first Dispatcher, 9.24.105.62, is shown in Example 5-23:
Example 5-23 Configuration file of the first Dispatcher ndcontrol executor start ndcontrol executor set nfa 9.24.105.62 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.105.39 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol cluster add 9.24.107.88 primaryhost 9.24.105.60 ndcontrol cluster set 9.24.107.88 primaryhost 9.24.105.60 ndcontrol port add 9.24.107.88:80 ndcontrol server add 9.24.107.88:80:9.24.105.63 ndcontrol server add 9.24.107.88:80:9.24.105.20 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log ndcontrol highavailability heartbeat add 9.24.105.62 9.24.105.60 ndcontrol highavailability backup add both auto 12345 ndcontrol highavailability reach add 9.24.105.1

228

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The configuration file generated on the second Dispatcher, 9.24.105.60, is shown in Example 5-24:
Example 5-24 Configuration file of the second Dispatcher ndcontrol executor start ndcontrol executor set nfa 9.24.105.60 ndcontrol cluster add 9.24.105.88 primaryhost 9.24.105.60 ndcontrol port add 9.24.105.88:80 ndcontrol server add 9.24.105.88:80:9.24.105.63 ndcontrol server add 9.24.105.88:80:9.24.105.20 ndcontrol cluster add 9.24.105.87 primaryhost 9.24.105.62 ndcontrol cluster set 9.24.105.87 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.87:80 ndcontrol server add 9.24.105.87:80:9.24.105.39 ndcontrol server add 9.24.105.87:80:9.24.105.59 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start Http 80 Http_80.log ndcontrol highavailability heartbeat add 9.24.105.60 9.24.105.62 ndcontrol highavailability backup add both auto 12345 ndcontrol highavailability reach add 9.24.105.1

5.6.3 Tests
Refer to 4.2.3, Tests on page 123 for more information on testing a high availability environment. Since we are using the auto recovery strategy, we tested the environment by simulating the failure of one of the servers. We stopped the Ethernet interface on the second Dispatcher (9.24.105.60) using the command below. # ifconfig en0 down This forces the first Dispatcher to take over the 9.24.105.88 cluster.

Chapter 5. Advanced features

229

After the first Dispatcher stops receiving the heartbeat from the second Dispatcher, it takes over all clusters, as shown in Example 5-25.
Example 5-25 High availability status after the failure of the second Dispatcher # ndcontrol high status High Availability Status: ------------------------Role ................. Primary Recovery strategy .... Auto State ................ Active Sub-state ............ Not Synchronized Primary host ......... 9.24.105.62 Port ................. 12345 Preferred target ..... n/a High Availability Status: ------------------------Role ................. Backup Recovery strategy .... Auto State ................ Active Sub-state ............ Not Synchronized Primary host ......... 9.24.105.60 Port ................. 12345 Preferred target ..... n/a Heartbeat Status: ----------------Count ................ 1 Source/destination ... 9.24.105.62/9.24.105.60 Reachability Status: -------------------Count ................ 1 Address .............. 9.24.105.1 reachable

230

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The first Dispatcher also adds the IP alias of the 9.24.105.88 cluster to its local network interface, since it runs the goActive script passing as parameter the IP address of the failed Dispatcher, as shown in Example 5-26.
Example 5-26 ifconfig after the failure of the second Dispatcher # ifconfig -a lo0: flags=e08084b<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT> inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255 inet6 ::1/0 en0: flags=4e080863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT, PSEG> inet 9.24.105.62 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.87 netmask 0xffffff00 broadcast 9.24.105.255 inet 9.24.105.88 netmask 0xffffff00 broadcast 9.24.105.255

After we brought the second Dispatcher back up, it took over its cluster (remember that we used auto recovery) and the first Dispatcher was once again in standby mode for this cluster.

5.7 Affinity
The affinity feature is enabled by configuring a clusters port to be sticky. When you set a port to be sticky, all subsequent requests sent to it will be forwarded to the same back-end server. You can enable the classical affinity by setting the port stickytime to the amount of time (in seconds) you want Network Dispatcher to keep all connections coming from the same IP address to be forwarded to the same back-end server. You can disable this feature by setting the port stickytime to zero. When Network Dispatcher is not using affinity, whenever a new TCP connection is received from a client, Network Dispatcher picks the right back-end server at that moment and forwards the packet to it. If the same client opens a new connection to the same cluster, it is treated as an unrelated connection, and Network Dispatcher will again analyze it and pick the right back-end server at that time.

Chapter 5. Advanced features

231

With affinity enabled, Network Dispatcher will forward the new connection to the same back-end server as the previous connection. Over time, the client will stop sending packets, and the affinity record will be erased. Each affinity record will be kept for the duration of the stickytime (in seconds). The new connections received within the stickytime are still forwarded to the same back-end server. If a new connection is received after that time, the affinity record is purged and a new back-end server is selected for it. Enhancements have been made to the classical affinity. We discuss them in the next sections. Note: Affinity is based on the IP address of the client. If the client is using a proxy or SOCKS server, all connections from the same network will use the same source IP address. If you are using affinity, all clients from those networks will get sticky to the same server on your cluster. Consider this when configuring your environment.

5.7.1 Server Directed Affinity API to control client-server affinity

The Server Directed Affinity API only applies to the Dispatcher component. The SDA feature provides an API that allows an external agent to influence the Dispatcher affinity behavior.

SDA features
Your application may have indicated that its server systems have the knowledge to direct client requests to particular server machines better than the Dispatcher can. Rather than having a client forwarded to the same server selected by Dispatchers load balancing, you may want the client to be forwarded to the server of your choice. The SDA feature provides this API. You can now write your own software to implement an SDA agent, which communicates with a listener in Dispatcher. It can then manipulate the Dispatcher affinity tables to: Query the contents Insert new records Remove records Records inserted in an affinity table by an SDA agent remain in the table indefinitely. They do not time out. They are removed only when the SDA agent removes them or if a Dispatcher advisor detects that the server is dead.

232

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Dispatchers SDA components

The Dispatcher implements a new socket listener to accept and handle requests from an SDA agent. When an SDA agent opens a connection with the Dispatcher, the listener will accept it and leave the connection open. Multiple requests and responses can flow over this persistent connection. The socket will close when the SDA agent closes it or if the Dispatcher detects an unrecoverable error. Inside the Dispatcher, the listener takes each request from the SDA agent, communicates with the appropriate affinity table in the Dispatcher executor kernel, and prepares a response for the SDA agent. For more information, refer to the following files after the Dispatcher has been installed. the API: samples/SDA/SDA_ API.htm the sample code for an SDA agent: samples/SDA/SDA_SampleAgent.java The files can be found in the Dispatchers install directory tree.

5.7.2 Cross port affinity

Cross port affinity only applies to the Dispatcher component. Cross port affinity is the sticky feature that has been expanded to cover multiple ports. For example, if a client request is first received on one port and the next request is received on another port, cross port affinity allows the dispatcher to send the client request to the same server. In order to use this feature, the ports must: share the same cluster address share the same servers have the same stickytime value (not zero) have the same stickymask value More than one port can link to the same cross port. When subsequent connections come in from the same client on the same port or a shared port, the same server will be accessed. The following is an example of the configuration of multiple ports with a cross port affinity to port 80:
# ndcontrol port set cluster:20 crossport 80 # ndcontrol port set cluster:30 crossport 80 # ndcontrol port set cluster:40 crossport 80

Chapter 5. Advanced features

233

After cross port affinity has been established, you have the flexibility to modify the stickytime value for the port. However, it is recommended that you change the stickytime values for all shared ports to the same value, otherwise unexpected results may occur. To remove the cross port affinity, set the cross port value back to its own port number.

5.7.3 Affinity address mask

Affinity address mask only applies to the Dispatcher component. Affinity address mask is a sticky feature enhancement to groups of clients based upon common subnet addresses. Specifying stickymask in the ndcontrol port command allows you to mask the common high-order bits of the 32-bit IP address. If this feature is enabled, when a client request first makes a connection to the port, all subsequent requests from clients with the same subnet address (represented by that part of the address which is being masked) will be directed to the same server. For example, if you want all incoming client requests with the same network Class A address to be directed to the same server, you set the stickymask value to 8 (bits) for the port. To group client requests with the same network Class B address, set the stickymask value to 16 (bits). To group client requests with the same network Class C address, set the stickymask value to 24 (bits). For best results, set the stickymask value when first starting the Network Dispatcher. If you change the stickymask value dynamically, results may be unpredictable. Keep in mind that there is interaction with cross port affinity: if you are enabling cross port affinity, stickymask values of the shared ports must be the same. See 5.7.2, Cross port affinity on page 233 for more information. To enable affinity address mask, issue an ndcontrol port command similar to the following:
# ndcontrol port set cluster:port stickymask 8

Possible stickymask values are 8, 16, 24 and 32. A value of 8 specifies that the first 8 high-order bits of the IP address (network Class A address) will be masked. A value of 16 specifies that the first 16 high-order bits of the IP address (network Class B address) will be masked. A value of 24 specifies that the first 24

234

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

high-order bits of the IP address (network Class C address) will be masked. If you specify a value of 32, you are masking the entire IP address, which effectively disables the affinity address mask feature. The default value of stickymask is 32.

5.7.4 Rule affinity override

With rule affinity override, you can override the stickiness of a port for a specific server. For example, let us say that you are using a rule to limit the number of connections to each application server, and you have an overflow server with an always true rule that says please try again later for that application. The port has a stickytime value of 25 minutes, but you dont want the client to be sticky to the overflow server. With rule affinity override, you can change the overflow server to override the affinity normally associated with that port. The next time the client requests the cluster, it is load balanced to the best available application server, not to the overflow server.

5.7.5 Cookie affinity

The cookie affinity feature applies only to Content Based Routing with WTE (for HTTP), which supports load balancing based on rules. It provides a new way to make clients sticky to a particular server. This function is enabled by setting the stickytime of a rule to a positive number, and setting the affinity to cookie. This can be done when the rule is added, or by using the rule set command. Once a rule has been enabled for cookie affinity, new client requests will be load-balanced using standard CBR algorithms, while succeeding requests from the same client will be sent to the initially chosen server. The chosen server is stored as a cookie in the response to the client. As long as the clients future requests contain the cookie, and each request arrives within the stickytime interval, the client will maintain affinity with the initial server. Cookie affinity is used to ensure that a client continues to be load balanced to the same server for some period of time. This is accomplished by sending a cookie to be stored by the client browser. The cookie contains: the cluster:port:rule that was used to make the decision, the server that was load balanced to, and a timeout timestamp for when the affinity is no longer valid. Whenever a rule fires that has cookie affinity turned on, the cookies sent by the client are examined. If a cookie is found to contain the identifier for the cluster:port:rule that fired, then the server that was load balanced to, and the expires timestamp are extracted from the cookie. If:

Chapter 5. Advanced features

235

the server is still in the set used by the rule, and its weight is greater than zero, and the expires timestamp is greater than the current time, the server in the cookie is chosen for load balancing. If any of the preceding three conditions are not met, a server is chosen using the normal algorithm. Once a server has been chosen, a new cookie is constructed using the five pieces of information (IBMCBRcluster:port:rule=timestamp), the timestamp will be the time that affinity expires. The cluster:port:rule and the server_chosen are encoded so that no information about the CBR configuration is revealed. An expires parameter is also inserted in the cookie. This parameter is in a format the browser can understand, and causes the cookie to become invalid two hours after the expires timestamp. This is done to avoid cluttering the clients cookie database. This new cookie is then inserted in the headers that go back to the client. If the client browser is configured to accept cookies, it will send back subsequent requests.

How to enable cookie affinity

To enable cookie affinity for a particular rule, use the rule set command:
# rule set cluster:port:rule stickytime 60 # rule set cluster:port:rule affinity cookie

Stickiness is set per rule. Two different rules can send the same client to two different servers. For one rule, the client will be sent to one server, for another rule the client could be sent to a different server. This is a requirement, and for CBR, the servers for rules will be different, so a server using a rule will probably not even exist for another rule. This is desirable because you can set up your rule that handles CGI or servlet requests to be sticky. Your rule for normal HTTP requests will use normal load balancing. Note: Refer to 11.2, CBR for HTTP on page 445 for a scenario using cookie affinity.

Why use cookie affinity

A sticky rule would normally be used for CGI or servlets that store client state on the server. The state is identified by a cookie ID (these are server cookies). The client state is only on the selected server, so the client needs the cookie from that server to maintain that state between requests.

236

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Normal client IP affinity

Normal client IP affinity for CBR is set by rule. To enable normal affinity use:
# rule set cluster:port:rule stickytime 300 # rule set cluster:port:rule affinity clientip

The default for rule affinity is the client IP, so if you have never set it to cookie you do not need to set it to clientip.

5.8 Logging
Network Dispatcher provides several logging methods. It writes information to text log files by default. You can also use a binary logging feature to analyze server statistics. We discuss both methods in the next sections.

5.8.1 Basic logging

Several components of Network Dispatcher generate logs. These logs are recorded in the following path for each platform: AIX: Sun: Linux: NT/2000: /usr/lpp/nd/<component>/logs /opt/nd/<component>/logs \Program Files\ibm\nd\<component>\logs /opt/nd/<component>/logs

Where <component> can be one of the following: Dispatcher, CBR or ISS. You can change the directory where the log files are created. In AIX, Linux and Solaris, you can edit the /usr/bin/ndserver script. Locate the ND_LOGDIR variable, and set it to the new path. On Windows platforms, you can edit the C:\WINNT\SYSTEM32\ndserver.cmd script and look for the same variable. For all operating systems, make sure that there are no spaces on either side of the equal sign and that the path ends in a slash (see Example 5-27).
Example 5-27 The ND_LOGDIR variable For UNIX platforms: ND_LOGDIR=/home/mylogdir/ For Windows platforms: set ND_LOGDIR=C:\MyLogDir\

Chapter 5. Advanced features

237

The following list shows the common log files that are created in the directories above: manager.log Http_80.log Telnet_23.log server.log hamon.log logs the Manager activities logs the HTTP advisor activities logs the Telnet advisor activities logs the ndserver activities logs the high availability monitor activities (high availability scripts execution - refer to Creating the high availability scripts on page 120)

Information can be stored in the log files at different levels: Level 0: logs only errors. Level 1 (the default): logs errors, headers and records of events that happen only once. Levels 2 through 4: log progressively more information, but are not always used. Level 5: the maximum verbosity level. You can change the verbosity of the logs by setting the loglevel.
# # # # ndcontrol ndcontrol ndcontrol ndcontrol set loglevel 5 manager set loglevel 3 manager reach set loglevel 2 advisor loglevel <advisor> <port> 4

You can use the same options for CBR, but you need to use cbrcontrol instead of ndcontrol. Note: Logging consumes resources and disk space. We recommend that you use the default values, unless you need to troubleshoot a problem.

5.8.2 Binary logging

The Dispatcher and CBR components of Network Dispatcher provide an optional binary logging facility. The purpose of this facility is to allow you to analyze server usage and trends. The Manager component must be running in order for information to be entered into the binary logs.

238

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Starting the logging facility

We enabled the logging facility with the Network Dispatcher GUI for a small cluster environment that we used in 4.1, Load balancing on page 70. To start capturing data to the log files, we right-clicked Manager in the left pane of the window and from the pop-up menu selected Start Logging... as shown in Figure 5-45.

Figure 5-45 Starting the binary logging facility

You can also enable logging by running the following command:

# ndcontrol log start

Several other log commands are also available:

# ndcontrol log stop # ndcontrol log status

Chapter 5. Advanced features

239

# ndcontrol log set interval seconds # ndcontrol log set retention hours

Statistics will be placed in the logs at a user-specified interval. The interval can be changed via the GUI or the command line.

Examining the log files

When the logging facility is turned on, one log is created at the start of every hour with the date and time as the name of the file. The logs are placed in the <install path>/<component>/logs directory, where <install path> varies by operating system and <component> can be Dispatcher, ISS or CBR.

Using the LOG_SampleReader sample Java program

The data is recorded in the log in binary format; therefore, users are not able to read the raw log files directly. For this reason, a sample program is provided to be used primarily as a reference for you when writing your own program to read the binary logs. It can also be used to extract one type of log entry. When the Dispatcher is installed, the sample program, called LOG_SampleReader.java, is placed in the location <install path>/lib/BinaryLog. In this section, we explain how the LOG_SampleReader sample Java program extracts the data from the binary logs. We also show how to invoke LOG_SampleReader and interpret its output. Log Data Format The installbase/lib/ibmnd.jar file for the Dispatcher component and the installbase/lib/ibmcbr.jar file for the CBR component provide the utilities (objects and their associated methods) that LOG_SampleReader uses to read the data from the log. The comments embedded in LOG_SampleReader.java inform us that the following types of records can be retrieved from the log: LOG_TimestampRecord LOG_ExecutorIDRecord LOG_ClusterIDRecord LOG_PortIDRecord LOG_ServerReportRecord

Each of these types has associated with it one or more methods which can be used to determine its value from the LOG_Record object. For example, the LOG_ServerReportRecord implements the following eight methods: c. getIPAddress() returns the server IP address. d. getWeight() returns the server weight of the server when the record was written. e. getTotalConnections() returns the total number of server connections when the record was written.

240

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

f. getActiveConnections() returns the number of active connections. g. getPortLoad() returns the port load of the server when the record was written. h. getHasPortLoad() returns a Boolean value indicating whether or not port loads were being provided for the server when the record was written to the binary log (port loads are obtained through an advisor). i. getSystemLoad() returns the system load of the server when the record was written. j. getHasSystemLoad() returns a Boolean value indicating whether or not system loads were being provided for the server when the record was written to the binary log (system loads can be provided, for example, using ISS). LOG_SampleReader.java extracts ServerReport data from the logs by using the LOG_Reader and LOG_Record objects and their associated methods. These methods scan through the log records searching for a record of type LOG_ServerReportRecord. When one is found, it is printed. Invoking the Sample Program The Dispatcher and CBR components <install path>/<component>/samples/BinaryLog directories each contain a small shell script that serves as a wrapper for LOG_SampleReader. It sets up the required CLASSPATH elements and location of the log directory before invoking LOG_SampleReader through the java command. Example 5-28 shows the Dispatcher wrapper file, ndlogreport.cmd, from our Windows NT system.
Example 5-28 ndlogreport.cmd script on Windows NT @echo off rem Set the NDBLOGDIR variable below to the directory that contains rem your log reader. set NDBLOGDIR=%IBMNDPATH%\dispatcher\samples\BinaryLog set NDCLASSPATH=%NDBLOGDIR%;%IBMNDPATH%\dispatcher\lib\ibmnd.jar java -DEND_LOG_DIRECTORY=%IBMNDPATH%\dispatcher\logs -cp %NDCLASSPATH% LOG_SampleReader %1 %2 %3 %4

Chapter 5. Advanced features

241

Example 5-29 shows the ndlogreport script from our AIX system, which is similar to the one provided in Linux and Solaris.
Example 5-29 ndlogreport shell script on AIX #!/bin/ksh # Set the NDBLOGDIR variable below to the directory that contains # your log reader. NDBLOGDIR=/usr/lpp/nd/dispatcher/samples/BinaryLog export CLASSPATH=\ $NDBLOGDIR:\ /usr/lpp/nd/dispatcher/lib/ibmnd.jar java -DEND_LOG_DIRECTORY=/usr/lpp/nd/dispatcher/logs LOG_SampleReader $1 $2 $3 $4

We invoked this script with four arguments representing the start date and time and end date and time of the server records we wanted to see from the log. We used the following command in a Dispatcher running on Windows NT:
Example 5-30 Generating a report C:\> cd program files\ibm\nd\dispatcher\samples\BinaryLog C:\> ndlogreport.cmd 2001/04/03 13:00 2001/04/03 20:00

Interpreting LOG_SampleReader Output The output from the above command is shown in Example 5-31:
Example 5-31 Output from ndlogreport command 2001/04/03-18:43:07.879,9.24.105.82,9.24.105.87,80,9.24.105.39,9,18,0,541,true,0,false 2001/04/03-18:43:07.879,9.24.105.82,9.24.105.87,80,9.24.105.59,10,18,0,130,true,0,false 2001/04/03-18:44:08.316,9.24.105.82,9.24.105.87,80,9.24.105.39,9,26,0,520,true,0,false 2001/04/03-18:44:08.316,9.24.105.82,9.24.105.87,80,9.24.105.59,10,26,0,131,true,0,false 2001/04/03-18:45:09.944,9.24.105.82,9.24.105.87,80,9.24.105.39,9,33,0,521,true,0,false 2001/04/03-18:45:09.944,9.24.105.82,9.24.105.87,80,9.24.105.59,9,33,0,130,true,0,false 2001/04/03-18:45:09.944,9.24.105.82,9.24.105.87,80,9.24.105.20,10,5,0,130,true,0,false 2001/04/03-18:46:10.391,9.24.105.82,9.24.105.87,80,9.24.105.39,10,63,0,511,true,0,false 2001/04/03-18:46:10.391,9.24.105.82,9.24.105.87,80,9.24.105.59,10,62,0,130,true,0,false 2001/04/03-18:46:10.391,9.24.105.82,9.24.105.87,80,9.24.105.20,0,22,0,130,true,0,false

There are 13 fields on each line: a. date b. time c. Dispatcher nonforwarding address d. cluster IP address e. port number f. server IP address

242

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

g. server weight h. total connections i. active connections j. port load of the server k. whether the Dispatcher is receiving port load information l. server load m. whether the Dispatcher is receiving server load information We can determine from the above output that at the start of the period, there were only two servers for the cluster, 9.24.105.39 and 9.24.105.59. Later on, another server was added to this cluster: 9.24.105.20. We can also determine that all servers were responding to the advisor (see the true value for field k) and they were not using ISS (see the false value for field m). This program can be modified to process the binary log information for any kind of analysis that you require.

Chapter 5. Advanced features

243

244

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Part 3

Part

Web Traffic Express

In this part, we describe the Web Traffic Express component of WebSphere Edge Server. The Web Traffic Express component of WebSphere Edge Server for Multiplatforms is also referred to in many publications as the caching proxy.

Copyright IBM Corp. 2001

245

246

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 6.

Web Traffic Express installation

This chapter covers the installation steps for Web Traffic Express in the following environments: AIX: see 6.1, Installing Web Traffic Express on AIX on page 248. Windows NT: see 6.2, Installing Web Traffic Express on Windows NT on page 262. Linux: see 6.3, Installing Web Traffic Express on Linux on page 274. Procedures for starting and stopping services and for uninstallation are also covered.

Copyright IBM Corp. 2001

247

6.1 Installing Web Traffic Express on AIX

In this section, we describe the hardware and software requirements for Web Traffic Express, the installation process, and how to start and stop Web Traffic Express on an AIX platform.

6.1.1 System hardware and software requirements

Any IBM RS/6000-based machine AIX Version 4.3.2 or higher (32-bit only) Asynchronous I/O enabled on the WTE machine. To verify that the required bos.rte.aio fileset is installed, issue the following command:
$ lslpp -l bos.rte.aio

To verify that asynchronous I/O is enabled, issue the following command:

# smit chgaio

Set the state to available in the STATE to be configured at system restart field. Note: You must have super user root as the local machine to configure asynchronous I/O. Note that changes to this setting will not be effective until the system has been restarted. If running WTE in a non-English language environment, edit the /etc/environment file, setting the LC__FASTMSG environment variable to the value false. 50 MB of available disk space for installation and documentation, plus additional space for logs. Note: We suggest you create a separate filesystem called /opt, to install WTE. If you do not create a separate file system for WTE and you plan to implement filtering using CyberPatrol (refer to 8.1.4, Blocking URLs using the CyberPatrol plug-in on page 336), you may need to increase your root filesystem by 30 MB. Any communication hardware adapter configured to use the TCP/IP stack for making network connections.

248

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

A minimum of 64 MB RAM, though greater amounts yield better performance. The following functions require an amount of RAM over and above the amount required for basic WTE operation: Caching of any type. WTE uses additional RAM as overhead for internal record keeping about each cache device. For cache devices smaller than 1 GB, the requirement for each device is 8.5 MB plus about 1% of the cache size on the device. For devices 1 GB and larger, the additional RAM requirement is 1% of the cache size on the device. Note: The fact that overhead is required for each device implies that it is more efficient to use fewer large devices than many small ones. Memory caching. The appropriate cache size (amount of additional RAM required) depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. A suggested minimum is 17 MB for reverse proxy, and the greater of 17 MB or 2 MB per user for forward proxy. If transparent proxy is supported, the suggested minimum is the same as for forward proxy. Free disk space for paging: a minimum of twice the amount of RAM is required. Free disk space for disk or file caching: this depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. Suggested minimum values are the same as described previously for memory caching. Note: For WTE to cache to disk or to a file, you must format the disk or file by using the htcformat command as described in Chapter 7, Web Traffic Express basic configuration on page 287. Netscape V4.7 or later in order to configure WTE using the Configuration and Administration forms. Note: Netscape Version 6 is not supported for use with Web Traffic Express Configuration and Administration forms. Content Based Routing (CBR) calls for the JRE (Java Runtime Environment) required by Network Dispatcher and the ND CBR component.

Chapter 6. Web Traffic Express installation

249

6.1.2 Installation
This section describes the steps to follow to install Web Traffic Express on the AIX platform. Before starting the installation process, you must choose which type of proxy your configuration calls for. The following are the different types of proxies that can be chosen during the installation process (refer to Figure 6-7 on page 256): Forward proxy: Intercepts HTTP requests generated by Web browser clients that are configured to use it as the proxy. Figure 6-1 illustrates this configuration.
WEB Clients

WTE Forward Proxy Server

Router

Content Server

Internet

Figure 6-1 Web Traffic Express acting as a forward proxy

250

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Reverse proxy: Intercepts HTTP requests destined for your content host. The following screen illustrates this scenario:
WEB Clients

WTE Reverse Proxy Server Content Server

Internet

Router

Figure 6-2 Web Traffic Express acting as a reverse proxy

Transparent proxy: Intercepts HTTP requests generated by Web browser clients that are not configured to use it as a proxy. This is possible because the router is configured to redirect all requests destined for port 80 to the proxy server. Figure 6-3 illustrates this scenario:
Content Server

WEB Clients
Router

Internet

WTE Transparent Proxy Server

Figure 6-3 Web Traffic Express acting as a transparent proxy

Chapter 6. Web Traffic Express installation

251

Once you have chosen the type of proxy for your configuration, you can proceed with the installation process. Our environment consists of an IBM RISC/6000 43P Model 150 with 768 MB RAM, two 9.1 GB hard disks, and one token-ring adapter. It is running AIX 4.3.3.0. 1. Log in as the root user:
% su root

Password: root_password 2. Create a directory on which to mount the CD-ROM, if it does not already exist:
# mkdir /cdrom

3. Insert the WebSphere Edge Server media in the CD-ROM drive. 4. Mount the CD-ROM:
# mount -v cdrfs -p -r /dev/cd0 /cdrom # cd /cdrom

5) Start the InstallShield Wizard:

# ./setup

252

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In the first window, you have to choose a setup language (see Figure 6-4).

Figure 6-4 Setup language

Click the Next button to continue installation. The next window is the Software License Agreement. After you read and accept this license, click the Accept button to continue.

Chapter 6. Web Traffic Express installation

253

WebSphere Edge Server consists of two components: Network Dispatcher and Web Traffic Express. In the next window (shown in Figure 6-5), select which component you want to install.

Figure 6-5 Choosing the WebSphere Edge Server component

Select Web Traffic Express, which is the caching proxy component, and click the Next button.

254

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The following window contains descriptions of the different types of configurations supported by Web Traffic Express.

Figure 6-6 Information on the types of proxy configurations

After reading this information and clicking Next, you are presented with a window where you can choose how the proxy server will function.

Chapter 6. Web Traffic Express installation

255

Figure 6-7 Selecting the function of the proxy server.

For a description of each type of proxy, refer to 6.1.2, Installation on page 250. We selected the Forward Proxy option. Click Next.

256

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

This brings you to the port selection window (see Figure 6-8). Here, you specify the port number on which Web Traffic Express will listen for requests from the clients. The default port is 80. You can change this to any available port.

Figure 6-8 Choosing the port number

Chapter 6. Web Traffic Express installation

257

You can customize the Web Traffic Express component using the browser-based Configuration and Administration forms. In the next window (Figure 6-9), you define the user ID and password which the WTE administrator will use to access this feature.

Figure 6-9 Defining user ID and password

Type the password and click Next.

258

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The InstallShield displays a summary of all selections and asks you to change them or proceed. This is the last chance to change anything before the installation of the files.

Figure 6-10 Confirming before copying files

After you confirm the installation options, InstallShield will copy all files to the /opt/IBMWTE directory.

Chapter 6. Web Traffic Express installation

259

Figure 6-11 Installation Summary window

Note: Web Traffic Express installation creates two log files in the /opt/IBMWTE directory. One log file, called trace.log, contains all the steps performed in the installation process. Another file, called install.log, lists all filesets installed. The last window of the installation process shows the path of the readme file. This file contains the latest information about any known problems.

6.1.3 Starting and stopping Web Traffic Express

On AIX, Web Traffic Express is a process called ibmproxy. The InstallShield Wizard starts the ibmproxy process as the final step of the installation with the default configuration settings. It also creates a Web Traffic Express initialization file in the /etc directory and includes it in the machines startup sequence. To stop this process, enter the following command:
# stopsrc -s ibmproxy

260

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

To start this process, enter:

# startsrc -s ibmproxy

Note: When Web Traffic Express starts, a file called ibmproxy-pid is created in the /opt/IBMWTE/usr/internet/server-root directory. This file contains the process ID (PID).

6.1.4 Uninstallation
To uninstall Web Traffic Express, follow these steps: 1. Log in as the root user:
% su root

Password: root_password 2. Stop the Web Traffic Express process as described in 6.1.3, Starting and stopping Web Traffic Express on page 260. 3. Create a directory on which to mount the CD-ROM, if it does not already exist.
# mkdir /cdrom

4. Insert the WebSphere Edge Server media in the CD-ROM drive. 5. Mount the CD-ROM.
# mount -v cdrfs -p -r /dev/cd0 /cdrom # cd /cdrom

6. Issue the setup command with the following option:

# ./setup -wteuninstall

Note: Web Traffic Express uninstallation creates one log trace file called unintsall.log on /opt/IBMWTE. This file contains all the steps performed in the uninstallation process. You can also uninstall using SMIT. The following filesets must be deleted: ibmwte.base.exe 3.6.0.0 ibmwte.base.ssl 3.6.0.0 ibmwte.doc.en_US 3.6.0.0 ibmwte.msg.en_US.base 3.6.0.0

Chapter 6. Web Traffic Express installation

261

6.2 Installing Web Traffic Express on Windows NT

In this section, we describe the installation requirements and how to start and stop Web Traffic Express on a Windows NT platform.

6.2.1 System hardware and software requirements

Any Intel x86 PC, including SMP models supported by Windows 2000 or Windows NT. Windows NT 4.0 with Service Pack 5 or higher, or Windows 2000. 50 MB of available disk space for installation and documentation, plus additional space for logs. Any communication hardware adapter configured to use the TCP/IP stack for making network connections. A minimum of 64 MB RAM, though greater amounts yield better performance. The following functions require an amount of RAM over and above the amount required for basic WTE operation: Caching of any type. WTE uses additional RAM as overhead for internal record keeping about each cache device. For cache devices smaller than 1 GB, the requirement for each device is 8.5 MB plus about 1% of the cache size on the device. For devices 1 GB and larger, the additional RAM requirement is 1% of the cache size on the device. Note: The fact that overhead is required for each device implies that it is more efficient to use fewer large devices than many small ones. Memory caching. The appropriate cache size (amount of additional RAM required) depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. A suggested minimum is 17 MB for reverse proxy, and the greater of 17 MB or 2 MB per user for forward proxy. If transparent proxy is supported, the suggested minimum is the same as for forward proxy. Free disk space for paging: a minimum of twice the amount of RAM is required. Free disk space for disk or file caching: this depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. Suggested minimum values are the same as described previously for memory caching.

262

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Note: For WTE to cache to disk or to a file, you must format the disk or file by using the htcformat command as described in Chapter 7, Web Traffic Express basic configuration on page 287. Netscape V4.72 or later, or Microsoft IE V5.0 or later is required to configure WTE using the Configuration and Administration forms. Note: Netscape Version 6 is not supported for use with Web Traffic Express Configuration and Administration forms.

Content Based Routing (CBR) demands the JRE (Java Runtime Environment) required by Network Dispatcher and the ND CBR component.

6.2.2 Installation
This section describes the steps to follow to install Web Traffic Express on the Windows NT platform. Our environment is an IBM PC 300PL with 256 MB RAM, 1.5 GB and 600 MB hard disk, and one Token Ring adapter. It is running Windows NT 4.0 with Service Pack 5. 1. Insert the WebSphere Edge Server media into the CD-ROM drive. 2. Click Start-> Run...-> Browse and choose setup.exe on the CD-ROM drive as shown in Figure 6-12.

Figure 6-12 Installation program name and path

Notice that D is the drive letter assigned to the CD-ROM drive. Click OK to start the installation.

Chapter 6. Web Traffic Express installation

263

The first window shows the Software License Agreement. After you read and accept this license, click Accept to continue the installation. In the next window, choose a setup language, as shown in Figure 6-13. Click Next.

Figure 6-13 Choosing a setup language

264

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The InstallShield Wizard lists the WebSphere Edge Server Window components (refer to Figure 6-14). WebSphere Edge Server consists of two components: Network Dispatcher and Web Traffic Express. In this window, you select which component you want to install.

Figure 6-14 Choosing the WebSphere Edge Server component

Select the Web Traffic Express option, which is the caching proxy component. Click Next.

Chapter 6. Web Traffic Express installation

265

InstallShield then prompts you to choose an installation directory. The default location is C:\ProgramFiles\IBM as shown in Figure 6-15.

Figure 6-15 Choosing the destination location

Click Next to proceed with the installation.

266

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The next window gives a description of the different types of configurations that are supported with Web Traffic Express (see figure below).

Figure 6-16 Information about types of proxy

After reading this information and clicking Next, you are presented with a window where you can choose how the proxy server will function.

Chapter 6. Web Traffic Express installation

267

Figure 6-17 Selecting the function of the proxy server.

For a description of each type of proxy, refer to 6.1.2, Installation on page 250. We selected the Forward Proxy option. Click Next. Important: The Transparent proxy configuration type is not available in the Windows environment.

268

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

This brings you to the port selection window (see Figure 6-18). Here, you specify the port number on which Web Traffic Express will listen for requests from the clients. The default port is 80. You can change this to any available port.

Figure 6-18 Choosing the port number

Chapter 6. Web Traffic Express installation

269

You can customize the Web Traffic Express component using the browser-based Configuration and Administration forms. In the next window (Figure 6-19), you define the user ID and password used by the WTE administrator to access this feature.

Figure 6-19 Defining user ID and password to access configuration and administration forms

Type the password and click Next.

270

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The InstallShield displays a summary of all selections and asks you to change them or proceed. This is the last chance to change anything before the installation of files.

Figure 6-20 Confirming before copying files

After you confirm the installation options, the InstallShield Wizard will copy all files to the destination location. When installation is complete, the InstallShield will display a window asking you to reboot the machine.

Chapter 6. Web Traffic Express installation

271

Figure 6-21 Installation Summary window

Note: Web Traffic Express installation creates one log trace file called trace.log on C:\ProgramFiles\IBM\WTE. This file contains all the steps performed in the installation process. The last window of the installation process shows the path of the readme file.

6.2.3 Starting and stopping Web Traffic Express

Web Traffic Express is added as an NT Service called IBM Web Traffic Express. It is automatically started during the boot process. You can stop the service by selecting it and clicking Stop (see below).

272

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 6-22 IBM Web Traffic Express Service

6.2.4 Uninstallation
The Web Traffic Express program group contains readme icons, Uninstall IBM Web Traffic Express and Key Management Utility. To uninstall, click Start -> Programs -> IBM WebSphere -> Edge Server -> Web Traffic Express -> Uninstall IBM Web Traffic Express. The window shown on Figure 6-23 is displayed.

Figure 6-23 Window to confirm the uninstallation

Chapter 6. Web Traffic Express installation

273

The following window displays the progress of the WTE uninstallation.

Figure 6-24 Uninstallation progress

If you click the Details... button you can see that the log files were not deleted. Also notice that the configuration files in the c:\Program Files\IBM\WTE directory, as well as some registry entries, were not deleted. If you would like to delete these registry entries, look for the following: EventMessage File Library NLSPath ImagePath

6.3 Installing Web Traffic Express on Linux

In this section, we describe the installation requirements and how to start and stop Web Traffic Express on a Linux platform.

274

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

6.3.1 System hardware and software requirements

Any Intel x86 PC including SMP models supported by one of the following Linux releases: Red Hat Linux 6.1 (kernel Version 2.2.12-20) Red Hat Linux 6.2 (kernel Version 2.2.16-3) SuSE Linux 6.4 (kernel Version 2.2.14) Free disk space for software and documentation: 50 MB, plus additional space for logs. Any communication hardware adapter configured to use the TCP/IP stack for making network connections. A minimum of 64 MB RAM, though greater amounts yield better performance. The following functions require an amount of RAM over and above the amount required for basic WTE operation: Caching of any type. WTE uses additional RAM as overhead for internal record keeping about each cache device. For cache devices smaller than 1 GB, the requirement for each device is 8.5 MB plus about 1% of the cache size on the device. For devices 1 GB and larger, the additional RAM requirement is 1% of the cache size on the device. Note: The fact that overhead is required for each device implies that it is more efficient to use fewer large devices than many small ones. Memory caching. The appropriate cache size (amount of additional RAM required) depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. A suggested minimum is 17 MB for reverse proxy, and the greater of 17 MB or 2 MB per user for forward proxy. If transparent proxy is supported, the suggested minimum is the same as for forward proxy. Free disk space for paging: a minimum of twice the amount of RAM. Free disk space for disk or file caching: this depends on the size and number of files that users retrieve from Web servers. Larger caches generally have higher cache hit rates. Suggested minimum values are the same as described previously for memory caching. Note: For WTE to cache to disk or to a file, you must format the disk or file by using the htcformat command as described in Chapter 7, Web Traffic Express basic configuration on page 287.

Chapter 6. Web Traffic Express installation

275

6.3.2 Installation
This section describes the steps to follow to install Web Traffic Express on the Linux NT platform. Our installation environment is a NetFinity 3000 with 256 MB RAM, a 9.1GB hard disk, and one Token Ring adapter. It is running Red Hat 6.2. 1. Log in as the root user:
% su root

Password: root_password 2. Create a directory in which to mount the CD-ROM, if it does not already exists.
# mkdir /mnt/cdrom

3. Insert the WebSphere Edge Server media in the CD-ROM drive. 4. Mount the CD-ROM.
# mount /dev/cdrom /mnt/cdrom

5. If you have Linux autorun feature enabled on the machine, the InstallShield wizard starts automatically, as shown in Figure 6-25.

Figure 6-25 Autorun feature

Click Yes to continue the installation. If you do not have the Linux autorun feature enabled, issue the following commands:
# cd /mnt/cdrom # ./setup

276

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

In the first window, you choose a setup language.

Figure 6-26 Choosing a setup language

Click Next to continue the installation.

Chapter 6. Web Traffic Express installation

277

The next window shows the Software License Agreement. After you read and accept this license, click Accept to continue the installation. The WebSphere Edge Server has two components: Network Dispatcher and Web Traffic Express. In the next window, you select which component you want to install.

Figure 6-27 Choosing the WebSphere Edge Server component

Select the Web Traffic Express option, which is the caching proxy component, and click Next.

278

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The next window gives a description of the different types of configurations that are supported with Web Traffic Express.

Figure 6-28 Information about type of proxy

After reading this information and clicking Next, you are presented with a window where you can choose how the proxy server will function.

Chapter 6. Web Traffic Express installation

279

Figure 6-29 Selecting the function of the proxy server.

For a description of each type of proxy, refer to 6.1.2, Installation on page 250. We selected the Forward Proxy option. Click Next.

280

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

This brings you to the window shown in Figure 6-30. Here, you can specify the port number on which Web Traffic Express will listen for requests from the clients. The default port is 80. You can change this to any available port.

Figure 6-30 Choosing the port number

Chapter 6. Web Traffic Express installation

281

You can customize the Web Traffic Express component using the browser-based Configuration and Administration forms. In the next window (Figure 6-31), you define the user ID and password to be used by the WTE administrator to access this feature.

Figure 6-31 Defining user ID and password to access configuration and administration forms

Enter a password and click Next.

282

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The InstallShield displays a summary of all selections and ask you to change them or proceed. This is the last chance to change anything before the installation of files.

Figure 6-32 Confirming before copying files

After you confirm the installation options, the InstallShield will copy all files to the /opt/IBMWTE directory.

Chapter 6. Web Traffic Express installation

283

Figure 6-33 Installation Summary window

Note: The Web Traffic Express installation creates one log trace file called trace.log on /opt/IBMWTE. This file contains all the steps followed in the installation process. The last window shows the location of the readme file. This file contains the latest information about any known problems.

6.3.3 Starting and stopping Web Traffic Express

On Linux, Web Traffic Express runs as a process called ibmproxy. The InstallShield Wizard starts the ibmproxy process as the final step of the installation. It also creates a Web Traffic Express initialization file in the /etc/rc.d/init.d directory and includes it in the machines startup sequence. Important: Linux includes an HTTP server that binds to port 80. Since WTE use the same port, you need to stop the HTTP server before starting WTE.

284

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

To stop the WTE process, use the following command:

# /etc/rc.d/init.d/ibmproxy stop

To start the WTE process, use this command:

# /etc/rc.d/init.d/ibmproxy start

Note: When the Web Traffic Express starts, a file called ibmproxy-pid is created in the /opt/IBMWTE/usr/internet/server-root directory. This file contains the process ID (PID).

6.3.4 Uninstallation
To uninstall Web Traffic Express, follow these steps: 1. Log in as the root user:
% su root

Password: root_password 2. Stop the Web Traffic Express process as described in 6.3.3, Starting and stopping Web Traffic Express on page 284. 3. Create a directory on which to mount the CD-ROM, if it does not already exist:
# mkdir /mnt/cdrom

4. Insert the WebSphere Edge Server in the CD-ROM drive. 5. Mount the CD-ROM.
# mount /dev/cdrom /mnt/cdrom # cd /mnt/cdrom

6. If you have Linuxs autorun feature enabled on the machine, the InstallShield Wizard starts automatically. Click Cancel. 7. Issue the setup command with the following option:
# ./setup -wteuninstall

Note: The Web Traffic Express uninstallation creates one log trace file called uninstall.log on /opt/IBMWTE. This file contains all the steps followed in the uninstallation process.

Chapter 6. Web Traffic Express installation

285

286

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 7.

Web Traffic Express basic configuration

In this chapter, we discuss the basic configuration and administration of WTE using both the command line interface and the Configuration and Administration forms browser interface.

Copyright IBM Corp. 2001

287

7.1 Basic configuration

Web Traffic Express can be configured and customized using the browser-based Configuration and Administration forms, or by editing the Web Traffic Express configuration file ibmproxy.conf. Both of these configuration methods will be discussed in this chapter.

7.1.1 Configuration environment

In our test scenario environment, we have three machines with Web Traffic Express installed. Table 7-1 and Figure 7-1 describe the specifics of our network configuration. This configuration will be used for all the following WTE chapters.
Table 7-1 Description of all machines Hostname rs600034 m238P4xl rhlinux1 m23kk904 m238p4yl rs60008 IP Address 9.24.105.61 9.24.105.83 9.24.105.63 9.24.105.39 9.24.105.82 9.24.105.59 Operating System AIX 4.3.3 Windows NT 4.0 Linux Red Hat 6.2 Windows NT 4.0 Windows NT 4.0 AIX 4.3.3 Service Caching Proxy Server Caching Proxy Server Caching Proxy Server Web Client Web Client Web Server

All machines are on the same network and in the itso.ral.ibm.com domain.

288

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

WTE Proxy Servers

9.24.105.83 M238P4XL 9.24.105.63 RHLINUX1 9.24.105.61 RS600034

WEB Server
9.24.105.59 RS60008

9.24.105.39 M23KK904

9.24.105.82 M238P4YL

WEB Clients
Figure 7-1 Test scenario

7.1.2 Using the Configuration and Administration forms

The Configuration and Administration forms GUI is a common way to set up and maintain the ibmproxy.conf configuration file. It presents a graphical interface using your Web browser. To access the administration forms, Web Traffic Express must be running on the server (for more information on starting and stopping WTE, refer to the corresponding platform in Chapter 6, Web Traffic Express installation on page 247). You need only a Web browser. To access these HTML forms, type either of the following URLs: http://your.server.name:port http://your.server.name/Frntpage.html:port In our case, we used the following URL: http://rs600034. By default, the port is port 80. We did not change this in our installation.

Chapter 7. Web Traffic Express basic configuration

289

Figure 7-2 shows the default front page supplied with Web Traffic Express.

Figure 7-2 Web Traffic Express front page

With this front page, you can view the documentation, visit the product support Web site or access the Configuration and Administration forms. To access the Administration forms, you must have a user ID and password. Both of these can be defined during production installation. Refer to Figure 6-9 on page 258 to see where the user ID and password are initially defined for our AIX installation.

290

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

7.1.3 Web Traffic Express Administration settings

Only authorized users can configure the ibmproxy.conf file via the Configuration and Administration forms interface. This section describes how to define another user ID and password to administer or modify Web Traffic Express. You can use the command line interface or the Configuration and Administration forms interface to define a different administrator user name and password. We show how to add a user with the Configuration and Administration forms in 8.1.2, Basic user authentication on page 320. In this section, we will use the command line interface. The WTE command to add an authorized user is htadm. If you input this command with no parameters, a short description of how to use the command is displayed, as shown on the following example.
Example 7-1 htadm syntax # htadm Administrative tool for Usage: -adduser pwfile -deluser pwfile -passwd pwfile -check pwfile -create pwfile

Server access authorization [user [password [real name]]] [user] [user [password]] [user [password]]

Note: The proxy server maintains its own list of user names, passwords, and group names.

Setting administrator user ID and password on AIX

On AIX systems, WTE administrator user names and passwords are stored in the WTE password file. The default file is webadmin.passwd. This password file is located in the /opt/IBMWTE/usr/internet/server_root/protect directory. To define a different administrator user name and password for use with the Configuration and Administration forms from a Web browser, use the htadm command as follows:
htadm -adduser password_file user-name password real-name

The htadm executable file is automatically installed in the /usr/sbin directory. However, for password_file, you must specify the full path or execute htadm in the directory where the password file resides.

Chapter 7. Web Traffic Express basic configuration

291

The real-name variable is not required. It is a comment or a name you want to use to identify the user name. WTE writes this variable to the password file, but does not do anything with it. If you do not specify the real-name variable, the system will prompt you for it, at which time you may press Enter to signify no identity, or enter a real user name. We entered the following commands:
# cd opt/IBMWTE/usr/internet/server_root/protect # htadm -adduser webadmin.passwd wteadmin admin "WTE Administrator"

We entered wteadmin as the user name and admin as the password. We now have two administrators defined to WTE. The password is written into the password file, webadmin.passwd. This file on our system contained the following two lines after entering the above command:
admin:bUckfUvSwcgZ.:administrator wteadmin:FGVS7ptGfgzOE:WTE Administrator

The password is encrypted in the password file. Interestingly, WTE encrypts each new password with a different key. You can see this by defining two users with the same password.

Setting administrator user ID and password on Windows NT

On Windows NT systems, user names and passwords are stored in the caching proxy server default password file, which is admin.pwd. This file is located in the Windows NT installation directory, which by default is c:\Program Files\IBM\WTE. To define a different administrator user name and password for use with the Configuration and Administration forms from a Web browser, use the htadm command as follows:
htadm -adduser password_file user-name password real-name

The htadm.exe is installed in the C:\Program Files\IBM\WTE\bin directory. This path is automatically added in the path system environment variable when WTE is installed. However, for password_file, you must specify the full path or execute htadm in the directory where the password file resides. The real-name variable is not required. It is a comment or a name you want to use to identify the user name. WTE writes this variable to the password file, but does not do anything with it. If you dont specify the real-name variable, the system will prompt you for it, at which time you may press Enter to signify no identity, or enter a real user name. We entered the following commands:
cd c:\Program Files\ibm\wte htadm -adduser admin.pwd wteadmin admin "WTE Administrator"

292

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

We entered wteadmin as the user name and admin as the password. We now have two administrators defined to WTE. The password that you enter is written into the password file, admin.pwd. This file on our system contained the following two lines after entering the above command:
admin:bFnr3RsF76DFg:administrator wteadmin:UmXJL9v2p5tpI:WTE Administrator

The password is encrypted in the password file. Interestingly, WTE encrypts each new password with a different key. You can see this by defining two users with the same password.

Setting administrator user ID and password on Linux

Defining a new administrator user ID and password in the Linux environment is very similar to the same process in the AIX environment. See Setting administrator user ID and password on AIX on page 291.

7.1.4 Configuring the basic settings

Click the CONFIGURATION AND ADMINISTRATION FORMS option on the home page (refer to Figure 7-2). The window shown in Figure 7-3 is displayed. You will be asked to authenticate; enter a username and the password of the administrator.

Figure 7-3 Entering Web Traffic Express administrator and password

We entered admin in the User Name field and admin in the Password field, and clicked OK. If the authentication is validated, the window shown in Figure 7-4 is displayed.

Chapter 7. Web Traffic Express basic configuration

293

Figure 7-4 Web Traffic Express Introduction window

Almost all customization is performed using the groups listed in the left pane. For the changes to take effect, you must restart the WTE server by clicking the Restart Server button. This restarts the ibmproxy daemon which will re-read the ibmproxy.conf file.

294

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

All updates made using Configuration and Administration forms are stored in ibmproxy.conf. This file contains directives that define the functionality of the WTE server.
Table 7-2 Location of the ibmproxy.conf file Operating System Windows NT 4.0 AIX 4.3.3 Linux Red Hat 6.2 Location c:\program files\ibm\wte /etc (the file in this directory is a link to /opt/IBMWTE/usr/internet/etc) /etc (the file in this directory is a link to /opt/IBMWTE/usr/internet/etc)

Note about Netscape 4.x browsers: If the Netscape browsers window is resized, the screen clears and a message is displayed informing the user that the browser does not support JavaScript. In order to properly redisplay the form, click the browsers Reload button (Netscape 6 not supported). Some directives are not refreshed with the Restart Server button. You must stop the server and restart it as described in Chapter 6, Web Traffic Express installation on page 247. All directives that are not refreshed are listed in Table 7-3.
Table 7-3 Directives not refreshed upon restart Task Group CGI Caching Logging Network Access Performace SNMP UNIX Process Control Miscellaneous Directives DisinheritEnv, InheritEnv Caching AcessLog, CacheAccessLog, ErrorLog, ProxyAccessLog, ServerRoot BindSpecifc, Hostname, ListenBacklog, Port MaxActiveThreads SNMP, SMNPCommunity, WebMasterEmail GroupID, UserID TransparentProxy

In the following sections, we describe configuration options using the groups available in the Configuration and Administration forms.

Chapter 7. Web Traffic Express basic configuration

295

Configuration host name

In a Windows NT installation, this directive is already defined. However, on the AIX and Linux platforms, you must configure it. Choose Server Configuration -> Basic Settings. The following window is displayed.

Figure 7-5 Basic Settings window

We input rs600034.itso.ral.ibm.com in the Host name field. Click Submit (not shown on Figure 7-5) to update the configuration file, ibmproxy.conf.

296

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The default of the Process ID File Location field is <WTE-root>/ibmproxy-pid. However, if you do not fill in this field, Configuration and Administration forms will generate an empty PidFile directive at the end of the ibmproxy.conf file. The same will happen with the InheritEnv and DisInheritEnv fields. After restarting the proxy server, the following error messages will be generated in the error log file:
Example 7-2 error.Apr042001 [04/Apr/2001:09:01:33 +0500] [OK] [host: ] Insufficient parameters for directive : PidFile [04/Apr/2001:09:01:33 +0500] [OK] [host: ] Insufficient parameters for directive : InheritEnv [04/Apr/2001:09:01:33 +0500] [OK] [host: ] Insufficient parameters for directive : DisInheritEnv

We recommend that you edit the ibmproxy.conf file and uncomment the lines of the directives shown in Example 7-3. The default values will then be used and the errors will not be logged.
Example 7-3 iibmproxy.conf ... # PidFile # InheritEnv # DisInheritEnv

For the changes to take effect, you must restart the WTE server by clicking the Restart button.

Chapter 7. Web Traffic Express basic configuration

297

Enable proxy function

To configure this option, choose Proxy Configuration -> Proxy Settings. The following window will be displayed.

Figure 7-6 Proxy Settings window

298

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

WTE is a proxy server when installed. The default setting automatically provides the proxy function for the following protocols: HTTP: process HTTP requests FTP: process FTP requests Gopher: process gopher requests SSL Tunneling: use any port to pass encrypted data, unaltered, between a client and a content server When the proxy returns dynamic data, such as output data from CGI programs, API programs, server-side includes, and Java servlets, it must buffer the data. You set the value of the buffer size in the Proxy buffer size field. We accepted the default value, which is 100 KB. The next field in the Proxy Settings form is the Proxy access log. This field allows you to specify the file name where WTE will log access requests that are proxied on this server. Refer to Figure 7-14 on page 312 to see how this information may be viewed. The fully qualified path is required. Each day at midnight, the WTE server, if it is running, starts a new log file. It uses the specified file name and appends a date suffix as an extension. Table 7-4 shows the default values for this field.
Table 7-4 Default proxy access log location Platform AIX Windows NT Linux Default path and file /opt/IBMWTE/usr/internet/server_root/logs/proxy \Program Files\IBM\WTE\logs\proxy /opt/IBMWTE/usr/internet/server_root/logs/proxy

Note: This log file can take up a significant amount of space on your file system. It is recommended that a different file system be created to hold the log files. We modified the value of the Proxy access log field and set it to /wte/logs/proxy. The /wte file system and /log directory were created earlier. Permissions are important when changing the default directory where the log files will reside. The WTE server will write to that directory as the servers user ID and group ID specified in the ibmproxy.conf configuration file (nobody/nobody by default; refer to Figure 7-5 on page 296). Therefore, if you have created a new directory for the log files, you must ensure that the WTE servers user ID can write to that directory.

Chapter 7. Web Traffic Express basic configuration

299

For the configuration changes to be written in the configuration files, click Submit. You will receive a reply reporting that the requested configuration changes have been completed successfully. This directive requires you to stop and start the server, as shown in Table 7-3 on page 295.

Enable pure proxy function

The WTE servers function is to be a caching and filtering proxy server. In theory, WTE could even work as a content server. However, for best performance, it is recommended that you use it only as a proxy server (no caching). In fact, by default, the initial settings instruct this component to act as a pure proxy server. You can see this if you select the Proxy Performance form from the navigation menu, as shown in Figure 7-7:

Figure 7-7 Performance settings window

300

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

We accepted the default configuration, so we did not need to restart the server for these settings.

Configuring cache function

Caching is done by locally saving copies of the files that clients request. These files may be saved in memory, to a raw disk partition, or to a disk file. WTE uses the cached files, when they are subsequently requested, to serve its clients quickly because it does not need to fetch the files from a remote Web server again. To enable the caching capabilities of WTE, you must define where the requested files will be cached. By default, the cache is defined in memory. In addition, the system memory is used to hold a cache index which reduces processing time for finding cached files. You configure the amount of memory used for the cache index. If memory caching is also configured, WTE manages both the index and the cache in the configured amount of memory. You may also configure WTE to maintain the cache in a raw disk device (unformatted partition), or in an OS file. These devices or files must be prepared using the custom formatting command htcformat.

How to use htcformat

On the AIX and Linux platforms, the htcformat executable file is automatically installed in the /usr/sbin directory. On the Windows platform, the htcformat.exe file is installed in the C:\Program Files\IBM\WTE\bin directory. This path is automatically added to the path system environment variable when WTE is installed. If you input this command in the command line, a short description of how to use it is displayed.
Example 7-4 htcformat syntax htcformat Usage: htcformat <devpath> [-blocksize <block size>] [-blocks <number blocks>] [-help] htcformat -file <filepath> [-blocksize <block size>] -blocks <number blocks> [-help]

To format the device or file, execute the command using the syntax described in Example 7-4. For a raw disk partition: The -blocksize and -blocks arguments are optional. The default block size is 8192 bytes. If you do not specify the -blocks argument, the disk partition will be filled with as many blocks as it can contain.

Chapter 7. Web Traffic Express basic configuration

301

On the Window NT platform, you must mark the cache device as unwritable. Use the disk utility to delete the device or partition you want to use, then re-create it without formatting it. This will cause the system to consider the device unavailable. To create the device in Windows NT, use the command shown in Example 7-5. The raw device path is defined as \\.\d: for d:
Example 7-5 Using Windows NT C:\>htcformat \\.\d: Are you sure you want to format \\.\d:? (y/n): y Formatting device \\.\d:, block size 8192, 524120 blocks ... Done

On the AIX platform, we first created a volume group. Then we defined a logical volume called lvwte. To create the device in AIX, use the command shown in Example 7-6. The raw device path is defined as /dev/rlvwte for /dev/lvwte.
Example 7-6 On AIX # htcformat /dev/rlvwte Are you sure you want to format /dev/rlvwte? (y/n): y Formatting device /dev/rlvwte, block size 8192, 20480 blocks ... Done

Note: On the Linux platform, caching directly to a storage device is not supported. Caches on Linux can be stored in cache file, or in RAM. For a file: The -blocksize argument is required because it determines the file size. The default block size is 8192 bytes and the -blocks argument must be at least 2049 bytes.
Example 7-7 Using Windows NT C:\>htcformat -file D:\cache\cache.wte -blocks 2049 Are you sure you want to format cache.wte? (y/n): y Formatting device cache.wte, block size 8192, 2049 blocks ... Done Example 7-8 Using the AIX platform # htcformat -file /cache/cache.wte -blocks 2049 Are you sure you want to format cache.wte? (y/n): y Formatting device cache.wte, block size 8192, 2049 blocks ... Done Example 7-9 Using the Linux platform [root@rhlinux1 wte]# htcformat -file /cache/cache.wte -blocks 2049 Are you sure you want to format cache.wte? (y/n): y Formatting device cache.wte, block size 8192, 2049 blocks ... Done

302

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

After formatting the cache devices, we can configure the cache. Select Cache Settings from Cache Configuration in the navigation pane to retrieve the Cache Settings form, as shown in the following figure:

Figure 7-8 Cache Settings window

In the Device Name field, enter the directory or file where the cache file is located. We input with the raw logical volume created on AIX.

Chapter 7. Web Traffic Express basic configuration

303

Note: The cache file is not deleted when you reinstall WTE. It can be accessed and used again when WTE is reinstalled. The cache access log file is used for logging hits on the proxy cache. This is only valid if the WTE server is running as a proxy. We stored the cache access log file in the /wte/logs/server file. Again, note that permissions are important when changing the default directory where the log files will reside. The WTE server will write to that directory as the user ID/group ID specified in the ibmproxy.conf configuration file (nobody/nobody by default). If you have created a new directory for the logs, you must ensure that the WTE servers user ID can write to that directory. For the configuration changes to be written in the configuration files, click Submit. This directive requires you to stop and start the server, as noted in Table 7-3 on page 295.

Garbage collection
Once caching is enabled, disk space and file maintenance are required. WTE provides a storage reclamation process for disk space and file management. This process should be enabled so that you can prevent the cache of your proxy server from growing beyond the maximum size that you set. Garbage collection is a process that deletes all expired files as defined by the Cache Expiration Settings form. Expired files should be the cached files that are no longer used. The WTE cache expiration is set by clicking Cache Configuration -> Cache Expiry Settings -> Cache Expiration Settings. The WTE Cache Expiration Settings form is shown in Figure 7-9. We accepted the default settings: Expire a cached HTTP file which has been unused for 2 days Expire a cached FTP file which has been unused for 3 days Expire a cached Gopher file which has been unused for 12 hours Set FTP Default Expiration time to 1 day Set Gopher Default Expiration time to 2 days Enable cached file expiration checking Disable caching of files due to expire within 10 minutes

304

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 7-9 Cache expiration settings

By default, the garbage collection process is enabled, and is performed every time the cached volume reaches the value specified in the High Water Mark field. The process stops when it reaches the value specified in the Low Water Mark field, shown in Figure 7-10.

Chapter 7. Web Traffic Express basic configuration

305

Figure 7-10 Garbage collections settings

Garbage collection provides three algorithms for choosing how files are removed from the cache. They are: bandwidth: the algorithm that optimizes network bandwidth responsetime: the algorithm that optimizes user response time blend: the algorithm that blends the two and gives you a balance of network bandwidth and user response time

306

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

When you are tuning the cache to minimize network bandwidth, larger files are given a lower priority for deletion. They are less likely to be removed during the storage reclamation. When you are tuning the cache to minimize response time, larger files are given a higher priority for deletion and, therefore, are more likely to be removed during garbage collection. The default value of the cache algorithm field is bandwidth. We chose to select blend because it gives you a balance of network bandwidth and user response time. To access the garbage collection settings form, click the Cache Configuration folder in the navigation frame. Then, select Garbage Collection Settings, as shown in Figure 7-10. Select Submit to write the configuration changes in the configuration file. For the changes to take effect, you must restart the WTE server and click Restart at the top right of the pane to restart the ibmproxy daemon, which will reread the ibmproxy.conf file.

7.1.5 Configuration of the ibmproxy.conf file

We have performed basic customization using the Configuration and Administration forms; the following example displays the directives that were changed on the ibmproxy.conf file.

Chapter 7. Web Traffic Express basic configuration

307

Example 7-10 All configured directives # Specify the protocols that this proxy Proxy http:* Proxy ftp:* Proxy gopher:* ... # PureProxy directive: PureProxy on ... # Caching directive: Caching ON ... # Logging directives CacheAccessLog /wte/logs/cache ProxyAccessLog /wte/logs/proxy ... # CacheDev directive: CacheDev /dev/rlvwte ... # CacheMemory directive: CacheMemory 16392 K ... # CacheDefaultExpiry directive: CacheDefaultExpiry http:* 0 CacheDefaultExpiry ftp:* 1 CacheDefaultExpiry gopher:* 2 ... # CacheUnused directive: CacheUnused http:* 2 days CacheUnused ftp:* 3 days CacheUnused gopher:* 12 hours ... # CacheTimeMargin directive: CacheTimeMargin 10 minutes ... # Gc (Garbage Collection) directive: Gc on ... # CacheAlgorithm directive: cacheAlgorithm blend ... # GcHighWater directive: GcHighWater 90 ... # GcLowWater directive: GcLowWater 60

server will forward:

days day days

308

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Note that the directives in the ibmproxy.conf file are not arranged in the same way as in the Configuration and Administration forms.

7.2 Tests
To test if our proxy server is working as expected, we must customize our browser and access a URL.

7.2.1 Configuring the browser

We configured Netscape Communicator 4.7 and Microsoft Internet Explorer 5 on the client machine (client IP address 9.24.105.82). Refer to Figure 7-1 on page 289 to see this configuration.

Netscape browser
To configure the Netscape browser to use our proxy server, click Edit -> Preferences and open the Advanced option. Choose Proxies and Manual proxy configuration and click View.... You will see the following window:

Figure 7-11 Netscape browser configuration

Chapter 7. Web Traffic Express basic configuration

309

We defined our AIX proxy server, rs600034, on port 80. Click OK to confirm the customizations.

Microsoft Internet Explorer browser

To configure the Microsoft Internet Explorer browser, click Tools -> Internet Options... and open the Connections tabs. Click the LAN Settings... button. You will see the following window:

Figure 7-12 Microsoft Internet Explore browser configuration

We defined our AIX proxy server, rs600034 on port 80. Click OK to confirm the customizations.

7.2.2 Testing the access

We performed the following steps to test our proxy configuration.

Client machine
We entered the URL w3.ibm.com in the browser on our client machine (IP address 9.24.105.82). The following window was displayed.

310

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 7-13 Testing access through the proxy server

Chapter 7. Web Traffic Express basic configuration

311

Proxy server
In the proxy access log (refer to Table 7-4 on page 299 to see where this log is defined), we see that the client machine sent multiple URL requests. In the Configuration and Administration forms, open the Server Activity monitor and choose Proxy Access Statistics. You will see the following window:

Figure 7-14 The proxy access statistics information

You can view information about which URLs were requested and which were satisfied from the cache. The first column contains the IP address of the machine that requested the URL.

312

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 8.

Advanced features
In this chapter, we cover details and scenarios describing some of the advanced features of Web Traffic Express. Included are some of the new features in this version of the Web Traffic Express: LDAP authentication (see 8.1.3, User authentication using LDAP on page 326) CyberPatrol filtering (see 8.1.4, Blocking URLs using the CyberPatrol plug-in on page 336) Dynamic caching (see 8.2.4, Caching of dynamic content on page 359)

Copyright IBM Corp. 2001

313

8.1 Security enhancements

In this section, we discuss features to improve the security of your environment. We show you how to restrict the access to WTE and how to hide the information that comes from Web browsers so that it is not passed on to Web servers. The following features are discussed in this section: Modifying HTTP header data - 8.1.1, Handling header information on page 314 Enable user authentication - 8.1.2, Basic user authentication on page 320 LDAP for user authentication - 8.1.3, User authentication using LDAP on page 326 CyberPatrol plug-in - 8.1.4, Blocking URLs using the CyberPatrol plug-in on page 336

8.1.1 Handling header information

When you send a request using any browser, the Web client includes HTML headers that provide additional information about the browser. Header generation occurs automatically when a request is sent. Web Traffic Express can hide or change some information from this HTML header to improve the security of your environment. In this scenario, we configure WTE to modify some of the header information passed on to the Web server. Headers contain information about the User-Agent, Client-IP and Referer. A header example is shown below:
Example 8-1 Sample HTML header User-Agent: Mozilla 4.7/WinNT Client-IP: 9.24.105.82 Referer: http://www.itso.com/index.html

Descriptions of the fields in this header are: User-Agent The value of this field provides browser and operating system information. Client-IP The value of this field provides the IP address of the client requesting the URL. Referer

314

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The value of this field provides the destination server with the URL of the link referring to this page. For this scenario and test, we used the following machines:
Table 8-1 Description of the machines Hostname m238p4yl rs600034 rs60008 IPAddress 9.24.105.82 9.24.105.61 9.24.105.59 Operating System Win NT Server 4.0 AIX 4.3.3 AIX 4.3.3 Service Web Client Caching Proxy Server Web Server

The machines are on the same network and they are all part of the itso.ral.ibm.com domain.

WEB Clients

WTE Proxy Server

Content Server IBM HTTP Server

Internet
9.24.105.59 RS60008

9.24.105.61 RS600034 9.24.105.82 M238P4YL

Figure 8-1 Environment for HTML header information testing

Chapter 8. Advanced features

315

The following example is an IP packet log captured by iptrace on the Web Server (9.24.105.59). It shows the header information of a typical HTTP request.
Example 8-2 iptrace: header information # iptrace -a -p 80 -d 9.24.105.59 -b trace1 # ipreport -Nnsr trace1 > trace1.log # pg trace1.log TCP: th_win=16060, th_sum=f954, th_urp=0 TCP: 00000000 47455420 2f204854 54502f31 TCP: 00000010 486f7374 3a20392e 32342e31 TCP: 00000020 390d0a43 6f6e6e65 6374696f TCP: 00000030 6565702d 416c6976 652c2054 TCP: 00000040 7261676d 613a206e 6f2d6361 TCP: 00000050 0a416363 6570743a 20696d61 TCP: 00000060 69662c20 696d6167 652f782d TCP: 00000070 6d61702c 20696d61 67652f6a TCP: 00000080 20696d61 67652f70 6a706567 TCP: 00000090 6167652f 706e672c 202a2f2a TCP: 000000a0 63657074 2d456e63 6f64696e TCP: 000000b0 7a69700d 0a416363 6570742d TCP: 000000c0 75616765 3a20656e 0d0a4163 TCP: 000000d0 2d436861 72736574 3a206973 TCP: 000000e0 35392d31 2c2a2c75 74662d38 TCP: 000000f0 3a206368 756e6b65 640d0a56 TCP: 00000100 48545450 2f312e30 20727336 TCP: 00000110 342e6974 736f2e72 616c2e69 TCP: 00000120 6f6d2028 49424d2d 50524f58 TCP: 00000130 452d5553 290d0a55 7365722d TCP: 00000140 743a204d 6f7a696c 6c612f34 TCP: 00000150 656e5d20 2857696e 4e543b20

2e310d0a 30352e35 6e3a204b 450d0a50 6368650d 67652f67 78626974 7065672c 2c20696d 0d0a4163 673a2067 4c616e67 63657074 6f2d3838 0d0a5445 69613a20 30303033 626d2e63 592d5754 4167656e 2e37205b 49290d0a

|GET / HTTP/1.1..| |Host: 9.24.105.5| |9..Connection: K| |eep-Alive, TE..P| |ragma: no-cache.| |.Accept: image/g| |if, image/x-xbit| |map, image/jpeg,| | image/pjpeg, im| |age/png, */*..Ac| |cept-Encoding: g| |zip..Accept-Lang| |uage: en..Accept| |-Charset: iso-88| |59-1,*,utf-8..TE| |: chunked..Via: | |HTTP/1.0 rs60003| |4.itso.ral.ibm.c| |om (IBM-PROXY-WT| |E-US)..User-Agen| |t: Mozilla/4.7 [| |en] (WinNT; I)..|

Configuring header options

Header options can be configured using the Configuration and Administration forms or by editing the WTE proxy configuration file ibmproxy.conf. We show how to configure the header using the Configuration and Administration forms. To access the header configuration form, click the Proxy Configuration folder in the navigation frame. Then, select Privacy Settings, as shown in Figure 8-2.

316

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-2 Privacy settings

In the Privacy Settings form: For security reasons, we did not select the Forward clients IP address to destination server box. This directive specifies whether the WTE proxy server should forward the requesting clients IP address to the destination server or not. If the box is checked, a header field will be generated:
Client-IP: IP_address_of_clientX

To improve client anonymity, we entered IBM_WTE_Proxy_Server in the User-agent string field. The value entered will overwrite the value sent by the client, which contained browser and operating system information.

Chapter 8. Advanced features

317

To notify users of any errors, we provided the e-mail address of the WTE administrator, [email protected], so that users can contact the WTE administrator if they encounter any errors or problems. The value entered should become the value of the From: header field. Click the Submit button to make the changes to the WTE configuration file, ibmproxy.conf. Click the Restart icon to refresh the WTE server. Once again, we collected an iptrace after the we completed the WTE header information changes. The results are listed in the example below.
Example 8-3 Header information changed # iptrace -a -p 80 -d 9.24.105.59 -b trace2 # ipreport -Nnsr trace2 > trace2.log # pg trace2.log TCP: 00000000 47455420 2f204854 54502f31 TCP: 00000010 486f7374 3a20392e 32342e31 TCP: 00000020 390d0a43 6f6e6e65 6374696f TCP: 00000030 6565702d 416c6976 652c2054 TCP: 00000040 7261676d 613a206e 6f2d6361 TCP: 00000050 0a416363 6570743a 20696d61 TCP: 00000060 69662c20 696d6167 652f782d TCP: 00000070 6d61702c 20696d61 67652f6a TCP: 00000080 20696d61 67652f70 6a706567 TCP: 00000090 6167652f 706e672c 202a2f2a TCP: 000000a0 63657074 2d456e63 6f64696e TCP: 000000b0 7a69700d 0a416363 6570742d TCP: 000000c0 75616765 3a20656e 0d0a4163 TCP: 000000d0 2d436861 72736574 3a206973 TCP: 000000e0 35392d31 2c2a2c75 74662d38 TCP: 000000f0 3a206368 756e6b65 640d0a56 TCP: 00000100 48545450 2f312e30 20727336 TCP: 00000110 342e6974 736f2e72 616c2e69 TCP: 00000120 6f6d2028 49424d2d 50524f58 TCP: 00000130 452d5553 290d0a46 726f6d3a TCP: 00000140 61646d69 6e406962 6d2e636f TCP: 00000150 7365722d 4167656e 743a2049 TCP: 00000160 54455f50 726f7879 5f536572

2e310d0a 30352e35 6e3a204b 450d0a50 6368650d 67652f67 78626974 7065672c 2c20696d 0d0a4163 673a2067 4c616e67 63657074 6f2d3838 0d0a5445 69613a20 30303033 626d2e63 592d5754 20777465 6d0d0a55 424d5f57 7665720d

|GET / HTTP/1.1..| |Host: 9.24.105.5| |9..Connection: K| |eep-Alive, TE..P| |ragma: no-cache.| |.Accept: image/g| |if, image/x-xbit| |map, image/jpeg,| | image/pjpeg, im| |age/png, */*..Ac| |cept-Encoding: g| |zip..Accept-Lang| |uage: en..Accept| |-Charset: iso-88| |59-1,*,utf-8..TE| |: chunked..Via: | |HTTP/1.0 rs60003| |4.itso.ral.ibm.c| |om (IBM-PROXY-WT| |E-US)..From: wte| |[email protected]| |ser-Agent: IBM_W| |TE_Proxy_Server.|

When we compare this result with Example 8-2 on page 316, we can see that: The From: header field is [email protected] The User-Agent: field no longer shows the browser and operating system information of the client, but the string IBM_WTE_Proxy_Server. You can also selectively block client header information using the Proxy Header Filtering form. This information is blocked on the proxy server machine and the

318

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Web server does not receive it. To access this form, click the Proxy Configuration folder in the navigation frame. Then, select Proxy Header Filtering, as shown in the Figure 8-3:

Figure 8-3 Proxy header filtering window

In Figure 8-3, we blocked User-Agent information. Click Submit to write the changes to the WTE configuration file ibmproxy.conf. Click the Restart icon to refresh the WTE server.

Chapter 8. Advanced features

319

We collected an iptrace after we blocked the User-Agent information. You can see the result in the example below:
Example 8-4 Header information without User-Agent # iptrace -a -p 80-d 9.24.105.59 -b trace3 # ipreport -Nnsr trace3 > trace3.log # pg trace3.log TCP: th_off=5, flags<PUSH | ACK> TCP: th_win=16060, th_sum=2bf3, th_urp=0 TCP: 00000000 47455420 2f204854 54502f31 2e310d0a TCP: 00000010 486f7374 3a20392e 32342e31 30352e35 TCP: 00000020 390d0a43 6f6e6e65 6374696f 6e3a204b TCP: 00000030 6565702d 416c6976 652c2054 450d0a41 TCP: 00000040 63636570 743a2069 6d616765 2f676966 TCP: 00000050 2c20696d 6167652f 782d7862 69746d61 TCP: 00000060 702c2069 6d616765 2f6a7065 672c2069 TCP: 00000070 6d616765 2f706a70 65672c20 696d6167 TCP: 00000080 652f706e 672c202a 2f2a0d0a 41636365 TCP: 00000090 70742d45 6e636f64 696e673a 20677a69 TCP: 000000a0 700d0a41 63636570 742d4c61 6e677561 TCP: 000000b0 67653a20 656e0d0a 41636365 70742d43 TCP: 000000c0 68617273 65743a20 69736f2d 38383539 TCP: 000000d0 2d312c2a 2c757466 2d380d0a 54453a20 TCP: 000000e0 6368756e 6b65640d 0a566961 3a204854 TCP: 000000f0 54502f31 2e302072 73363030 3033342e TCP: 00000100 6974736f 2e72616c 2e69626d 2e636f6d TCP: 00000110 20284942 4d2d5052 4f58592d 5754452d TCP: 00000120 5553290d 0a46726f 6d3a2077 74656164 TCP: 00000130 6d696e40 69626d2e 636f6d0d 0a0d0a

|GET / HTTP/1.1..| |Host: 9.24.105.5| |9..Connection: K| |eep-Alive, TE..A| |ccept: image/gif| |, image/x-xbitma| |p, image/jpeg, i| |mage/pjpeg, imag| |e/png, */*..Acce| |pt-Encoding: gzi| |p..Accept-Langua| |ge: en..Accept-C| |harset: iso-8859| |-1,*,utf-8..TE: | |chunked..Via: HT| |TP/1.0 rs600034.| |itso.ral.ibm.com| | (IBM-PROXY-WTE-| |US)..From: wtead| |[email protected].... |

When we compare this result with Example 8-3 on page 318, we can see that the User-Agent field does not appear.

8.1.2 Basic user authentication

In this section, we show how to configure WTE to enable user authentication. Only authorized users can access files and forms on the proxy server, as covered in 7.1.3, Web Traffic Express Administration settings on page 291. A user is required to be authenticated with a user ID and password. To configure this feature, follow these steps: Create a user and password file. To access the basic user authentication form, click the Server Configuration folder in the navigation frame. Then, select User Administration; with this option, you can add a user, delete a user, verify if the user already exists or change the password. In the following window, we will add a new user.

320

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-4 Add user

After filling in the required fields, click Submit to add the user. The following example shows the commands used to create the files required for the Add User form shown above.
Example 8-5 Commands used to create Add User form files # cd /opt/IBMWTE/usr/internet/server_root/protect # cat wteusers.group wteusers: jonh # cat wteusers.passwd jonh:WED2Ye1pOaqek:Jonh Doe

Define a new protection setup. A protection setup contains details about what to do with a request that matches a request template. To access the

Chapter 8. Advanced features

321

document protection form, click the Server Configuration folder in the navigation frame. Then, select Document Protection, as shown below in Figure 8-5.

Figure 8-5 Document protection: part 1

The URL request template field defines the name of the proxy server resource that you want to protect. We entered http:*. So, in our case, all client requests starting with http: will cause the WTE server to prompt for a user ID and password.

322

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The Named protection field defines the name of a new protection setup. We entered PROT-PROXY. We added this new protection after the PROT-ADMIN protection setup.

Note: If you want to protect the access to all the WTE servers functions, then the only directive to use would be: Protect * PROT-PROXY. When you click Submit, the following window is displayed.

Chapter 8. Advanced features

323

Figure 8-6 Document protection: part 2

This window has more directives associated with protection setup. These directives are used to specify the permissions (read, write, delete) a user or group has when manipulating the proxy server configuration and files.

324

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The Server Identifier field specifies the name that identifies the protection setup to requesters. When the proxy server sends a requester a prompt for a user ID and password, it will also include the name you specify as the value of the Server Identifier subdirective. Most browsers display this name in the prompt. In this way, the requester is capable of understanding which proxy server sent the prompt, and can decide which user ID and password to send back. The PasswdFile and GroupFile fields specify the path and name of the password file and group file to be used. In our case, we defined this subdirective as /opt/IBMWTE/usr/internet/server_root/protect/wteusers.passwd and /opt/IBMWTE/usr/internet/server_root/protect/wteusers.group. A password file contains a list of user IDs and passwords. Each user ID has one valid password defined for it. Note that the user ID in the password file does not have any relation to the addresses or host name machines of the requester, nor with the users defined in the underlying operating system. A password file is created in the proxy server machine itself. To create and maintain password files, you can access the Configuration and Administration forms, as we showed above, or you can use the htadm command, as shown in 7.1.3, Web Traffic Express Administration settings on page 291. The other fields allow you to specify how one or more of the HTTP methods are to be accessed. You can specify different access levels for GET and POST methods. We inputted All to GET and POST, so any user in the password file specified by the PasswdFile subdirective could issue a GET or POST request. You can also specify a mask. If you specify All@96.*.*.* in the GET field, then WTE will allow access to all users in the password file, but only if the request comes in from an IP address starting with 96. Any other request will be rejected. Click Submit to write the configuration changes to the configuration file. For the changes to take effect, you must restart the WTE server. With the protection setup, we received a prompt from the proxy server requesting a valid user ID and password as defined in the password file (see the following figure).

Chapter 8. Advanced features

325

Figure 8-7 Request for User ID and password for authentication

Configuration of the ibmproxy.conf file

In the following example, we show the directives that were included in the ibmproxy.conf file for the configuration described above.
Example 8-6 Directives included for the above configuration # Protection directive: Protection PROT-PROXY { GroupFile /opt/IBMWTE/usr/internet/server_root/protect/wteusers.group PasswdFile /opt/IBMWTE/usr/internet/server_root/protect/wteusers.passwd PostMask All PutMask All GetMask All AuthType Basic ServerID ProxyServer } # Protect /admin-bin/* PROT-ADMIN Protect /reports/* PROT-ADMIN Protect /Usage* PROT-ADMIN Protect http:* PROT-PROXY

8.1.3 User authentication using LDAP

Web Traffic Express provides a plug-in that supports the use of an LDAP database as a WTE registry for the storing of user authentication information. This plug-in enables the authentication of clients using an LDAP directory and user defined policy information.

326

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The following LDAP directory servers are currently supported: IBM SecureWay Directory 3.2.1 Netscape Directory Server 4.12 Novel NDS eDirectory 8.5 (not on AIX) The Configuration and Administration forms are not available for editing the WTE LDAP configuration directives. A text editor must be used to edit the LDAP configuration and policy file, pac.conf. The machines used in our scenario for testing this plug-in are described in the following table.
Table 8-2 Description of the machines Hostname m238p4xl rs600034 m238p4yl IPAddress 9.24.105.83 9.24.105.61 9.24.105.82 Operating System Win NT Server 4.0 AIX 4.3.3 AIX 4.3.3 Service IBM SecureWay Directory LDAP Server 3.2.1 Caching Proxy Server and Netscape Directory SDK C4.11 Web Client

The machines are on the same network and they belong to the same itso.ral.ibm.com domain.
LDAP Server User Authentication

WTE Proxy Server WEB Client

9.24.105.83 M238P4XL

Web Content Server

9.24.105.82 M238P4YL

9.24.105.61 RS600034

Internet

Figure 8-8 Environment for LDAP Authentication Testing

Chapter 8. Advanced features

327

Note: As of the writing of this redbook, you must install the Netscape Directory SDK for C4.11 and WTE in the same machine to run this plug-in. Four Netscape directory libraries are required to use the LDAP plug-in. You can download them from http://www.iplanet.com/downloads/developer/

Table 8-3 Required LDAP plugin libraries Platform AIX Path WTE files libpacwte.a.so libpacman.a Netscape libraries libldapssl41.so libplc3.so libplds3.so libnspr3.so libldapssl41.dll libplc3.dll libplds3.dll libnspr3.dll

/opt/IBMWTE/usr/internet/lib/plugins/pac

Windows

c:\program files\IBM\WTE\bin\plugins\pac

libpacwte.dll libpacman.dll

Linux

/usr/lib

libpacwte.so libpacman.so

On an AIX platform, symbolic links are created from /usr/lib to the /opt/IBMWTE/usr/internet/lib/plugins/pac WTE libraries. All Netscape libraries must be copied to /usr/lib.

Configuring LDAP authentication

Web Traffic Express will authenticate the users using the LDAP database. Therefore, we must add clients in our IBM SecuryWay Directory - LDAP Server. To test our scenario we defined: Country: c=us Organization: o=ibm Organization Unit: ou=ITSO Common Name: cn=Ana Mostardinha and cn=Cristiane Ferreira Username: uid=analums and uid=crismari We input two distinguishable names: dn=Ana Mostardinha,ou=ITSO,o=IBM,c=us dn=Cristiane Ferreira,ou=ITSO,o=IBM,c=us

328

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

When using LDAP as a WTE registry, all authentication is performed by the LDAP Server. You do not access the Configuration and Administration forms using the logon and password defined in the installation process of Web Traffic Express. To prevent users from accessing the Configuration and Administration forms, we recommend disabling the WTE Configuration and Administration forms. Edit the ibmproxy.conf file and comment the following lines:
Example 8-7 Disabling the Configuration and Administration forms # ===================================================================== # # # Mapping rules # # ===================================================================== # ... # Pass /admin-bin/webexec/*.style /opt/IBMWTE/usr/internet/server_root /admin-bin/webexec/*.style # Pass /admin-bin/webexec/*.NLS /opt/IBMWTE/usr/internet/server_root /admin-bin/webexec/*.NLS # Pass /admin-bin/webexec/*.props /opt/IBMWTE/usr/internet/server_root /admin-bin/webexec/*.props # Pass /admin-bin/webexec/*.class /opt/IBMWTE/usr/internet/server_root /admin-bin/webexec/*.class # Pass /admin-bin/webexec/*.gif /opt/IBMWTE/usr/internet/server_root /admin-bin/webexec/*.gif # Pass /admin-bin/webexec/*.jar /opt/IBMWTE/usr/internet/server_root /admin-bin/webexec/*.jar # Pass /admin-bin/webexec/*.html /opt/IBMWTE/usr/internet/server_root /admin-bin/webexec/*.html # Pass /admin-bin/webexec/images/* /opt/IBMWTE/usr/internet/server_root/ admin-bin/webexec/images* # Pass /admin-bin/webexec/* /opt/IBMWTE/usr/internet/server_root/admin-b in/webexec/* ... # Exec /admin-bin/* /opt/IBMWTE/usr/internet/server_root/admin-bin/* ... # ===================================================================== # # # User authentication and document protection # # ===================================================================== # ... # Protect /admin-bin/* PROT-ADMIN # Protect /reports/* PROT-ADMIN # Protect /Usage* PROT-ADMIN

Chapter 8. Advanced features

329

The LDAP plug-in configuration and policy information are defined in the pac.conf file. This file is located in the same directory as the ibmproxy.conf file, as described in Table 7-2 on page 295. The following example describes the attributes in the pac.conf file and their possible values:
Example 8-8 Terminology of the pac.conf file [PAC_MAN_SERVER] hostname: fully qualified hostname of proxy server. port: free port number to be used LDAP plugin daemon (pacd). default_policy: [grant/deny], policy for users not covered by policies below. conn_type: [ssl], to allow aal connection between pacd and LDAP Server. If you do not want an ssl connection, comment this option. [LDAP_SERVER] hostname: fully qualified hostname of the LDAP directory server. port: port used by LDAP directory server ssl_port: ssl port used by LDAP directory server admin_dn: an distinguished name with read access to the entire LDAP directory server. search_base: the root of the LDAP directory search_key: username field as defined in the LDAP directory. [PACWTE_PLUGIN] hostname_check: [true/false] allow DNS lookups. Must have DNS lookup turned on for ibmproxy to work. [POLICY] id: id to which this policy pertains. type: [0,1,2,3] LDAP entry which is a group(0), is a not group (1), IP address (2) and hostname (3). grant: [true/false] means to get access, otherwise means deny for this id. propagate: [true/false], means the access rights (grant or deny) will be propagated to its descendants or members. If the id is a group, the value of propagate will default to TRUE. stop_entry: [entry/NULL] the access right propagation will be stopped on this entry and its sub-tree of this entry, stop_entry may be a set of the entries. If the id is a group stop_entry=NULL. Excepton_entry: [entry/NULL] The access right will not be propagated on this entry,be propagated on its sub-tree or members of this entry. exception_entry may be a list of the entries. exception_entry might be applied to all type of IDs. Multiple policies are supported. If no policies exists, there is no restriction for the access permissions. [cache]

330

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

cred_cache_enable: [true/false] cred_cache_min_size: [100-64000] initial number of cred cache entries cred_cache_max_size: [100-64000] cache can grow to this ceiling cred_cache_expiration: [300-86400] seconds policy_cache_enabled: [true/false] policy_cache_min_size: [100-64000] initial number of policy cache entries policy_cache_max_size: [100-64000] cache can grow to this ceiling policy_cache_expiration: [300-86400] seconds The cache will automatically double upon reaching 75% full

We edited the pac.conf file on AIX and configured it for our scenario.
Example 8-9 Configured pac.conf file [PAC_MAN_SERVER] hostname:rs600034.itso.ral.ibm.com port:2222 default_policy:deny [LDAP_SERVER] hostname:m238p4xl.itso.ral.ibm.com port:389 admin_dn:cn=root search_base:o=ibm,c=us search_key:uid [PACWTE_PLUGIN] hostname_check:false [POLICY] type:1 id:cn=admin,ou=ITSO,o=ibm,c=us grant:true propagate:true [POLICY] type:1 id:cn=Ana Mostardinha,ou=ITSO,o=ibm,c=us grant:true propagate:true [POLICY] type:1 id:cn=Cristiana Ferreira,ou=ITSO,o=ibm,c=us grant:false propagate:true [CACHE]

Chapter 8. Advanced features

331

cred_cache_enabled:FALSE cred_cache_min_size:100 cred_cache_max_size:64000 cred_cache_expiration:86400 policy_cache_enabled:FALSE policy_cache_min_size:100 policy_cache_max_size:64000 policy_cache_expiration:86400

On the LDAP_SERVER stanza, the admin_dn is an administration user with read access of the LDAP database. In our environment, it is cn=root. The password for this user is stored in the pac_ldap.cred file. You must create this file and add to it the password admin_dn. You must also change the permission for this file to nobody. This file is located in <WTE_home>/pac/creds. Initially, this file contains a text password. However, when the LDAP plugin daemon pacd starts, it will encrypt this password. We created this file and added the password to it. After updating or changing the pac.conf file, you must restart the LDAP plugin daemon pacd. On an AIX and Linux platform, run the pacd_restart.sh command. On a Windows platform, run pacd_restart.bat. These scripts restart the daemon without interrupting WTE. To start the pacd daemon, enter the following command.
pacd -h <WTE_home> -C directory_of_pac.conf

where WTE_home is /opt/IBMWTE/usr/internet/server_root on an AIX or Linux platform and c:\Program files\IBM\WTE on a Windows platform. The WTE LDAP plugin includes three routines: 1. pacwte_auth_init()-- starts and initializes LDAP plugin daemon 2. pacwte_auth_policy()-- authenticates and checks the policies 3. pacwte_shutdown()-- kills the LDAP plugin daemon. There are four directives: 1. 2. 3. 4. ServerInit Authentication Authorization ServerTerm

The directives use the plugin routines and must be added in the API directives section of the ibmproxy.conf file. Edit the file, find, uncomment and change, if necessary, the following lines.

332

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

For an AIX platform:

Example 8-10 LDAP plugin on AIX and directive syntax # ===== LDAP Plugin ===== # ServerInit path_of_shared_library:pacwte_auth_init path_of_conf_policy_file ServerInit /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a.so:pacwte_auth_ init /etc/pac.conf # Authorization </URL> path_of_shared_libraries:pacwte_auth_policy # Authorization http://* /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a .so:pacwte_auth_policy # Authentication BASIC path_of_shared_library:pacwte_auth_policy Authentication BASIC /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a.so:pa cwte_auth_policy # ServerTerm path_of_shared_library:pacwte_shutdown ServerTerm /opt/IBMWTE/usr/internet/lib/plugins/pac/libpacwte.a.so:pacwte_shutd own

Note that we do not uncomment the Authorization directive on the AIX platform, because our scenario shows only user authentication. For a Windows platform:
Example 8-11 LDAP plugin on Windows and directive syntax # ===== LDAP Plugin ===== #ServerInit path_of_shared_library:pacwte_auth_init path_of_conf_policy_file ServerInit C:\Progra~ 1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwte_auth_init C:\Progra~ 1\IBM\WTE\pac.conf # Authorization </URL> path_of_shared_libraries:pacwte_auth_policy # Authorization http://* C:\Progra~1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwt e_auth_policy # Authentication BASIC path_of_shared_library:pacwte_auth_policy Authentication BASIC C:\Progra~1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwt e_auth_policy # ServerTerm path_of_shared_library:pacwte_shutdown ServerTerm C:\Progra~ 1\IBM\WTE\bin\plugins\pac\pacwte.dll:pacwte_shutdown

For a Linux platform:

Example 8-12 LDAP plugin on Linux and directive syntax # ===== LDAP Plugin ===== # ServerInit path_wte_libraries:pacwte_auth_init path_of_conf_police_file # ServerInit /usr/lib/libpacwte.so:pacwte_auth_init /etc/pac.conf

Chapter 8. Advanced features

333

# Authorization </URL> path_of_shared_libraries:pacwte_auth_policy # Authorization http://* /usr/lib/libpacwte.so:pacwte_auth_policy # Authentication BASIC path_of_shared_library:pacwte_auth_policy Authentication BASIC /usr/lib/libpacwte.so:pacwte_auth_policy # ServerTerm path_of_shared_library:pacwte_shutdown ServerTerm /usr/lib/libpacwte.so:pacwte_shutdown

To authorize users to access the requested URL through the defined proxy server, you must define what type of protection to use. For our scenario, all HTTP requests will be authenticated. Edit the ibmproxy.conf file, find the Protection directive and add the lines which we have highlighted in bold in the following example:
Example 8-13 Editing the ibmproxy.conf file # ===================================================================== # # # User authentication and document protection # # ===================================================================== # # Protection PROT-PROXY-LDAP { PostMask All GetMask All AuthType Basic ServerID LDAP_Authentication } Protection PROT-ADMIN { ServerId Private_Authorization AuthType Basic GetMask All@(*) PutMask All@(*) PostMask All@(*) Mask All@(*) PasswdFile /opt/IBMWTE/usr/internet/server_root/protect/webadmin.pa sswd } Protect /admin-bin/* PROT-ADMIN Protect /reports/* PROT-ADMIN Protect /Usage* PROT-ADMIN Protect http://* PROT-PROXY-LDAP

334

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Note that there is no PasswdFile field and no GroupFile field because the authentication will be done by LDAP and not by WTE. For the last step, we stopped and restarted the proxy server (described in Chapter 6, Web Traffic Express installation on page 247).

Logging
There are four levels of log files that are controlled by the environment variable, PAC_DEBUG_LEVEL. All logs are generated on two sides: one side is the client side (Web Traffic Express Server) the other side is the server side (LDAP plugin - pacd). Logs are requested as follows: The default log is the Error log file. This file is created with the name PacErr_side.date. The Audit log file is created as PacLog_side.date. To create this log, you must set PAC_DEBUG_LEVEL=2. The Socket Package log file is created as PacPac_side.date. To create this log, you must set PAC_DEBUG_LEVEL=4. The Trace log file is created as PacTra_side.date. To create this log, you must set PAC_DEBUG_LEVEL=9. side is defined as Client for the WTE logs or Server for the pacd logs. date is in the format mmmddyyyy. All logs are located in <WTE_server>/logs. Logging consumes significant resources and disk space. Set PAC_DEBUG_LEVEL=9 to receive the maximum amount of information. After defining the log variable, we restarted the LDAP plugin daemon (pacd). On AIX and Linux, run pacd_restart.sh and on Windows run pacd_restart.bat.

Chapter 8. Advanced features

335

Testing the configuration

To test our LDAP plugin configuration, we configured our browser to access the proxy server with host name RS600034, as described in 7.2.1, Configuring the browser on page 309. We made a request to the URL http://www.ibm.com. Because we defined in the ibmproxy.conf file that all http://* requests would be authenticated, the following window was displayed:

Figure 8-9 LDAP authentication window

We logged in as a user that is defined to the LDAP directory server and has access to our proxy server as defined in the pac.conf file. So the requested page was displayed. If you try to access the proxy server with an undefined user or a user that is defined in pac.conf with deny access, the following window will be displayed:

Figure 8-10 LDAP authorization failed.

8.1.4 Blocking URLs using the CyberPatrol plug-in

The CyberPatrol filtering plug-in allows Web content filtering based on the SurfControl database of acceptable and unacceptable Web sites. You can define which Web sites categories to permit or deny access to.

336

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

SurfControl maintains the database of Web sites the content of which can be subject to filtering. Two filtering methods are available. Choosing the CyberNOT method blocks access to Web sites that are listed in the negative categories. Choosing the CyberYES method allows access only to Web sites included in the positive categories. In addition, the plug-in allows you to specify additional Web sites to block or to allow. Currently, there are sixteen categories to allow access and fourteen categories to block access. Below are all categories and their hexadecimal codes listed in this database:
Table 8-4 CyberPatrol categories CyberNOT Violence/Profanity - 0x0001 Partial Nudity - 0x0002 Full Nudity - 0x0004 Sexual Acts - 0x0008 Gross Depictions -0x0010 Intolerance - 0x0020 Satanic/Cult -0x0040 Drugs/Drug Culture - 0x0080 Militant/Extremist - 0x0100 Sex Education - 0x0200 Questionable/Illegal and Gambling 0x0400 Alcohol and Tobacco - 0x0800 Sports and Entertainment - 0x1000 Search Engines - 0x8000 CyberYES Games and Toys - 0x0001 Art, Books and Music - 0x0002 Movies and TV- 0x0004 Outdoors and Sports- 0x0008 Oceans and Space- 0x0010 Pets, Animals and Dinosaurs- 0x0020 Other Interesting Stuff- 0x0040 Vacation and Travel- 0x0080 Puzzles and Hobbies- 0x0100 School Work- 0x0200 Volunteer and Help- 0x00400 Kids- 0x0800 Teens- 0x1000 Reference Materials - 0x2000 Schools on the Net- 0x4000 Parents and Teacher- 0x8000

The precompiled encrypted URL lists are the heart of the CyberPatrol filter. You can find more information about filtering categories on the SurfControl Web site, http://www.surfcontrol.com.

Chapter 8. Advanced features

337

The SurfControl database is delivered with the WTE product and installed automatically. The CyberLists, memory loadable databases, registration data and communication logs are stored in <WTE_server>/filter/cyberpatrol on the AIX platform and <WTE_server>\bin\plugins\filter\cyberpatrol on the Windows platform. The database files should be periodically updated to maintain a current list of Web sites that SurfControl has ranked. One year of free database updates is included with Web Traffic Express. There are some considerations to remember when using this plug-in: Configuration overhead Some predictable performance impact on the proxy. However, all the domain names, IP addresses and directory names in the memory loadable databases are hashed, thus providing fast lookup, and the performance impact is not so severe. A major benefit is that this feature can be used to block URLs that are inappropriate for your specific workplace environment. You have access control and a reduction in network traffic.

Configuration
First, you must configure WTE to consult the CyberPatrol filtering plug-in. Edit the ibmproxy.conf file (this file is located as described on Table 7-2 on page 295). Find and uncomment the following directives: For an AIX platform:
Example 8-14 CyberPatrol API on AIX # ===== CyberPatrol Plugin ===== ServerInit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfi lterInit PreExit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfilte rMain

For a Windows platform:

Example 8-15 CyberPatrol API on Windows ===== CyberPatrol Plugin ===== ServerInit C:\Progra~1\IBM\WTE\bin\plugins\filter\cyberpatrol\cpfilter.dll: cpfilterInit PreExit C:\Progra~1\IBM\WTE\bin\plugins\filter\cyberpatrol\cpfilter.dll: cpfilterMain

338

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

For a Linux platform:

Example 8-16 CyberPatrol API on Linux # ===== CyberPatrol Plugin ===== ServerInit /usr/lib/libcpfilter.so:cpfilterInit PreExit /usr/lib/libcpfilter.so:cpfilterMain

Next, from the Configuration and Administration forms pane, click Plugin Configuration -> CyberPatrol Filtering. You will see the following window:

Figure 8-11 CyberPatrol filtering: part 1

Chapter 8. Advanced features

339

We checked the Enable CyberPatrol Filtering option (the box is unselected by default). You can specify when this filtering will be running. By default, it runs all the time. In the next configuration section, you choose what kind of filtering method to use. The CyberNOT setting denies access to specified Web sites, while the CyberYES setting allows access only to specified Web sites. We chose the CyberNot option and selected the Search Engine Queries box, as shown in Figure 8-12.

340

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-12 CyberPatrol filtering: part 2

For the configuration changes to be written to the configuration files, click Submit. The filter database must be updated based on our selection of categories to filter. Click the Update Filter Files button at the bottom of the configuration page to perform this function. This configuration-specific filter database is loaded with WTE when it starts. Each time you change the filtering method (CyberNOT or CyberYES) or select different categories, you must rebuild the proxys database.

Chapter 8. Advanced features

341

Note: If you did not create a separate filesystem to install WTE on the AIX platform, you may need to increase your root filesystem by 30 MB. When the process ends, you see the following window:

Figure 8-13 CyberPatrol filtering: part 3

These directives require you to stop and start the server, as shown in Chapter 6, Web Traffic Express installation on page 247.

342

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

. Note: You can use the command line to update the filter database. Issue the GenCyberDB command from the /usr/sbin directory for AIX, or the <install_path>\bin\plugins\filter\cyberpatrol directory for Windows.

Testing the configuration

To test this feature, we made a request to the URL http://www.altavista.com in our browser which was configured as described in 7.2.1, Configuring the browser on page 309. This URL leads to a search engine home page and it will be denied access, as shown in Figure 8-14.

Figure 8-14 Denied access window

Chapter 8. Advanced features

343

Additional sites to filter

You can add other URLs or keywords to filter, but these work together with the type of filtering you have selected, so if you chose CyberNOT, then additional sites and keywords can be blocked. If you chose CyberYES, additional sites and keywords will also be allowed. The following window shows this configuration:

Figure 8-15 Adding to CyberPatrol filtering

We added the URL http://www.dilbert.com. This page will now be denied because we are using the CyberNOT option. For the configuration changes to be written in the configuration files, click Submit. Do not forget to recreate the database by clicking the Update Filter Files button. Stop and start the server as shown in Chapter 6, Web Traffic Express installation on page 247.

344

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

When trying to access this site, we received a window similar to the one shown in Figure 8-14.

Downloading the CyberPatrol database

To update the CyberPatrol database, you must register with SurfControl. There are two links on the Configuration and Administration forms for these functions. One is for registering and the other one is for downloading the database. Refer to the upper right part of Figure 8-11 on page 339 for the location of these links. We used the Microsoft Internet Explorer browser to register and download the CyberPatrol database, because as of the writing of this redbook, there is a problem with the Netscape browser. The fix which will allows you to use the Netscape browser is being built and you can download it from the Web site http://www.ibm.com/software/webservers/edgeserver/. To access the registration forms, we clicked the link here in the text: If you have not registered with SurfControl to receive updated CyberLists databases, click here to register in Plugin Configuration -> CyberPatrol Filtering in the Configuration and Administration forms (refer to Figure 8-11 on page 339). The following window was displayed:

Chapter 8. Advanced features

345

Figure 8-16 CyberPatrol proxy registration form

We filled out our information and clicked Submit. When the registration process finished, we were shown the following window:

346

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-17 Registration forms: successful registration window

One file was created called registration.data which contained all our information. We then updated the CyberPatrol database. We clicked the link here in the text: If you already have subscribed for database updates, click here to download updated CyberLists from SurfControl in Plugin Configuration -> CyberPatrol Filtering in the Configuration and Administration forms. The following window was shown:

Chapter 8. Advanced features

347

Figure 8-18 CyberPatrol download window

We clicked the Download button. After this process finished, we were shown the following window:

348

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-19 CyberList download window

You can automatize this process if you have sucessfully registered yourself as described above. To download the database, use the command cpreg -d. After the download has completed, it is necessary to regenerate the memory database. Run the GenCyberDB command. Below we show an example for each platform:

Chapter 8. Advanced features

349

On AIX platforms, you can set the crontab file:

Example 8-17 Updating the crontab file on an AIX platform # crontab -e add the following two lines: 00 01 * * * 6 (cd /opt/IBMWTE/usr/internet/server_root/filter/cyberpatrol/; ./cpreg -d) >/dev/null 2>&1 00 02 * * 6 (cd /usr/sbin/; ./GenCyberDB) > /dev/null 2>&1 Every Saturday at 01:00am the download CyberList will be done, and at 02:00 the GenCyberDB command will run to generate the memory loadable database.

On Windows platforms, you can set Schedule Tasks:

Example 8-18 Configuring Schedule Tasks on a Windows platform To schedule the CyberList download: From the START menu choose Settings-> Control Panel -> Scheduled Tasks -> Add Scheduled Task. The Scheduled Task Wizard window will be displayed. Click the Next button to continue the process. Click Browse on the next window. Find C:\Program Files\IBMWTE\bin\plugins\filter\cyberpatrol\cpreg and click Open. Select the weekly radio button then click Next. Select 01:00am and Saturday. On the last window, you must select Open advanced properties for this task when I click Finish option. Click the Finish button. Another panel will be displayed. In the Run box, add -d at the end. The entry will look like this: C:\Program Files\IBMWTE\bin\plugins\filter\cyberpatrol\ cpreg.exe -d. Click OK button. To schedule the update Filer Files: Perform the same steps as for the download with the following changes: Change the time to 2:00. Change the binary is GenCyberDB Do not add the -d option the end of command.

On Linux platforms, you can set the crontab file:

Example 8-19 Updating the crontab file on a Linux platform # crontab -u root -e add the following two lines: 00 01 * * * 6 (cd /opt/IBMWTE/usr/internet/server_root/filter/cyberpatrol/; ./cpreg -d) >/dev/null 2>&1 00 02 * * 6 (cd /usr/sbin/; ./GenCyberDB) > /dev/null 2>&1 Every Saturday at 01:00am the download CyberList will be done, and at 02:00 the GenCyberDB command will run to generate the memory loadable database.

350

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Logging
One file, dnload.log, is created when you send the registration form. This file contains all logs of the registration and download steps. This file is stored in /opt/IBMWTE/ usr/internet/server_root/filter/cyberpatrol/ on AIX and Linux platforms and in C:\Program Files\IBM\WTE\bin\plugins\filter\cyberpatrol\ on Windows platforms. There is also a log for the CyberPatrol Filtering Engine. The environment variable, FILTER_DEBUG_ON is used to control this log. This variable switches the verbose mode on and off. The log is written to the stderror.Mmmddyyyy file.

Configuration of the ibmproxy.conf file

In Example 8-20, we show the directives that were updated in the ibmproxy.conf file to configure the CyberPatrol plug-in.
Example 8-20 All configured CyberPatrol directives # ===== CyberPatrol Plugin ===== ServerInit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfil terInit PreExit /opt/IBMWTE/usr/internet/lib/plugins/cyberpatrol/libcpfilter.o:cpfilter Main # # Cyber Patrol Filter directives: FilterCyberPatrolOnOff on FilterActivePeriodOneFrom 00:00 FilterActivePeriodOneUntil 24:00 FilterActivePeriodTwoFrom 00:00 FilterActivePeriodTwoUntil 00:00 FilterActiveWeekDays Sunday Monday Tuesday Wednesday Thursday Friday Saturday FilterCyberPatrolList CyberNOT FilterCyberPatrolRuleMask 0x8000 FilterAdditionalURL www.dilbert.com

8.2 Cache content management

Most network managers that use a caching function to minimize network bandwidth costs want to get the best hit rate possible. That is, they want to increase the probability that user requests can be satisfied using content within the cache versus using a network request.

Chapter 8. Advanced features

351

There are several decisions an administrator needs to make: Which documents are kept in the cache? How many documents can be cached? How long are they considered current? When is the cache refreshed? In this section, we show how to configure those features using the Configuration and Administration forms.

8.2.1 Controlling which documents are cached

You can specify which documents should be cached, how long they should be cached, and which documents should never be cached. To define which documents should and should not be cached, choose Cache Configuration -> Cache filters. You will see the following window:

352

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-20 Caching filters

Chapter 8. Advanced features

353

8.2.2 Cache freshness

It is important that the WTE administrator ensure that cached documents are consistent with the original documents located at the originating Web server. For each document that has been cached, WTE computes a time at which the document will expire: For HTTP documents, the header of the document generated by the Web server contains the expiration information. For FTP documents, WTE generates its own Last-Modified: header information to compute expiration times. The Web server can indicate the expiration time in several ways when entering header information in the HTTP response. The permitted header information is in the following order of preference: The Web server can specify how long the document is valid after it has been received. The Web server can specify the exact time at which the document should be considered expired. The Web server can indicate the time when a document was last modified. In this case, the caching and filtering proxy server performs a sequence of operations based on a class of parameters specified in the configuration file and calculates how long the document will be valid. When a document is found in the cache, but has expired, WTE issues a special request to the Web server. This request is called if-modified-since. The Web server sends back the document to the caching and filtering proxy server only if the document has been modified since it was last received by the proxy. Otherwise, the Web server only sends a message indicating that the document has not been modified, and does not send the entire file. At this point, the caching and filtering proxy server can serve the page to the client. The settings that determine expiration time for cached files are controlled by the Cache Expiry Settings in the Configuration and Administration forms. Choose Cache Configuration -> Cache Expiry Settings. You will see the following options: URL Expiration: Allows you to set a default expiration time for cached files based on theirs URLs. Cache Expiration Settings: Define how long WTE keeps unused cached files in the cache. Different values can be set for HTTP, FTP, and Gopher files. Last modified Factor: Calculates an expiration date for cached files with no expiration dates in their headers.

354

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Time Limit for cached files: Defines how long WTE keeps cached files in the cache. Time limits are set based on request templates, and you can specify files that should be discarded as well as files that should be revalidated when the time limit has been reached.

8.2.3 Automatic cache refreshing

Typically, the most common proxy servers cache a particular page only after a user requests it. WTE, in addition to this default caching, has a cache agent that provides automatic caching and gives more control to the administrator. The cache agent can retrieve specified URLs even before they are effectively requested and refresh the cache automatically. The cache refresh takes place when the proxy server activity is low (by default, every night at midnight, local time) and all the retrieved pages are ready in the cache to provide faster service, even if it is the first time that a user requests them. Automatic cache refreshing has two sources it can use to refresh the cache: It can load specific URLs defined by the administrator. In this way, the administrator can specify a certain set of pages that must be loaded by the cache agent when it starts. It can load the most popular URLs from the previous days activity. To obtain these, the cache agent checks the cache access log, sorts it by frequency of requests and then picks the most popular pages. It can refresh the top number of pages as specified by the administrator. The cache agent can use both sources of input. Optionally, the cache agent can follow a specified level of HTML links on the pages it is loading and cache all of those linked pages. This operation is also known as delving. It is not necessary that the linked pages reside on the same host as the source page; the cache agent can retrieve them even if they reside on other hosts. The cache agent offers a very useful service: caching is performed even before cached pages are effectively requested, so that the average response time is minimized. Moreover, the cache is built before user activity increases, typically at night. However, turning on the cache agent forces the caching and filtering proxy server machine to be busy even during hours of low activity. Moreover, configuring the cache agent to perform delving requires giving more control to the caching and filtering proxy server administrator. For example, disabling delving from high-level pages, such as Web indexes or search sites, is recommended; otherwise, multiple requests for large numbers of pages will be generated.

Chapter 8. Advanced features

355

How to set up automatic cache refreshing

The following figures demonstrate how to set up a cache refresh agent using the Configuration and Administration forms. Follow these steps: You can specify the URLs to load. Choose Cache Configuration -> Cache Preload. The window shown below is displayed:

Figure 8-21 Cache Preload window

By default, the cache daily agent is enabled to run everyday at 3:00 AM. You specify which URLs to load with the option cache in the Cache status field. You can also exclude URLs from being preloaded with the option ignore in the same field. We used the URL http://www.uol.com for this test. Click Submit to update the proxy configuration file. This directive requires you to stop and start the server, as shown in Table 7-3 on page 295.

356

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

To configure other cache agent options and the number of popular URLs to load, choose Cache Configuration -> Cache refresh. The following window is displayed:

Figure 8-22 Cache Refresh window

Chapter 8. Advanced features

357

To configure delving, select the Load linked pages field. You use a menu to configure how to apply to delve. To delve for all pages, choose always. For no pages, choose never. For administrator-specified pages only, choose admin. To delve for links within documents served only from the cache, choose topn. We used the default configuration, so we did not need to restart the server for these settings. The cache agent loads and then refreshes the cache in the following order: Specific pages the administrator has specified Popular pages from the cache access log Additional pages, by following links from the pages that have already been loaded; these are retrieved if the maximum number of pages has not been reached.

Starting cache refreshing manually

When the cache refresh is enabled, the cache refresh agent is automatically run at the specified time. However, you can run the cache agent manually at any time. In AIX, the cache refresh agent executable is cacheagt. If you input this command on the screen, you receive a small description on how to use it.
Example 8-21 Cache agent syntax # ./cacheagt -Usage: cacheagt [ options ... ] where options include: -r <proxy config file> -vv <verbose mode>

The manual cache refresh agent commands by platform are listed in the following table.
Table 8-5 Cache agent commands Platform AIX 4.3.3 Windows Linux command /usr/sbin/cacheagt c:\program files\ibm\wte\bin\cacheagt.exe /usr/sbin/cacheagt

358

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

8.2.4 Caching of dynamic content

The JavaServer Pages (JSP) and servlet caching function enables Web Traffic Express to cache JSP and servlet results generated from a WebSphere Application Server. WTE pulls the JSP results from the application servers dynamic cache, which is configured with a WTE caching proxy plug-in module. The main advantages WTE offers by caching dynamically generated content are: Reducing the load on the content hosts Reducing the load on the application servers Speeding the delivery of requested resources to end users The ability to dynamically cache JavaServer Pages and servlets was introduced in WebSphere Application Server V3.5 with PTF3. WAS uses an in-memory cache to improve performance. The dynamic cache function of WAS is enabled if the following file is present within WAS: /WebSphere/AppServer/properties/dynacache.xml A specific servlet is enabled for caching by adding an entry for it in: /WebSphere/AppServer/properties/servletcache.xml Web Traffic Express communicates with WebSphere Application Server using an external cache adapter (ECA) plug-in provided with WTE. The following figure illustrates an overview of the dynamic caching support included in WTE and its relationship with the WAS dynacache.

Client

WTE
Initial Request

Response
Check for cache control header

S W e e r b v e r

WAS
JSPs Servlets

dynacache
WTE External Cache Adapter

cache
Invalidate cache entries

Figure 8-23 WTE dynamic caching with WAS

Chapter 8. Advanced features

359

Currently, the external cache interface exports only fully composed public pages (cannot use cookies in the session information). Private pages are not cached by the proxy. Note: Dynacache functionality is supported in WebSphere Application Server 3.5 with PTF3 applied. The following table lists a matrix of WebSphere Application Server and Web Traffic Express dynacache compatibility.
Table 8-6 WAS and WTE dynacache support WAS version WTE V3.6 (WSES V1.0.3) WTE V3.6.01 WTE V3.6.0.1 plus fix V3.5.3 Dynacache support Dynacache support NO dynacache support V3.5.4 and V4.0 NO dynacache support NO dynacache support Dynacache support

Configuring WTE and WAS for dynamic caching

The Configuration and Administration forms are not available for configuring this feature. A text editor must be used to edit the ibmproxy.conf, dynacache.xml and servletcache.xml files. We used the following configuration for our scenario:

360

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

9.24.105.61 RS600034

9.24.105.39 M23KK904

WTE Proxy Server

WAS Server

WEB Clients
9.24.105.82 M238P4YL
Figure 8-24 Dynamic caching scenario

We followed these steps to configure WTE dynamic cache support. First, we created a Web application called ITSO in the WebSphere Application Server. We included two example servlets: CalcServlet - calculates the timestamp and calls TimeServlet TimeServlet - calculates the timestamp CalcServlet is defined as cacheable, while TimeServlet is not. Using these servlets, we can have two different timestamps displayed at the same moment. The following window shows our WebSphere Application Server configuration.

Chapter 8. Advanced features

361

Figure 8-25 WebSphere Application Server configuration

The source code for TimeServlet.java and CalcServlet.java can be found in Appendix A, Java source code on page 475 The dynamic caching function of Web Traffic Express includes two JavaBean components: WteAdapter.class and WteMetaDataGeneratorImpl.class, located in <WTE_root>/classes/com/ibm/wte. You must copy these class files into <WAS_root>/properties. We used FTP to send these .class files to the WAS server. The example below shows the steps to follow:
Example 8-22 Transferring necessary files c:> hostname M23KK904 c:> cd \WebSphere\AppServer\properties c:\WebSphere\AppServer\properties> ftp rs600034 Connected to rs600034.itso.ral.ibm.com.

362

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

220 rs600034 FTP server (Version 4.1 Thu Apr 6 14:11:28 CDT 2000) ready. Name (rs600034:root): root 331 Password required for root. Password: 230 User root logged in. ftp> cd /opt/IBMWTE/usr/internet/classes/com/ibm/wte 250 CWD command successful. ftp> bin 200 Type set to I. ftp> mget *.class mget WteAdapter.class? y 200 PORT command successful. 150 Opening data connection for WteAdapter.class (6550 bytes). 226 Transfer complete. 6550 bytes received in 0.001107 seconds (5778 Kbytes/s) local: WteAdapter.class remote: WteAdapter.class mget WteMetaDataGeneratorImpl.class? y 200 PORT command successful. 150 Opening data connection for WteMetaDataGeneratorImpl.class (1682 bytes). 226 Transfer complete. 1682 bytes received in 0.000357 seconds (4601 Kbytes/s) local: WteMetaDataGeneratorImpl.class remote: WteMetaDataGeneratorImpl.class ftp> quit 221 Goodbye.

The WebSphere Application Server communicates with WTE using an external cache adapter (ECA) supplied with WTE (WteAdapter.class file FTPed to the WAS in the previous step). On the WebSphere Application Server we must edit dynacache.xml and add an externalCacheGroup entry to enable the WTE proxys ECA. WAS reads the dynacache.xml file at startup. WAS provides an example file named dynacache.sample.xml to help identify how to add plug-in entries. The following table describes where this sample file is located.
Table 8-7 Sample dynacache.xml file location Platform AIX Windows path \usr\WebSphere\AppServer\properties c:\websphere\properties

We copied this file to a new file called dynacache.xml. We then edit this file and add the WTE externalCacheGroup plug-in entry.
Example 8-23 Updated dynacache file <?xml version="1.0"?> <cacheUnit>

Chapter 8. Advanced features

363

<cache size="1000" priority="1" /> <externalCacheGroups> <group id="IBM-WTE-ITSO" > <member address="rs600034.itso.ral.ibm.com" adapterBeanName="com.ibm.wte.WteAdapter" /> </group> </externalCacheGroups> </cacheUnit>

Note: One application server can be configured to serve multiple proxy servers simply by adding additional group id statements. To enable WAS dynamic caching of servlets, you must have entries for them in the servletcache.xml file on the WebSphere Application Server. In addition, for Web Traffic Express to cache a servlets results, the servlet must be marked as externally cacheable in the servletcache.xml file. The servletcache.xml file is located in the same directory as the dynacache.xml file described in Table 8-7 on page 363. There is an example file called dynacache.sample.xml to help configure this entry. We copied this file to a new file called serveltcache.xml, edited it, and input the servlet entries. The following example lists our file:
Example 8-24 serveltcache.xml file <?xml version="1.0" ?> <!DOCTYPE servletCache (View Source for full doctype...)> <servletCache> <servlet> <metagenerator class="com.ibm.wte.WteMetaDataGeneratorImpl" /> <externalcache id="IBM-WTE-ITSO" /> <path uri="/webapp/ITSO/CalcServlet" /> <timeout seconds="100" /> </servlet> </servletCache>

Edit the WTE ibmproxy.conf file (this file is located as described in Table 7-2 on page 295). Find and uncomment the following directives which enable Web Traffic Express to receive invalidate messages sent by the WebSphere Application Server: For an AIX platform:
Example 8-25 JSP caching function on AIX # ===== JSP Plugin ===== Service /WES_External_Adapter /opt/IBMWTE/usr/internet/lib/plugins/dynacache/ libdyna_plugin.o:exec_dynacmd

364

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

For a Windows platform:

Example 8-26 JSP caching function on Windows # ===== JSP Plugin ===== Service /WES_External_Adapter C:\Progra~1\IBM\WTE\bin\plugins\dynacache\ dyna_plugin.dll:exec_dynacmd

For a Linux platform:

Example 8-27 JSP caching function on Linux # ===== JSP Plugin ===== Service /WES_External_Adapter /usr/lib/libdyna_plugin.so:exec_dynacmd

For a Solaris platform:

Example 8-28 JSP caching function on Solaris # ===== JSP Plugin ===== Service /WES_External_Adapter /opt/IBMWTE/usr/internet/lib/plugins/dynacache/libdyna_plugin.so:exec_dynacmd

Add a directive for each external cache manager in the ibmproxy.conf file. The syntax is as follows:
ExternalCacheManager <ExternalCacheManager ID> <MaxExpiryTime>

The ExternalCacheManager ID must match the ID set in the externalCacheGroups: group id attribute in the dynacache.xml config file in the WAS (refer to Example 8-23 on page 363). The MaxExpiryTime is the default expiration time set for resources cached on behalf of the external cache manager. If the external cache manager fails to invalidate a resource within the specified time, the resource will expire automatically in the specified time. We edited the ibmproxy.conf file and added the following line:
ExternalCacheManager IBM-WTE-ITSO 20 minutes

WTE will classify all the cached entries as expired in 20 minutes. WTE caches only contents from a WAS that has an ExternalCacheManager matching that which is specified in the ibmproxy.conf file. The WebSphere Application Server sends an invalidate message to all Web Traffic Express servers defined in the externalCacheGroup entry in the dynacache.xml file. When the JSP or servlet results are invalidated, WTE clears this invalid entry from its cache. Entries in the dynamic cache can be invalidated when: The dynamic cache garbage collector removes an entry during cache congestion The timeout set in the servlet entry expires

Chapter 8. Advanced features

365

 An external agent or application invokes the dynamic cache API to invalidate cache entries. Stop the WTE proxy server and restart it again, as described in Chapter 6, Web Traffic Express installation on page 247. The following considerations should be kept in mind when using this feature: Dynacache will export only fully cacheable pages. Pages must be public. Invalidate messages delivery is not reliable. No secure connection for invalidates.

Testing the configuration

To test our configuration, we customized the Web clients browser (9.24.105.82) to access the Web Traffic Express proxy server (RS600034), as shown in 7.2.1, Configuring the browser on page 309. We made some requests to http://9.24.105.39/webapp/ITSO/CalcServlet. We receive the following window in response:

Figure 8-26 Timestamp window

366

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The CalcServlet was cached while TimeServlet was reexecuted.

8.3 SOCKS client support

Flexible-client SOCKS allows the caching and filtering proxy server to reside behind a firewall or SOCKS server without sharing the same physical machine with this firewall or SOCKS server. With this feature, the WTE server does not have to forward all the requests to the SOCKS server. On the contrary, requests for Web content located in the intranet can be routed directly to the destination content server. This way, workload on the SOCKS server is dramatically reduced, and the end user will experience a better response time. The following two figures illustrate the benefit of using a flexible-client SOCKS WTE proxy server.

Web Server SOCKS/ Firewall WEB Client

Intranet

Internet

Proxy Server

Figure 8-27 Client intranet access without flexible-client SOCKS support

Chapter 8. Advanced features

367

Web Server SOCKS/ Firewall WEB Client

Intranet

Internet

Proxy Server

Figure 8-28 Client intranet with flexible-client SOCKS support

The flexible-client SOCKS enhancement implemented in WTE improves security by allowing a firewall server to be isolated from the proxy server. It reduces the load on the firewall, since the internal requests are handled by the proxy server. Moreover, the administrator can easily specify which IP addresses or domains should be contacted directly by the proxy server and which ones should be contacted through the SOCKS server. Note: WTE is a SOCKS4 client and does not include a SOCKS server. A SOCKS server is included in the Tivoli SecureWay Firewall. If your system does not already have a SOCKS configuration file, a default SOCKS configuration file will be installed with Web Traffic Express.
Table 8-8 Location of the SOCKS configuration file Operating System Windows NT 4.0 AIX 4.3.3 Linux Red Hat 6.2 Location c:\program files\ibm\wte\socks.cnf /etc/socks.conf (the file in this directory is a link to /opt/IBMWTE/usr/internet/etc/en_US/etc/socks.conf.exp) /etc/socks.conf

8.3.1 How to set up flexible-client SOCKS

We will use the following configuration for our scenario:

368

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

9.24.105.59 RS60008

9.24.105.61 RS600034

Web Server

WTE Proxy Server SOCKS/ Firewall

Internet traffic

Intranet Traffic

Intranet

Internet

Web Server

WEB Clients
9.24.105.82 M238P4YL

socks.itso.ral.ibm.com

Figure 8-29 Flexible-client SOCKS scenario

We will detail the steps to set up a flexible-client SOCKS environment for our network configuration, using the Configuration and Administration forms: Configure the direct (intranet) connections. Use direct connections for requests that should not pass through the SOCKS server. Choose Proxy

Chapter 8. Advanced features

369

Configuration -> SOCKS Configuration -> Direct Connections. We see the window shown in Figure 8-30:

Figure 8-30 Direct connections configuration

We input the IP address 9.24.105.0, Subnet Mask 255.255.255.0; we set Port to Equal to and entered Port Number 80. Therefore, all HTTP (port 80) traffic within this network will have direct connections. Click Submit to update the SOCKS configuration file. Configure SOCKS (Internet) connections. Use SOCKS connections for requests that should pass through the SOCKS server. Choose Proxy

370

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Configuration -> SOCKS Configuration -> SOCKS Connections. We see the window shown in Figure 8-31.

Figure 8-31 SOCKS connections configuration

Chapter 8. Advanced features

371

We input the IP address 0.0.0.0, Subnet Mask 0.0.0.0, we set Port to All ports and we entered the SOCKS server host name socks.itso.ral.ibm.com. Therefore, traffic requests for all the other networks will pass through the SOCKS server. Click Submit to update the SOCKS configuration file. Click the Restart icon to restart the WTE server.

8.3.2 Testing the configuration

To test if our environment is configured correctly, we customized the Web clients browser (9.24.105.82) to access the Web Traffic Express proxy server (RS600034), as described in 7.2.1, Configuring the browser on page 309. We performed an intranet access request and an Internet access request. For intranet access, we entered the URL http://RS60008/WSsamples/index into the Web clients browser. It returned the WAS index home page. The following is a log captured by the tcpdump command on the Web Traffic Express proxy server machine, showing the communication between machines:
Example 8-29 Intranet access # tcpdump -n tcp tcpdump: listening on en0 11:18:59.788124229 9.24.105.82.3077 > 9.24.105.61.80: P608128506:608128867(361) ack 3979850168 win 7334 (DF) 11:18:59.789552182 9.24.105.61.32957 > 9.24.105.59.80: S4235327931:4235327931(0 ) win 16384 <mss 1460> 11:18:59.790066792 9.24.105.59.80 > 9.24.105.61.32957: S2814790083:2814790083(0 ) ack 4235327932 win 16060 <mss 1460> 11:18:59.790150186 9.24.105.61.32957 > 9.24.105.59.80: . ack 1 win 16060 11:18:59.790298159 9.24.105.61.32957 > 9.24.105.59.80: P 1:402(401) ack 1 win16 060 11:18:59.792816445 9.24.105.59.80 > 9.24.105.61.32957: P 1:1299(1298) ack 402 win 16060 11:18:59.793818765 9.24.105.61.80 > 9.24.105.82.3077: P 1:1389(1388) ack 361 win 16060

The captured trace data shows that all traffic from the Web client browser (9.24.105.82) goes through the WTE proxy server (9.24.105.61) and is answered by the intranet Web Server(9.24.105.59). This verifies that the 9.24.105.0 networks traffic was not sent to the SOCKS server.

372

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

For Internet access, we entered the URL http://www.cnn.com on the Web clients browser. It returned the CNN main page. The following is a log captured by tcpdump on the Web Traffic Express proxy server.
Example 8-30 Internet access # tcpdump -n tcp 13:33:53.208493801 9.24.105.82.3198 > 9.24.105.61.80: P616538424:616538864(440) ack 4260219018 win 7370 (DF) 13:33:53.209958470 9.24.105.61.33215 > 9.14.3.72.1080: S1966035796:1966035796(0 ) win 16384 <mss 1460> (DF) 13:33:53.281441233 9.14.3.72.1080 > 9.24.105.61.33215: S2404853300:2404853300(0 ) ack 1966035797 win 16060 <mss 1460> 13:33:53.281529391 9.24.105.61.33215 > 9.14.3.72.1080: . ack 1 win 16060 (DF) 13:33:53.281630783 9.24.105.61.33215 > 9.14.3.72.1080: P 1:9(8) ack 1 win 16060 (DF) 13:33:53.385213417 9.24.105.61.80 > 9.24.105.82.3198: . ack 440 win 16060 13:33:53.496049862 9.14.3.72.1080 > 9.24.105.61.33215: . ack 9 win 16060 13:33:53.496122573 9.24.105.61.33215 > 9.14.3.72.1080: P 9:13(4) ack 1 win16060 (DF)

From data captured, we see that the request destined for the Internet Web server is sent by the WTE proxy server (9.24.105.61) to the SOCKS server (9.14.3.72).

8.4 Proxy auto-configuration

Web Traffic Express supports automatic proxy configuration, a technology that matches a feature implemented in Netscape Navigator 2.0 or later and Microsoft Internet Explorer Version 4.0 or later. Automatic proxy configuration implies that the client browser does not need to be configured to point to a specific proxy or SOCKS server, as long as it points to a proxy automatic configuration (PAC) file. The advantage of this feature is that it allows the server administrator to update only this file if there is any change to the proxy or SOCKS server configuration. Client browsers need not be reconfigured. This feature can also be used to reroute requests when servers are down, to balance workload or to send specific URLs to specific proxies. Automatic proxy configuration is a browser function that enables more dynamic server selection. A PAC file is a JavaScript file that consists of functions called by the client browser before each URL retrieval. The function will return values indicating whether a proxy server, a SOCKS server, or a direct connection will be used to service the request. This file can also redirect the request if the initial connection to be used is down.

Chapter 8. Advanced features

373

Note: The PAC file is reloaded when a browser is restarted.

8.4.1 How to write a PAC file

The proxy automatic configuration file must be named xxxx.pac and defines the FindProxyForURL function:
Example 8-31 PAC file syntax function FindProxyForURL(url, host) { ... }

We will define a proxy automatic configuration file for the following scenario (the machines are described on Table 7-1 on page 288).
9.24.105.83 M238P4XL 9.24.105.61 RS600034

WTE Proxy Servers

9.24.105.39 M23KK904

Intranet
9.24.105.63 RHLINUX1

WEB Clients

9.24.105.82 M238P4YL

Internet

Firewall Firewall

Web Server

Figure 8-32 Autoproxy scenario

The following factors are defined in our scenario:

374

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The client browser with IP address 9.24.105.39 (M23KK904) must access the intranet/Internet through the proxy server with IP address 9.24.105.83 (M238P4XL) as first option. The client browser with IP address 9.24.105.82 (M238P4YL) must access the Intranet/Internet through the proxy server with IP address 9.24.105.61 (RS600034) as first option. The proxy server with IP address 9.24.105.63 contains the autoproxy file configuration and will be the backup proxy server. The script shown in the following example will instruct the browsers to route requests as described above.
Example 8-32 File autoproxy.pac function FindProxyForURL(url, host) { hostip=myIpAddress(); if (isInNet(hostip, "9.24.105.39","255.255.255.0")) { return "PROXY 9.24.105.83:80; " + "PROXY 9.24.105.63:80; " + "DIRECT"; } else if (isInNet(hostip,"9.24.105.82", "255.255.255.0")) { return "PROXY 9.24.105.61:80; " + "PROXY 9.24.105.63:80; " + "DIRECT"; } else { return "PROXY 9.24.105.63:80; " + "DIRECT"; } }

In our customization, if the primary proxy fails, the client browser will automatically point to the secondary proxy server. This is indicated in the PAC file as the proxy server with IP address 9.24.105.63. If both the primary and secondary proxies are unavailable, the browser will attempt to connect using a direct connection to the Internet. For those client IP addresses not defined in the PAC file, they will access the Internet through the proxy server with IP address 9.24.105.63.

Chapter 8. Advanced features

375

8.4.2 How to set up an automatic proxy

Now we will describe how to configure this option using the Configuration and Administration forms. The Configuration and Administration forms support for automatic proxy configuration is limited. Choose Proxy Configuration -> Proxy Auto-Configuration. The following window will be displayed:

Figure 8-33 Proxy Auto-Configuration window

We selected the Create new file radio button and entered a PAC file name, autoproxy.pac. Click Submit to create the file. The following window will be displayed:

376

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-34 Proxy Auto-Configuration window, next step

In this window, we defined the primary and secondary proxy servers as 9.24.105.83 and 9.24.105.63. We checked the Route DIRECT option for the browser to attempt a direct connection if both proxies servers are down. Click Submit to write the configuration changes to the autoproxy.pac file. After processing the form, Web Traffic Express displays the PAC file name in the Proxy Auto-Configuration form, as shown in Figure 8-35.

Chapter 8. Advanced features

377

Figure 8-35 Proxy Auto-Configuration window, next step

378

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The autoproxy file created using the steps listed above is shown in the following example:
Example 8-33 Autoproxy.pac file generated by Configuration and Administration forms //Created by WTE remote config-DO NOT EDIT //PRI PROXY 9.24.105.83 80 END //SEC PROXY 9.24.105.63 80 END //DIR yes END function FindProxyForURL(url, host) { return "PROXY 9.24.105.83:80; " + "PROXY 9.24.105.63:80; " + "DIRECT"; }

The file generated by the Configuration and Administration forms does not contain all the necessary entries to allow us to configure our scenario. We had to edit the autoproxy.pac file to add the entries as shown in Example 8-32 on page 375. The locations of the proxy automatic configuration file are listed in Table 8-9.
Table 8-9 Location of the PAC file Operating System Windows AIX 4.3.3 Linux Red Hat 6.2 Location c:\Program Files\IBM\WTE\HTML\pacfiles /opt/IBMWTE/usr/internet/server_root/pub/pacfiles /opt/IBMWTE/usr/internet/server_root/pub/pacfiles

8.4.3 Tests
To test if our autoproxy environment is working as planned, we must customize our client browser and access a URL.

Configuring the browser

We configured Netscape Communicator 4.7 and Microsoft Internet Explorer 5 on the client machines with IP addresses 9.24.105.82 and 9.24.15.39.

Netscape browser
To configure the Netscape browser, click Edit -> Preferences and open the Advanced option. Choose Proxies, click the Automatic Proxy Configuration, and enter the automatic proxy configuration URL. You will see the following window:

Chapter 8. Advanced features

379

Figure 8-36 Netscape browser preference settings

We entered the URL http://9.24.105.63/pacfiles/autoproxy.pac. This URL is where the PAC file is located, as defined in our scenario. We then clicked the Reload button to activate the changes.

380

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Microsoft Internet Explorer browser

To configure the Microsoft Internet Explorer browser, click Tools -> Internet Options... and open the Connections tab. Click the LAN Settings... button. You will see the following window:

Figure 8-37 Microsoft Internet Explorer browser LAN settings

We entered the URL http://9.24.105.63/pacfiles/autoproxy.pac. This URL is where the PAC file is located, as defined in our scenario. Click OK to activate the changes

Testing the access

We followed the steps below to test our configuration.

Client machines
We entered the URL http://www.cnn.com in the browser for the client machines with IP addresses of 9.24.105.82 and 9.24.105.39.

Chapter 8. Advanced features

381

The request from the client machine with IP address 9.24.105.82 was taken by proxy server 9.24.105.61- RS600034. To verify this, we viewed the access log on this proxy server. In the Configuration and Administration forms, open Server Activity Monitor -> Proxy Access Statistics. You see the following window:

Figure 8-38 The Proxy Access Statistics information window

You can see information about which URLs were requested on the WTE proxy server RS600034. The first column contains the IP address of the machine that requested the URL. In this case, the request was from the client with IP address 9.24.105.82. The request of the client machine with IP address 9.24.105.39 was taken by the proxy server, 9.24.105.83 - M238P4XL. To verify this, we viewed the access log on this proxy server. In the Configuration and Administration forms, open Server Activity Monitor -> Proxy Access Statistics. You see the following window:

382

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-39 The Proxy Access Statistics information window, next step

You can see information about which URLs were requested on the WTE proxy server M238P4XL. The first column contains the IP address of the machine that requested the URL. In this case, the request was from the client with IP address 9.24.105.39. Next, we disabled the proxy server with IP address 9.24.105.83 - M238P4XL. We then entered the URL http://www.cnn.com in the browser for the client machine with IP address 9.24.105.39. The request from this machine was taken by the secondary proxy server with IP address 9.24.105.63 - RHLINUX1, as defined in the PAC file. We can confirm this by opening Server Activity Monitor -> Proxy Access Statistics

Chapter 8. Advanced features

383

in the Configuration and Administration forms on this proxy server. You see the following window:

Figure 8-40 Process Access Statistics window, final step

You can see that the first column contains the IP address of our Web client (24.105.39). It is defined to access the WTE proxy server RS600034. However, this WTE server is disabled. The client requirement was automatically sent to the secondary WTE proxy server, RHLINUX1.

8.5 Proxy chaining

Proxy chaining is a function that enables you to chain proxies together and to identify domains to which requests should be passed without going through a proxy. Note: Proxy chaining is supported in a forward proxy environment only.

384

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

A client sends its request first to the local WTE server. If the local WTE server does not find the file in its cache, it sends a proxy request to another proxy at a higher level. This WTE server, in turn, sends the request to a higher level proxy in the chain, and so forth. The last proxy in the chain will direct the request to the Internet and return the requested content back through the chain to the local WTE server, which returns it to the client. All the proxies in the chain have the option of caching Web files. Note: The more proxies you chain together, the greater the latency you will introduce. It is recommended that you chain together no more than two proxies. These are the advantages to using proxy chaining: Proxies at a lower level, which are closer to the client that originated the request, benefit from the caches of the higher level proxies. Proxy chaining reduces the load on the highest level proxy (typically, the proxy closest to the firewall) and ultimately on the content server, since lower level proxies may already have the document cached. The larger the number of users, the higher the probability that a proxy server already has a Web document in its cache. Considering that high-level proxies serve a larger number of clients, many requests that cannot be honored by a low-level proxy can be resolved by higher level proxies, since other groups of clients may have already requested the same files. These are the disadvantages: A high-level proxy should have a larger cache, since it has to honor the requests of a large number of users. Proxy chaining greatly increases response time for requests, especially for noncacheable files or for those cacheable files that have not yet been cached by any proxies in the hierarchy. The risk of failure in a chain increases with each additional node. WTE allows the creation of proxy chains based on the protocol of the request to be forwarded (HTTP, FTP or Gopher). In other words, a proxy server can be configured to send all incoming requests with a particular protocol to a higher level proxy server in the chain. At the same time, you can specify the domains to which requests should be passed without going through a proxy.

Chapter 8. Advanced features

385

8.5.1 How to set up proxy chaining

We will use the following configuration for our test scenario:
9.24.105.63 RHLINUX1

9.24.105.61 RS600034

WTE Proxy Servers

9.24.105.82 M238P4YL

Web Content Server

socks.itso.ral.ibm.com

Intranet Firewall Firewall Web Server

9.24.105.59 RS60008

Internet

WEB Clients
9.24.105.39 M23KK904

Figure 8-41 Proxy chaining scenario

We will set up our proxy chaining scenario using the Configuration and Administration forms. In the WTE Proxy Server with IP address 9.24.105.61 (RS600034), choose Proxy Configuration -> Proxy Chaining and Non-Proxy Domains. You will see the window shown in Figure 8-42:

386

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 8-42 Proxy Chaining and Non-Proxy Domains window

We configured this proxy server, RS600034, to forward all HTTP requests that are not in its own cache to the second-level proxy, RHLINUX1. We also defined a non-proxy domain configuration, RS6008.itso.ral.ibm.com. This is the Web Server which we would not need a proxy to access.

Chapter 8. Advanced features

387

8.5.2 Testing the configuration

To test if our environment is working as expected, we customized the two Web clients browsers as shown in 7.2.1, Configuring the browser on page 309: 9.24.105.82 to access the Web Traffic Express proxy server RS600034 9.24.105.39 to access the Web Traffic Express proxy server RSLINUX1 The first access we entered was a URL in the browser on Web client 9.24.105.39. We made a request to URL http://www.rural.com.br. Since this client is configured to use the RHLINUX1 proxy server, the request is taken by this proxy server. This proxy server does not find the file in its own cache file, so the request is sent to the SOCKS server (9.14.3.72) and returns the results to the proxy server RHLINUX1. This proxy server caches this request, and also sends it to the Web clients browser. The following examples show the cache access log and a log captured by tcpdump. No cache access was performed on RHLINUX1. As can be seen in Example 8-34, the proxy cache is empty.
Example 8-34 RHLINUX1 empty proxy cache. [root@rhlinux1 /]# hostname rhlinux1 [root@rhlinux1 /]# cd /opt/IBMWTE/usr/internet/server_root/logs/ [root@rhlinux1 logs]# tail -f cache.Mar202001

The log captured by tcpdump shows the request being sent to SOCKS server.
Example 8-35 tcpdump log file on RHLINUX1 15:34:56.029598 eth0 < 9.24.105.39.1285 > 9.24.105.63.www: P 1:291(290) ack 1 win 8760 (DF) 15:34:56.029651 eth0 > 9.24.105.63.www > 9.24.105.39.1285:. 1:1(0) ack 291 win31830 (DF) 15:34:56.209796 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7663818 0,nop,wscale 0> (DF) 15:34:59.208525 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7664118 0,nop,wscale 0> (DF) 15:35:05.208520 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7664718 0,nop,wscale 0> (DF) 15:35:17.208518 eth0 > 9.24.105.63.3640 > 9.14.3.72.socks: S 1406934733:1406934733(0) win 32120 <mss 1460,sackOK,timestamp 7665918 0,nop,wscale 0> (DF) 15:35:17.254737 eth0 < 9.14.3.72.socks > 9.24.105.63.3640: R 0:0(0) ack 1406934734 win 0 15:35:17.257990 eth0 > 9.24.105.63.www > 9.24.105.39.1285: P 1:1461(1460) ack 291 win 32120 (DF) 15:35:17.258178 eth0 > 9.24.105.63.www > 9.24.105.39.1285: FP 1461:2033(572) ack 291 win 32120 (DF)

388

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

For the second access, we entered the same URL on the other Web client browser, 9.24.105.82. Since this client is configured to use the RS600034 proxy server, the request is taken by this proxy server. This proxy server does not find the file in its own cache, so it sends this request to the next highest level proxy server in the chain, which is the proxy server RHLINUX1. The proxy server RHLINUX1 looks up its own cache and finds the Web page there. It then sends this Web page back to proxy server RS600034. The proxy server RS600034 caches the Web page and sends it to the Web clients browser. The following examples show the cache access log of the two proxy servers and a log captured by tcpdump. As can be seen in Example 8-36, the proxy cache on RS600034 is empty.
Example 8-36 RS600034 empty proxy cache # hostname rs600034 # cd /wte/logs # tail -f cache.Mar202001

The log captured by tcpdump on the RS600034 shows the request being sent to another proxy server, RHLINUX1.
Example 8-37 tcpdump log file on RS600034 # tcpdump -n tcp 17:39:14.659175488 9.24.105.82.1533 > 9.24.105.61.80: P 1:425(424) ack 1 win 8760 (DF) 17:39:14.669557379 9.24.105.61.37871 > 9.24.105.63.80: S 2311065038:2311065038(0) win 16384 <mss 1460> 17:39:14.669693755 9.24.105.63.80 > 9.24.105.61.37871: S 1129326489:1129326489(0) ack 2311065039 win 32120 <mss 1460> (DF) 17:39:14.669771519 9.24.105.61.37871 > 9.24.105.63.80:. ack 1 win 16060 17:39:14.669921225 9.24.105.61.37871 > 9.24.105.63.80: P 1:496(495) ack 1 win 16060 17:39:14.670173814 9.24.105.63.80 > 9.24.105.61.37871:. ack 496 win 31625 (DF) 17:39:14.690600987 9.24.105.82.1534 > 9.24.105.61.80: S 26876359:26876359(0) win 8192 <mss 1460> (DF) 17:39:14.690701416 9.24.105.61.80 > 9.24.105.82.1534: S 1816308422:1816308422(0) ack 26876360 win 16060 <mss 1460>

There was access to the cache in the proxy server RHLINUX1 for proxy server 9.24.105.61 (RS600034) as can be seen in Example 8-38.
Example 8-38 RHLINUX1 proxy cache [root@rhlinux1 /]# hostname rhlinux1 [root@rhlinux1 /]# cd /opt/IBMWTE/usr/internet/server_root/logs/ [root@rhlinux1 logs]# tail -f cache.Mar202001 9.24.105.61 - - [20/Mar/2001:18:03:23 +0500] GET http://www.rural.com.br/ HTTP/1.0 200 1831 9.24.105.61 - - [20/Mar/2001:18:03:24 +0500] GET http://www.rural.com.br/index.swf HTTP/1.0 200 24674

Chapter 8. Advanced features

389

The third example demonstrates accessing a non-proxy domain. This means that we specify a URL in the client browser that is in the domain to which the WTE server should directly connect, and only used when you are doing proxy chaining. This directive does not apply when the proxy goes through a SOCKS server; use socks.conf for that purpose. In our scenario, we defined the Web Server RS60008 as a non-proxy domain (refer to Figure 8-42 on page 387). We requested a Web page from this Web server, http://rs60008, from the Web clients browser, 9.24.105.82. The request is sent to the proxy server. The proxy server does not find the file in its own cache file. Instead of forwarding the request to the second-level proxy, it sends it to Web Server RS60008 directly. The Web server returns the results to the proxy server RS600034. The proxy server on RS600034 sends the response to the Web client. So a proxy chaining environment supports a low-level proxy to forward requests to the Web server directly instead of using a higher-level proxy.

8.5.3 Configuration of the ibmproxy.conf file

The following example shows the directives that were updated in the ibmproxy.conf file on RS60034 to configure the proxy chaining.
Example 8-39 The directives for proxy chaining on RS60034 http_proxy http://rhlinux1.itso.ral.ibm.com/ no_proxy rs60008.itso.ral.ibm.com

8.6 Logging and performance

In this section, we show you how to use the Server Activity Monitor of Web Traffic Express. This tool monitors statistics and access log entries, and uses the statistics to assist the WTE administrator in the tuning of WTE for performance improvements. We also describe how the data collected from the Server Activity Monitor can be used to tune WTE for better performance.

390

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

8.6.1 Activity Monitor

The Server Activity Monitor enables you to display: Activity statistics Network statistics Access statistics Proxy access statistics Cache statistics Cache refresh summary Note: The server creates new log files every day at midnight. If the server is not running at midnight, new logs are created when it is first started the next day.

Chapter 8. Advanced features

391

Activity statistics
The Activity Statistics page provides information on the number of threads (connections) used and available, server response time, throughput, number of requests processed and also number of errors. To view the Activity Statistics page, click Server Activity Statistics -> Activity Statistics in the Configuration and Administration forms (see Figure 8-43). Click Refresh to update the information.

Figure 8-43 Activity Statistics window

392

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Network statistics
The Network Statistics page provides information about the network on which the proxy is running, such as data rate, bytes sent and received, and bandwidth. To view the Network Statistics window, click Server Activity Monitor -> Network Statistics in the Configuration and Administration forms (see Figure 8-44). Click Refresh to update the information.

Figure 8-44 Network Statistics window

Chapter 8. Advanced features

393

Access statistics
The Access statistics page provides information found in the access log files, including users accessing your proxy, IP addresses, user IDs (for protected pages), date and time of access, and the HTTP method used (GET, PUT, POST, DELETE). To view the Access Statistics window, click Server Activity Monitor -> Access Statistics in the Configuration and Administration forms, as shown in Figure 8-45. Click Refresh to update the information.

Figure 8-45 Access Statistics window

394

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Proxy access statistics

The Proxy Access Statistics page provides information on the proxy activity, such as which URLs were requested and if they were retrieved from the cache (the blue lines). Following the URLs are the return codes given to the client, and the file size in bytes. To view the Proxy Access Statistics window, click Server Activity Monitor -> Proxy Access Statistics in the Configuration and Administration forms, as shown in Figure 8-46. Click Refresh to update the information.

Figure 8-46 Proxy Access Statistics window

Chapter 8. Advanced features

395

Cache statistics
The Cache Statistics page provides the status of the proxy cache, such as whether the cache is currently operational. For example, if the cache is currently operational, the Cache Statistics page will display the message: The proxy cache is in normal operation To view the Cache Statistics page, click Server Activity Monitor -> Cache Statistics in the Configuration and Administration forms, as shown in Figure 8-47. Click Refresh to update the information.

Figure 8-47 Cache Statistics window

396

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Cache refresh summary

The Cache Refresh Summary provides information about: Number of URLs specified in the configuration file that were refreshed Number of pages refreshed during the previous nights cache access log Number of URLs that the cache agent refreshed Number of URLs remaining in the queue after reaching the maximum allowed number of URLs The cache agent must have run at least once to display any information. Note: To run the cache agent, the hostname directive must be configured as described in Configuration host name on page 296 To view the Cache Refresh Summary page, click Server Activity Monitor -> Cache Refresh Summary.

8.6.2 Configuration for performance improvements

This section describes the WTE configuration settings you can modify to tune your server.

Understanding performance settings

Each time your server receives a request from a client, it uses a thread to perform the requested action. If no threads are available, the server holds the request until more threads become available. The MaxActiveThreads directive in the WTE configuration file specifies the maximum number of active threads. Clients will experience slower response times or even failures if no active threads are available. The failure would be indicated by an error message such as:
Error 400 - Proxy Error Unable to connect to remote host or host not responding

Chapter 8. Advanced features

397

If the number of active connections is very low, you can lower the amount of virtual memory used by lowering the MaxActiveThreads setting. To change the active threads started by WTE, click Server Configuration -> System Management -> Performance. Click Submit to update the information after making any changes. The Performance window is shown in the following figure:

Figure 8-48 Performance window

398

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The following factors may impact the server response time: Network speed Traffic on the local area network (LAN) Number of clients requesting pages from your server Number of threads set on your server Other applications running on your server System resources Change the values of the Persistent Connections Timeout and Maximum Requests fields to specify the characteristics of a persistent connection. A persistent connection allows the server to accept multiple requests and to send responses over the same TCP/IP connection. By increasing the number of maximum requests, overall throughput will also be increased, because the server will not have to establish a separate TCP/IP connection for each request and response. Also, the TCP/IP connection is used more efficiently because of its characteristics. However, keep in mind that persistent connections require network bandwidth as well as a dedicated server thread. The Persist timeout specifies the amount of time that the server will wait between client requests before cancelling a persistent connection.

Chapter 8. Advanced features

399

To view the active threads started by WTE, click Server Activity Statistics -> Activity Statistics. Click Refresh to update the information. The following figure shows the Activity Statistics window of the WTE:

Figure 8-49 Activity Statistics window

Timeouts: closing connections automatically

Timeouts are used to control the server processing time. The default values are appropriate for most requests and need not be changed unless necessary.

400

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

To change the timeout values of WTE, click Server Configuration folder -> System Management -> Timeouts. The Timeouts window is shown in the following figure:

Figure 8-50 Timeouts window

Notice that: The Input Timeout field is used to set the time allowed for a client to send a request after making a connection to the server. A client first connects to the server and then sends a request. If the client does not send a request within the amount of time specified in this field, the server drops the connection. Take note that if you are using persistent connections, the persistent connection timeout specifies the time to wait for the client to send another request.

Chapter 8. Advanced features

401

The Read Timeout field is used to specify the time allowed without network activity before the connection is cancelled. The Output Timeout field is used to set the maximum time allowed for your server to send output to a client. The time limit applies to requests for local files and to requests for which the server is acting as a proxy. However, it does not apply to requests that start a local CGI program. If the server does not send the complete response within the amount of time specified in this field, the server drops the connection. The Script Timeout field is used to set the time allowed for a CGI program started by the server to finish. When the time runs out, the server ends the program. The Persist Timeout field specifies the amount of time the server will wait between client requests before cancelling a persistent connection.

402

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Configuring for proxy performance

Click Proxy Configuration -> Proxy Performance (see the following figure):

Figure 8-51 Proxy Performance window

Fill in the Proxy Performance form by clicking the following fields: Run as a pure proxy Select this box if you want to increase proxy performance by strictly running a proxy server. The alternative would be to run WTE as a content server too. If you want to use caching, leave the box unselected.

Chapter 8. Advanced features

403

Allow persistent connections Use this setting to allow clients to force the server to keep an open connection with them. This decreases the portion of the client's log time spent requesting documents from the proxy. However, maintaining persistent connections requires network bandwidth as well as a dedicated server thread. Do not allow persistent connections if your setup limits the number of available threads. Use SOCKS configuration file Select this box if you want the proxy to look at the SOCKS configuration file to decide whether or not to connect through the SOCKS server, as described in How to set up flexible-client SOCKS on page 368. Send HTTP/1.0 to downstream servers Select this if you have downstream servers which do not correctly handle requests from HTTP/1.1 clients. Run as a transparent proxy Select this box if you want your WTE server to run as a transparent proxy. To Identify how FTP URL paths should be resolved, select one of the following: absolute paths: this option resolves to fully-specified paths with respect to the root directory. relative paths: this option resolves with respect to the home directory. After you have completed your selections, click Submit to make the changes to the configuration file and restart the WTE server.

8.6.3 WTE logging

Web Traffic Express can be customized to provide access and error logs. The logs are specified by the following directives: AccessLog - used for logging local document requests. ErrorLog - used for logging any errors. ProxyAccessLog - used for logging proxy requests. CacheAccessLog - used for logging hits on the proxy cache (only valid if the WTE server is running as a proxy). EventLog - used for logging all events of the proxy server. The directory where the logs are installed by default changes by platform, as shown in Table 7-4 on page 299.

404

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The default log file names are shown in the following table:
Table 8-10 Default log file names Logs AccessLog ErrorLog ProxyAccessLog CacheAccessLog File name local error proxy cache

These logs are stored on disk at midnight every day; the server closes the current log files and creates new log files for the next day. If the server is not running at midnight, it starts a new log file the first time you start it on a given day. This feature cannot be changed. When creating the file, the server uses the file name you specify and appends a date suffix. The date suffix is in the format Mmmddyyyy, where: Mmm are the first three letters of the month. dd is the day of the month. yyyy is the year.

Chapter 8. Advanced features

405

The path and name of these log files can be customized using the Log Files form. Click Server Configuration -> Logging -> Log Files, as shown below:

Figure 8-52 Log Files window

We selected the Log information to Syslog box as we also want send the log information to the underlying operating system. On AIX and Linux, the syslog file is /etc/syslog.conf. We configured it via the following steps: Add the following lines to the /etc/syslog.conf file:
user.err /wte/logs/wte.log # For error information user.info /wte/logs/wte.log # For access information

406

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Restart the syslogd daemon with the following command:

refresh -s syslogd

We changed the path of the log files to the file system /wte, which we created earlier. The default path is /usr/lpp/internet/server_root. So all log paths were changed to /wte/logs. Press Submit to update the configuration file. You must stop and start the WTE server (see Table 7-3, Directives not refreshed upon restart on page 295).

Log archiving
Log archiving is recommended as a means to maintain WTE server log files. The Log Archiving form supports two log archiving methods: Compress Purge You can also choose to have no log archiving by using the option None. On our AIX platform, we set the Log Archiving method to Compress. In particular, we configured our WTE server to compress logs older than 30 days and delete them only after 90 days. The following command can be used to tar, compress and remove the log files: tar -cf /tmp/log%%DATE%%.tar %%LOGFILES%% ; compress /tmp/log%%DATE%%.tar ; rm %%LOGFILES%% The command must be entered on one line. This will tar all the logs from the specified date range, compress the resulting tar file, and delete the original logs. Additional sample compress commands are included in ibmproxy.conf in the Compress Directives.

Chapter 8. Advanced features

407

This configuration can be issued in the Log Archiving window, accessible by clicking Server Configuration -> Logging -> Log Archiving, as shown below:

Figure 8-53 Log Archiving window

Access log exclusions

Access log exclusions allow you to control what is logged. You can exclude data based on specific directories and/or files, user agents, host names or IP addresses, methods, MIME types, and return codes. Click Server Configuration -> Logging -> Access Log Exclusions. As this is a long form, we will show it in sections.

408

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Let us suppose that we do not want to log the requests to the /Docs and /admin-bin directories, which are the documentation and forms directories, respectively, as well as the /tunetips.html file. In this case, we will add the entries as shown in the form section below:

Figure 8-54 Access Log Exclusions window- Excluded Directories/Files section

We do not add any entry in the user agent section. In this section, we can define which user agent we do not want to log.

Chapter 8. Advanced features

409

The following window shows that we can configure our WTE server to not log requests coming from the machine with IP address 9.179.105.82. Notice that the wildcard character (*) is accepted.

Figure 8-55 Access Log Exclusions window- Excluded Host section

410

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Since we do not want to log requests for files of the image/gif type, we only select the image/gif box, as shown in the last section of the Access Log Exclusions form, shown below:

Figure 8-56 Access Log Exclusions window- last section

Chapter 8. Advanced features

411

412

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Part 4

Part

Combining Network Dispatcher and Web Traffic Express

Copyright IBM Corp. 2001

413

414

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Chapter 9.

Multiple Web Traffic Express proxy servers

In this chapter, we discuss using several Web Traffic Express proxy servers to process high traffic demand on a network, and two main ways to do this. We describe the configuration of a scenario using Network Dispatcher to load balance the back-end Web Traffic Express proxy servers.

Copyright IBM Corp. 2001

415

9.1 Using Autoproxy or Network Dispatcher

In todays networking environment, there is great demand for fast Internet access. Using a proxy cache will improve the response time for your Internet users, but what can you do if a single proxy server is not enough? In 8.4, Proxy auto-configuration on page 373, we discussed the configuration of a feature called Autoproxy. This feature allows you to work with multiple Web Traffic Express proxy servers and to define which client IP addresses will use each proxy server. This is a fixed load balancing, since it does not analyze any data from the servers in order to actually balance the load equally among them. For a small- or medium-sized network, this would be a good choice, since the configuration is simple, and you will work with only one product (no need for extra machines). For large networks or for heavy traffic networks, we recommend using Network Dispatcher to load balance proxy servers. Network Dispatcher can balance the load among all proxy servers, and using the WTE advisor it can analyze the response time for each server and use this information when calculating the server weight. You can work with Network Dispatcher and Web Traffic Express on the same machine (except on a Windows system). This is a collocation scenario. However, we recommend you use a separate machine for Network Dispatcher, for better performance. In the next sections, we describe a simple scenario showing Network Dispatcher load balancing multiple Web Traffic Express proxy servers.

416

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

9.2 Scenario description

In our scenario, we set up two Web Traffic Express proxy servers and one Network Dispatcher, which will load balance all browser requests to these two proxy servers (see Figure 9-1).

9.24.105.63 rhlinux1

9.24.105.61 rs600034

Web Traffic Express proxy servers

Cluster: 9.24.105.89 wses3 9.24.105.62 rs600037

Network Dispatcher

Figure 9-1 Network Dispatcher load balancing Web Traffic Express proxy servers

This scenario works like the scenario described in 4.1, Load balancing on page 70, except that the Dispatcher will be load balancing proxy servers instead of Web servers. All browsers and clients must be configured to use the cluster IP address (9.24.105.89) as the proxy address. They will send all HTTP requests to the Dispatcher, which will load balance to one of the proxy servers. You can also use Internet Caching Protocol (ICP) or Remote Cache Access (RCA) to improve your environment by allowing each WTE server to check the contents or exchange information about other WTE servers cache. Refer to the products readme file for more information on ICP, and refer to Web Traffic Express Users Guide Version 1.0, GC09-4567-00 for more information on RCA.

Chapter 9. Multiple Web Traffic Express proxy servers

417

9.3 Configuration
The Web Traffic Express proxy servers were configured as described in Chapter 7, Web Traffic Express basic configuration on page 287. Network Dispatcher was configured as in the scenario we described in 4.1, Load balancing on page 70. We followed the steps listed below: Start the Executor Add the 9.24.105.89 cluster, using the local Dispatcher as primary host and configuring this address on the local network adapter (see Figure 9-2).

Figure 9-2 Adding the WTE cluster

Add port 80. In our scenario, both proxy servers are listening to port 80. Make sure that all your proxy servers are listening to the same port. If it is not port 80, create the corresponding port in this step. Add the servers 9.24.105.61 and 9.24.105.63. Start the Manager. Start the WTE advisor, as shown in Figure 9-3. In our previous scenarios, we used the HTTP advisor. However, this time we are not load balancing Web servers, so we need to start an advisor specific to the service that the Dispatcher will load balance.

418

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 9-3 Starting the WTE advisor

We also set the stickytime so the Dispatcher would send all requests from a client to the same proxy server where their previous requests were sent. We recommend setting the stickytime in this scenario so the cached pages from a certain user will be kept on the same proxy server. To set the stickytime, click Port:80 in the left pane of the window, then click Configuration Settings in the right pane. Locate the field Sticky time (seconds), fill in the value you want (we used 300 seconds), and click Update Configuration at the bottom of the window to activate the changes (see Figure 9-4).

Chapter 9. Multiple Web Traffic Express proxy servers

419

Figure 9-4 Setting the stickytime

The configuration on the Dispatcher is ready. The next step is to set up the proxy servers to respond to the cluster IP address (see Configuration of the back-end servers on page 95). On the proxy server 9.24.105.61, which is an AIX machine, we ran the following command:
# ifconfig lo0 alias 9.24.105.89 netmask 255.255.255.0

On the proxy server 9.24.105.63, which is a Linux machine, we ran the following command:
# ifconfig lo:1 9.24.105.89 netmask 255.255.255.0

420

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

After running the commands above, the advisors began receiving responses from both proxy servers, as shown in Figure 9-5.

Figure 9-5 Advisors output for the WTE cluster

Any request sent to the cluster address will now be redirected to the proxy servers. The last thing to do is to configure the Web browsers to send the requests to the Dispatcher IP address (see 7.2.1, Configuring the browser on page 309).

Chapter 9. Multiple Web Traffic Express proxy servers

421

We performed our tests using Netscape Communicator V4.7; the proxy configuration is shown below:

Figure 9-6 Browser configuration

9.3.1 Configuration file

If you prefer to use the command line interface to configure the scenario described above, you can use the commands shown in Example 9-1.
Example 9-1 Configuration file ndcontrol executor start ndcontrol executor set nfa 9.24.105.62 ndcontrol cluster add 9.24.105.89 primaryhost 9.24.105.62 ndcontrol port add 9.24.105.89:80 ndcontrol server add 9.24.105.89:80:9.24.105.61 ndcontrol server add 9.24.105.89:80:9.24.105.63 ndcontrol cluster configure 9.24.105.89 en0 255.255.255.0 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 40 40 20 0 ndcontrol advisor start wte 80 wte_80.log

422

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

9.4 Tests
First, we did a test from only one browser, to make sure that the sticky option is working as expected. We configured this browser to use the Dispatcher as a proxy server, then we requested a few URLs. Figure 9-7 shows the monitor output while we did this test. Note that all requests are forwarded to only one of the proxy servers, 9.24.105.61. This confirms that while we used only one client IP address, the Dispatcher kept all requests forwarded to the same proxy server.

Figure 9-7 Monitor output: requests from only one client

Chapter 9. Multiple Web Traffic Express proxy servers

423

After that, we tested using several clients to increase the traffic to the Dispatcher. This time, since we were using several IP addresses, there was load balancing among all proxy servers (see Figure 9-8).

Figure 9-8 Monitor output: using several browsers

424

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

10

Chapter 10.

Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy
In this chapter, we discuss the combination of a wildcard cluster and Web Traffic Express to create a transparent proxy scenario.

Copyright IBM Corp. 2001

425

10.1 Introduction
If Web Traffic Express is configured to operate as a transparent proxy, the client software is totally unaware of the existence of the intermediate proxy server. Normally, if a client browser uses a proxy server, then the Web browser must be configured to specify the address and port of the proxy server. This is no longer necessary with transparent proxy, because the client is unaware that an intermediate proxy is in the network. To use transparent proxy, the router is programmed to redirect requests to the WTE transparent proxy. WTE then intercepts all HTTP requests on port 80 that are targeted at some server on the Internet. The request is parsed and processed, and may be satisfied from the transparent proxy's cache. The router must be capable of dealing with the requests for other services that will also come from the clients. This router may be a dedicated router, or it can be a machine running Network Dispatcher. We recommend you study carefully your environment to determine which one is the best solution for you.

10.2 Scenario description

In the WTE scenarios which we showed in the previous chapters, all clients were configured to send the HTTP requests to the proxy server. The other packets were sent to the default router, which will forward them to the next router or to the proper destination. An example of this scenario is shown in Figure 10-1.
9.24.105.61 rs600034 Web browsers Web Traffic Express

9.24.105.1 Router

Default route
Figure 10-1 Basic WTE scenario

HTTP requests

426

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The impact of this scenario is that you need to configure all clients to use WTE. If you want to make it transparent for the user, the router needs to redirect all packets sent to port 80 to WTE. In this scenario, we include an extra machine running Network Dispatcher (ND) to perform the task of redirecting the packets based on the service being requested. In our scenario, we used two machines: the first one is running Network Dispatcher (using only the Dispatcher component) and the second one is running Web Traffic Express. The Dispatcher works as a router, receiving all packets from the clients, no matter what the final destination of those packets may be. You can also install both WTE and ND on the same machine. All packets sent to port 80 that are received by the Dispatcher will be forwarded to WTE, which is configured as a transparent proxy. The packets sent to ports other than 80 will be forwarded to a router; this is the default router of the local network (see Figure 10-2).
9.24.105.61 9.24.105.62 rs600034 rs600037 Web Traffic Network Express Dispatcher

9.24.105.1 Router

Web browsers

Port 80 All ports except 80

Figure 10-2 Transparent proxy scenario

Default route

In this scenario, the only configuration needed for the clients is the default route, which must be set to the Dispatcher IP address. The following table shows more information on the machines we used for this scenario.

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

427

Table 10-1 Machines used in this scenario Host Name rs600037 rs600034 m238p4yl IP Address 9.24.105.62 9.24.105.61 9.24.105.82 Operating System and Software AIX 4.3.3 WebSphere Edge Server 1.0.3 AIX 4.3.3 WebSphere Edge Server 1.0.3 Windows NT 4.0 Service Pack 5 Netscape Communicator V4.7 Service Dispatcher Caching proxy Web client

These machines were all connected to the same Ethernet network.

10.3 Configuration
As we mentioned previously, the only Network Dispatcher component we need for this scenario is the Dispatcher. Refer to Chapter 3, Network Dispatcher installation on page 27 for information about installing Dispatcher. WTE needs to be installed as a transparent proxy. Refer to Chapter 6, Web Traffic Express installation on page 247 for information. If you did not install WTE as a transparent proxy, you can still set it after the installation. Open the Configuration and Administration forms, click Proxy Configuration->Proxy Performance, and make sure that the Run as a transparent proxy checkbox is selected as shown in Figure 10-3. If it is not, you must select it, click Submit at the bottom of the window, and restart WTE.

428

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 10-3 Selecting transparent proxy

Refer to Chapter 7, Web Traffic Express basic configuration on page 287 for more information about configuring WTE. As we mentioned before, the only requirement for WTE in this scenario is that it must run as a transparent proxy. You can configure it further according to your needs. You can test WTE by configuring a browser to use it as an HTTP proxy. Make sure that WTE is working properly before configuring Network Dispatcher.

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

429

After you have tested WTE and confirmed that it is working properly, follow the steps below to configure a wildcard cluster in Network Dispatcher: Using the GUI, connect to the Dispatcher machine. Start the Executor. Add the cluster 0.0.0.0 (using the local Dispatcher as primary host). See Figure 10-4.

Figure 10-4 Adding the wildcard cluster

Add port 80 to the wildcard cluster (see Figure 10-5). Port 80 is the port that WTE is listening to. In this scenario, we do not recommend that you use a different port.

Figure 10-5 Adding port 80 to the wildcard cluster

Add a server to port 80 using the IP address of the WTE proxy server. If you installed both WTE and ND on the same machine, you must use the NFA of

430

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

the Dispatcher machine (see Figure 10-6). This is the only server we need to add to this cluster. In this case, the Dispatcher will not perform any load balancing.

Figure 10-6 Adding the server to the wildcard cluster

The next step is to add a wildcard port to the wildcard cluster. Add port 0 to the wildcard cluster as shown in Figure 10-7. The wildcard port will handle all packets received by the Dispatcher, except for the packets sent to port 80.

Figure 10-7 Adding port 0 to the wildcard cluster

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

431

Add a server to port 0, using the router IP address. The Dispatcher will forward to this server all packets sent to ports other than 80. We used the IP address of the default router of the local network (see Figure 10-8).

Figure 10-8 Adding the router IP address to port 0

Start the Manager. After you start the Manager, you can also start the WTE advisor, as shown in Figure 10-9.

Figure 10-9 Starting the WTE advisor

432

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The clients do not need any configuration, but their default route must be the Dispatcher IP address (see Figure 10-10).

Figure 10-10 Default route on the client machine

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

433

Make sure that the browser on the client machine is set to use a direct connection to the network. Figure 10-11 shows the configuration on a Netscape Communicator V4.7 browser.

Figure 10-11 Browser configuration

434

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

10.4 Network Dispatcher configuration file

If you want to use the command line interface to configure the Dispatcher, use the commands shown in Example 10-1.
Example 10-1 Configuration file for the wildcard cluster ndcontrol executor start ndcontrol executor set nfa 9.24.105.60 ndcontrol cluster add 0.0.0.0 primaryhost 9.24.105.60 ndcontrol port add 0.0.0.0:80 ndcontrol server add 0.0.0.0:80:9.24.105.61 ndcontrol port add 0.0.0.0:0 ndcontrol server add 0.0.0.0:0:9.24.105.1 ndcontrol manager start manager.log 10004 ndcontrol manager proportions 49 49 2 0 ndcontrol advisor start wte 80 wte_80.log

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

435

10.5 Tests
To test this scenario, we configured a client machine with the default router as the Dispatcher IP address, and the browser set to use a direct connection to the network, as we described in the previous sections. From that browser, we requested the URL http://www.yahoo.com. As shown in Figure 10-12, we received the home page.

Figure 10-12 Browser showing the requested home page

436

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

But how can we make sure that the request was sent to the Dispatcher, and then forwarded to WTE? To verify this, we opened the Dispatcher monitor on port 80, and requested several home pages again; we confirmed that the Dispatcher was receiving the requests and forwarding them to WTE (see Figure 10-13).

Figure 10-13 Server monitor showing the packets sent to WTE

We also looked at the proxy access log on WTE to make sure that our request had been properly received. We used the Configuration and Administration forms and clicked Server Activity Monitor->Proxy Access Statistics, as shown in Figure 10-14. You can see the request to http://www.yahoo.com in the statistics. This confirms that the Dispatcher forwarded the packet properly to WTE, and that WTE was able to fulfill the request.

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

437

Figure 10-14 Proxy Access Statistics window

These tests confirmed that port 80 is working properly. Now, we need to test the exceptional situation of connections to other ports that should be redirected to the router, and not to WTE. In that case, the router should forward the packet according to its routing table. To perform this test, we initiated a Telnet session to a remote machine on a separate network. Telnet uses port 23, so when the Dispatcher receives the packets to this connection, it should forward them to the router (the server that was added to the wildcard port).

438

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

From the same client machine, we ran the following command:

# telnet 9.179.160.114

The connection was successfully established, as shown in Figure 10-15:

Figure 10-15 Telnet to a remote machine

To make sure that this connection was sent to the Dispatcher and then to the router, we ran iptrace at the Dispatcher level. Iptrace is a tool shipped with AIX to collect packets from the network. In order to use iptrace, you need to install the fileset bos.net.tcp.server. Two SYN packets are shown in Example 10-2. Actually, they represent the same packet. The first one is being received by the Ethernet adapter on the Dispatcher machine; the second one is the same packet, but this time it is being sent to another machine.
Example 10-2 Iptrace of the Telnet connection ETH: ====( 60 bytes received on interface en0 )==== 20:24:59.575788775 ETH: [ 00:06:29:b0:94:63 -> 00:04:ac:97:53:c8 ] type 800 (IP) IP: < SRC = 9.24.105.82 > IP: < DST = 9.179.160.114 > IP: ip_v=4, ip_hl=20, ip_tos=0, ip_len=44, ip_id=48058, ip_off=0 DF IP: ip_ttl=128, ip_sum=2282, ip_p = 6 (TCP) TCP: <source port=2708, destination port=23(telnet) > TCP: th_seq=1207f47, th_ack=0

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

439

TCP: th_off=6, flags<SYN> TCP: th_win=8192, th_sum=d084, th_urp=0 TCP: mss 1460 ETH: ETH: IP: IP: IP: IP: TCP: TCP: TCP: TCP: TCP: ====( 60 bytes transmitted on interface en0 )==== 20:24:59.575812788 [ 00:04:ac:97:53:c8 -> 10:00:5a:99:d4:f1 ] type 800 (IP) < SRC = 9.24.105.82 > < DST = 9.179.160.114 > ip_v=4, ip_hl=20, ip_tos=0, ip_len=44, ip_id=48058, ip_off=0 DF ip_ttl=128, ip_sum=2282, ip_p = 6 (TCP) <source port=2708, destination port=23(telnet) > th_seq=1207f47, th_ack=0 th_off=6, flags<SYN> th_win=8192, th_sum=d084, th_urp=0 mss 1460

This packet was sent from 9.24.105.82 (the client machine that started the Telnet connection) with destination 9.179.160.114. The port is 23, which is the Telnet port. Since the source and destination IP addresses do not change while the packet goes through routers (and they must not!), the only way to know which machines redirected this packet is to look at the MAC addresses that appear in each packet. In the second line of each packet on the iptrace, you can see the MAC address of the machines that sent and received the packet. The first one was sent from 00:06:29:b0:94:63 to 00:04:ac:97:53:c8. The second one was sent from 00:04:ac:97:53:c8 to 10:00:5a:99:d4:f1. You can determine the local MAC addresses by running the command netstat -in (see Example 10-3).
Example 10-3 Output of the netstat -in command Name Mtu Network Address Ipkts Ierrs lo0 16896 link#1 1111 lo0 16896 127 127.0.0.1 1111 lo0 16896 ::1 1111 en0 1500 link#2 0.4.ac.97.53.c8 50229 en0 1500 9.24.105 9.24.105.62 50229 Opkts Oerrs 1115 1115 1115 85850 85850 Coll 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0

440

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

This shows that the MAC address of the local Ethernet adapter is 00:04:ac:97:53:c8. An easy way to determine the MAC addresses of the other machines is to look at the ARP table of the Dispatcher. The entries in the ARP table are deleted frequently, so the MAC addresses are not kept there for a long time. We looked at the ARP table right after our test, since we knew that all IP addresses would be there. To view the ARP table, you can use the command arp -an (see Example 10-4).
Example 10-4 Output of the arp -an command ? (9.24.105.1) at 10:0:5a:99:d4:f1 [ethernet] ? (9.24.105.82) at 0:6:29:b0:94:63 [ethernet] ? (9.24.105.60) at 0:4:ac:17:34:68 [ethernet] ? (9.24.105.61) at 0:4:ac:97:52:97 [ethernet]

We came up with the following table after comparing the iptrace and the output of netstat -in and arp -an.
Table 10-2 Results of the test Packet 1 2 Source MAC 00:06:29:b0:94:63 00:04:ac:97:53:c8 Destination MAC 00:04:ac:97:53:c8 10:00:5a:99:d4:f1 Sent from 9.24.105.82 9.24.105.62 Received by 9.24.105.62 9.24.105.1

The results of this analysis show that the packet was sent to the Dispatcher, which then redirected the packet to router 9.24.105.1. Finally, you do not need to change anything in the routing table of the Dispatcher to make this scenario work. We used the Dispatcher machine with only the basic routes (the ones that are automatically added when you configure the network interface). You can see the routing table of the Dispatcher in Example 10-5.
Example 10-5 Routing table of the Dispatcher # netstat -rn Routing tables Destination Gateway Flags Refs Route Tree for Protocol Family 2 (Internet): 9.24.105/24 9.24.105.62 U 2 127/8 127.0.0.1 U 2 Route Tree for Protocol Family 24 (Internet v6): ::1 ::1 UH 0

Use

If

PMTU

Exp

Groups

4858 102

en0 lo0

lo0 16896

Chapter 10. Using a Network Dispatcher wildcard cluster with Web Traffic Express for transparent proxy

441

If we try to ping the 9.179.160.114 machine, it cannot reach the destination, as shown in Example 10-6.
Example 10-6 Ping failed # ping 9.179.160.114 PING 9.179.160.114: (9.179.160.114): 56 data bytes 0821-069 ping: sendto: Cannot reach the destination network. ping: wrote 9.179.160.114 64 chars, ret=-1 0821-069 ping: sendto: Cannot reach the destination network. ping: wrote 9.179.160.114 64 chars, ret=-1

We set the configuration this way just to make sure that Network Dispatcher was handling all packets, and not the TCP stack of the machine. Configure the machine for your environment with the routes you need, and the traffic from the other machines will be handled by the Dispatchers clusters.

442

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

11

Chapter 11.

Content Based Routing (CBR)

In this chapter, we discuss the Content Based Routing (CBR) component of Network Dispatcher.

Copyright IBM Corp. 2001

443

11.1 Introduction
In this chapter, we will document scenarios using Content Based Routing (CBR). See IBM Network Dispatcher Users Guide Version 3.0 for Multiplatforms, GC31-8496-05 for more information on how CBR works. Content Based Routing can be configured with WTE for HTTP servers. In addition, it can be configured as a CBR proxy (without WTE) for IMAP and POP3 servers. However, CBR cannot be configured for both on the same Network Dispatcher machine.

11.1.1 Installation of the CBR function

Refer to Chapter 3, Network Dispatcher installation on page 27 for information on how to install any Network Dispatcher component. When you reach the point where you must choose which component to install, select the following three items to install CBR on your machine: Content Based Routing Runtime Content Based Routing Administration Content Based Routing License

444

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The following figure shows these three items being selected:

Figure 11-1 Installing the CBR components

If you want to use CBR for HTTP, you must also install Web Traffic Express. Refer to Chapter 6, Web Traffic Express installation on page 247 for more information.

11.2 CBR for HTTP

CBR works with WTE to proxy client requests to specified servers. CBR, along with WTE, filters Web page content using specified rules.

11.2.1 CBR scenario

In this scenario, we demonstrate how to use CBR for HTTP by configuring a basic environment.

Chapter 11. Content Based Routing (CBR)

445

WTE configuration overview

The WTE component of WebSphere Edge Server for Multiplatforms has the functionality to work with the CBR component of Network Dispatcher. The steps required to start the WTE server are different on UNIX systems than on Windows systems. However, on all these platforms, WTE can be started with all of the default values without making any modifications to the configuration file. Refer to Chapter 7, Web Traffic Express basic configuration on page 287 for more information about configuring WTE.

WTE configuration file CBR modifications

When WTE is installed, the WTE configuration file, ibmproxy.conf, contains many lines of preconfigured WTE directives. It includes four lines to configure CBR, but these lines are commented out. Search for the directives shown in Example 11-1 and uncomment them. Note that the ServerInit directive is written using only one line. The next line is PreExit. Make sure that the path is correct on the predefined directives.
Example 11-1 Modifications to the ibmproxy.conf file # ===== CBR Plug-in ===== ServerInit C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndServerInit CBR_CLIENT_KEYS_DIRECTORY=C:\Progra~1\IBM\nd\admin\keys\cbr,CBR_SERVER_KEYS_DIRECTORY=C:\Progra ~1\IBM\nd\cbr\key,END_INSTALL_PATH=C:\Progra~1\IBM\nd,C:\Progra~1\IBM\nd\cbr\lib;C:\Progra~1\IB M\nd\cbr\lib\ibmcbr.jar;C:\Progra~1\IBM\nd\admin\lib\ChartRuntime.jar,11099,C:\Progra~1\IBM\nd\ cbr\logs\,C:\Progra~1\IBM\nd\cbr\configurations\ PreExit C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndPreExit PostExit C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndPostExit ServerTerm C:\Progra~1\IBM\nd\cbr\lib\libndcbr.dll:ndServerTerm

Add these lines to the WTE configuration file and save the file. Note: If you are using Windows NT, you must complete the configuration by adding the following to the Path system environment variable:
C:\Program Files\nd\cbr\lib;C:\Program Files\Ibm\Java13\jre\bin\classic

In this case, we had to reboot the machine after changing the Path variable, so that the changes could take effect for Control Panel. Perform the proper changes depending on the operating system you are using. Now you can restart WTE.

446

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

CBR configuration
Once the ibmproxy.conf file has been modified with the four lines to enable CBR, and the WTE proxy server has been restarted, then CBR configuration can begin. In this scenario, we defined two clusters, A and B, each with two servers. Cluster A had two servers, an AIX server and a Linux server, and the two servers in cluster B were Windows NT systems.

Network environment
The following figure is a graphical representation of our scenario environment:

Web servers
9.24.105.20 9.24.105.63 rs600036 rhlinux1 9.24.105.39 9.24.105.95 m23kk904 m23ff426

Cluster A 9.24.105.87

CBR/WTE
9.24.105.82 m238p4yl

Cluster B 9.24.105.88

Figure 11-2 CBR scenario

Cluster, port, server and rule configuration

Once the CBR-enabled proxy server was running, we used the Network Dispatcher GUI to configure our CBR cluster. To start the GUI, we clicked Start->Programs->IBM WebSphere->Edge Server->IBM Network Dispatcher. We right-clicked the Content Based Routing component and we selected Connect to Host from the pop-up menu.

Chapter 11. Content Based Routing (CBR)

447

If, at the point where you try to connect to the host to set the configuration either through the Network Dispatcher GUI or with the cbrcontrol commands, you receive a key related error message, then it is possible that there is an error in the four lines that were added to your ibmproxy.conf file. In our scenario, we then started the Executor, added our clusters, and to each cluster added a port and two servers (we followed the same procedure to add clusters and servers as we did with the Dispatcher).

Figure 11-3 Basic cluster configuration

We will now show you details of how to add two rules to the 9.24.105.87 cluster. To configure the content rules that CBR uses to determine among which servers to load balance the request, we right-clicked Port:80 in the navigation portion of the GUI and selected Add Rule... from the pop-up menu.

448

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Of course, you should plan the logic that you want CBR to follow before you start adding rules to your configuration. The network interface card of the CBR machine must be aliased to all the cluster addresses used in the configuration (see Figure 11-4). However, the loopback interface on the TCP server machines does not need to be aliased; for those who are familiar with this kind of aliasing operations in a Dispatcher scenario, it may seem unusual that similar aliasing is not done with CBR.

Figure 11-4 Aliases on the network interface

The reason for this is that when the packet is sent from the client machine, its destination IP address is the IP address of one of the clusters. When the packet arrives at the CBR machine, the packet is accepted because the network interface card on the CBR machine has been aliased to that cluster IP address. WTE receives the request and offers CBR the opportunity to examine its clusters and rules for a match. If a match is found, URL name translation is performed. If and when the proxy server sends the request to a clustered TCP server (the selection of which was done by CBR), WTE proxies a new request to the TCP server with the modified URL and a destination IP address that is the IP address of the TCP server machine. When the TCP server replies, its response goes

Chapter 11. Content Based Routing (CBR)

449

back to the CBR server, and is cached by WTE, if caching is enabled and if the WTE caching algorithms require caching of that particular page. For this reason, there is no need to alias the cluster IP address to the loopback interface on the TCP server machine. Contrarily, Dispatcher keeps the cluster IP address as the destination address of the packet and identifies the TCP servers through their Media Access Control (MAC) addresses. For this to work, the TCP servers need to have the cluster IP address aliased on their loopback interface. An advantage of this is that the servers response can flow directly to the client, without any need for it to be proxied by the Dispatcher. This would not be a good solution with CBR, however, because it would not allow caching. Another positive factor is that a CBR cluster does not have the same restriction as a Dispatcher cluster concerning the location of the TCP servers relative to the Dispatcher or CBR server. With CBR, because WTE proxies the request to the servers, there is no requirement for the servers to be located on the same LAN as the CBR machine. Our next step was to create a content rule to direct all requests to cluster 9.24.105.87, which contained the string products.html in the URL to server 9.24.105.63. To accomplish this, we right-clicked Port:80->Add Rule->Content, and in the Add Rule dialog window, we filled in the fields as follows:

Figure 11-5 Adding the first content rule

450

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

We assigned the name productpage to this rule. We did not put a value in the Priority field for the rule. Priorities establish the order in which rules will be reviewed. This parameter accepts integer values. If you do not specify the priority of the first rule you add, CBR will set it by default to 1. When a subsequent rule is added, by default its priority is calculated to be 10 plus the current lowest priority of any existing rule. For example, assume you have an existing rule whose priority is 30. You add a new rule and set its priority at 25 (which is a higher priority than 30). Then you add a third rule without setting a priority. The priority of the third rule is calculated to be 30 + 10 = 40, and so on. The Affinity type field contains one of two possible values: Client IP or Cookie. Client IP affinity as used in the Dispatcher component can also be used with CBR. The cookie affinity feature applies only to the CBR component and provides a way to make clients sticky to a particular server. This function is enabled by setting the stickytime of a rule to a positive number, and setting the affinity to Cookie. We left the default value of Client IP in the Affinity type field. In the next rule, we demonstrate setting cookie affinity and stickytime. The Pattern field is used to define the pattern of characters that CBR will match against each client request. The pattern must not contain any spaces and can make use of the special characters listed in the following table:
Table 11-1 Special characters allowed in the Pattern field Character * ( ) & | ! Function Matches 0 to x of any character Used for logic grouping Used for logic grouping Logical AND Logical OR Logical NOT

Chapter 11. Content Based Routing (CBR)

451

The reserved keywords shown in the following table must always be followed by the equal (=) sign:
Table 11-2 Keywords followed by the equal (=) sign Keyword client url protocol path refer user Value Client IP address URL in request Protocol section of URL Path section of URL Referred URL (quality of service) User ID section of URL

In our case, we made use of the URL reserved word and the wildcard (*) character. We specified the pattern as:
url=http://*/products.html

This rule specifies that any URL that is a request for the page products.html will be a match. Other examples of valid rule patterns are: url=http://*/*.gif client=9.32.* (path=index/*.gif&protocol=http)|(client=9.1.2.3) !(path=*.jpeg) The last item in the dialog window shown in Figure 11-5 on page 450 is a scrollable list of server addresses you may choose from. In this case, we chose the server with the IP address 9.24.105.63, meaning that we wanted this server to serve the client requests containing the string products.html in the URL. We then clicked OK in the dialog window shown in Figure 11-5 on page 450. The next rule was added to send requests to both servers in the 9.24.105.88 cluster for client requests for the page purchase.html. In this case, however, we specified an affinity type of Cookie and a stickytime of 30 seconds. This means that when a client request to the cluster is made and the URL matches the rule (that is, contains the string purchase.html), CBR would load balance the request to the best server of the two; then, all subsequent HTTP requests from that client to this cluster address would also go to that server for a period of 30 seconds. We called this rule purchase. The dialog window in this case appeared as follows:

452

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 11-6 Adding the second rule

We then clicked OK in the window above. The GUI reported that the configuration was updated with the added rules, as shown in the following figure:

Chapter 11. Content Based Routing (CBR)

453

Figure 11-7 Configuration showing both rules added

CBR Manager and Advisors

As with the Dispatcher, starting the Manager and using the Advisors is optional. The Manager can be activated with the command:
cbrcontrol manager start

The typical advisor that is used with CBR is the HTTP advisor, which can be launched by entering the following command:
cbrcontrol advisor start http port

where port is the port configured for the cluster.

454

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Saving the configuration

In our scenario, once we finished configuring CBR, we saved the configuration to a file. To do this from the Network Dispatcher GUI, we right-clicked Host and selected Save Configuration File As.... In the resulting pop-up window, we were prompted to enter the name of the configuration file in which we wanted to save this information. We entered the file name default.cfg in the Save Configuration pop-up window. By default, this file is placed in the directory <install path>/<component>/configurations, where <install path> varies depending on the operating system, and the <component> in this case is cbr. The configuration file is saved in ASCII format and contains the list of commands that would be necessary to reconfigure your environment. Following is the content of the default.cfg file:
Example 11-2 CBR configuration file
cbrcontrol cluster add 9.24.105.88 cbrcontrol port add 9.24.105.88:80 cbrcontrol server add 9.24.105.88:80:9.24.105.39 cbrcontrol server add 9.24.105.88:80:9.24.105.95 cbrcontrol cbrcontrol cbrcontrol cbrcontrol cbrcontrol rule rule rule rule rule add 9.24.105.88:80:purchase type content pattern url=http://*/purchase.html priority 1 useserver 9.24.105.88:80:purchase 9.24.105.39 useserver 9.24.105.88:80:purchase 9.24.105.95 set 9.24.105.88:80:purchase stickytime 30 set 9.24.105.88:80:purchase affinity cookie

cbrcontrol cluster add 9.24.105.87 cbrcontrol port add 9.24.105.87:80 cbrcontrol server add 9.24.105.87:80:9.24.105.63 cbrcontrol server add 9.24.105.87:80:9.24.105.20 cbrcontrol rule add 9.24.105.87:80:productpage type content pattern url=http://*/products.html priority 1 cbrcontrol rule useserver 9.24.105.87:80:productpage 9.24.105.63

The saved configuration file is interesting because it contains each of the cbrcontrol commands that could also be entered from the command line to configure the same environment. If you choose to configure CBR with commands, you can save your configured environment with the command:
cbrcontrol file save configurationfilename

Chapter 11. Content Based Routing (CBR)

455

You can then subsequently reload the configuration file with the command:
cbrcontrol file load configurationfilename

Scenario results
We placed simple HTML files in the document root directories on each of our Web servers. On the Web server with IP address 9.24.105.63, we placed a file named products.html and another called purchase.html. On the Web server with IP address 9.24.105.20 we placed a different file, also called products.html. Recall that we had defined a rule specifying that client requests containing the string products.html in the URL be load balanced between both of these servers. Each of the files uniquely identified the server on which it was located, as well as the name of the page. In a CBR scenario, the client machine does not need any special configuration. In particular, client browsers must not be set to redirect all the requests to the CBR machine. For example, in a real-life situation, it would not be appropriate to require end-users to reconfigure their Web browsers before accessing a Web site that uses CBR. The use of CBR in a Web site is completely transparent to end-users.

456

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Client IP affinity demonstration

For our first request, with our browser we requested the products.html page from our cluster IP address and received the following page in response:

Figure 11-8 Accessing the cluster

To verify that our productpage rule was matched by this request, we used the cbrcontrol persistent session command rule rep. The command line version of this is:
cbrcontrol rule report cluster:port:rule

Chapter 11. Content Based Routing (CBR)

457

As with other cbrcontrol commands, short forms of the keywords can be used. In order to specify that you would like the report to include all clusters, ports and rules, use only the two colon (::) delimiters in the command line. The command and its output are shown in Example 11-3.
Example 11-3 statistics for the rules C:\> cbrcontrol cbrcontrol>>rule rep :: --------------------------------------------Cluster Address: 9.24.105.88 Port:80 --------------------------------------------Rule Report: -----------Rule name ...................... Priority ....................... Times Fired .................... Number of Servers ..............

purchase 1 0 2

--------------------------------------------Cluster Address: 9.24.105.87 Port:80 --------------------------------------------Rule Report: -----------Rule name ...................... Priority ....................... Times Fired .................... Number of Servers ..............

productpage 1 11 1

Since the stickytime is set to 0 seconds, Web requests from the same client should not stick to the same Web server, and subsequent requests should normally load balance between the two Web servers defined in cluster 9.24.105.87. If the stickytime were set to a positive number of seconds, CBR would choose one of the two servers the first time a client request arrives that matches the productpage rule, and then would redirect all the client requests coming from the same IP address to the same server until the stickytime expires. In this case, however, we forced the CBR server to redirect all the requests to the Web server with IP address 9.24.105.63, because this was the only server available in the productpage rule.

458

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Cookie Affinity demonstration

The next test we performed demonstrated the use of Cookie affinity. Web cookies are simple pieces of information passed between the client Web browser and the Web server during an HTTP transaction. Cookies do not contain any information about the client that the server does not already have and they cannot do anything on the client machine that the client itself cannot already do, provided the browser is within the specifications. Cookies were introduced as an answer to a fundamental problem of the Web's underlying HTTP 1.0 protocol: the lack of a state, or a persistent connection. The first time a client accesses a Web site that serves cookies, the chosen server sends a cookie to the client browser, identifying the server and giving some information about which URLs the cookie is good for. The next time the client visits one of those URLs, the browser includes the cookie in its request. As long as the clients future requests contain the cookie, and each request arrives within the stickytime interval, the client will maintain affinity with the initial server. Prior to making a request for the purchase page that will trigger the 30 second affinity rule, we examined the contents of the local cookie file (cookies.txt) in the browser directory on our client machine; it was empty:
Example 11-4 Cookies before the test C:\Program Files\Netscape\Users\default>type cookies.txt # Netscape HTTP Cookie File # http://www.netscape.com/newsref/std/cookie_spec.html # This is a generated file! Do not edit. yp.yahoo.com TRUE / FALSE 1262383331 v=2&m=city&d=%01Durham%0 0%01-78.838402%015%01c www.aol.com FALSE FALSE 986502134 YP.us myCookie nm_b

Recall that we had enabled a 30 second cookie affinity on our purchase rule by setting the stickytime to 30, and setting the affinity to Cookie when we created the rule. This could also have been set with the command:
cbrcontrol rule set

Once a server is selected by CBR to respond to our request, subsequent requests were also responded to by the same server. In the 30 second period, we made three requests for http://9.24.105.88/purchase.html. Each request was responded to by the same server:

Chapter 11. Content Based Routing (CBR)

459

Figure 11-9 Accessing using Cookie affinity

We set the browser to warn us before setting a cookie, so it showed the following window before accessing the home page:

Figure 11-10 Information about the cookie

460

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

We verified that the cookie was set on our browser machine by examining the cookie.txt file:
Example 11-5 Cookies after the test C:\Program Files\Netscape\Users\default>type cookies.txt # Netscape HTTP Cookie File # http://www.netscape.com/newsref/std/cookie_spec.html # This is a generated file! Do not edit. yp.yahoo.com TRUE / FALSE 1262383331 v=2&m=city&d=%01Durham%0 0%01-78.838402%015%01c www.aol.com FALSE FALSE 986502134 9.24.105.88 FALSE / FALSE 986511773 -54-29-1007656-5932-42-6048-10045+986504573541! YP.us myCookie IBMCBR nm_b

Even though the contents of the cookie are not understandable by us, they are meaningful to the Web server and browser. After the 30 second stickytime expired, we saw that the CBR server was allowed to choose the other Web server to satisfy the request from the client.

11.2.2 WTE CacheByIncomingUrl directive

A WTE directive has been added for use with CBR in the ibmproxy.conf configuration file. This directive, CacheByIncomingUrl, specifies whether to use the incoming URL or the outgoing URL as the basis for generating cache file names. The values of this directive can be on or off. If CacheByIncomingUrl is set to on, the incoming URL will be used to generate the cache file name. In other words, when this directive is set to on, WTE keeps the original URL and uses it to cache the page that it gets back from the Network Dispatcher-changed URL. On the other hand, if off is specified, CBR rule matching and load balancing will be done on the incoming URL and the resulting URL will be used to generate the cache name. In this case, WTE uses the Network Dispatcher-changed URL to cache the page. The CacheByIncomingUrl directives default value is off in the ibmproxy.conf file. Notice that CacheByIncomingUrl is a hidden directive of WTE. In other words, it is possible to alter its value only by manually editing the configuration file ibmproxy.conf; there is no way to change the value of this directive through the Configuration and Administration forms.

Chapter 11. Content Based Routing (CBR)

461

Notice also that a Web page is cached only if WTE decides it should be. WTE decides this by looking at the Expires tag. If there is none, it estimates an expiration time via the Last-Modified parameter. You should keep this in mind if you are testing CBR caching with sample pages you have just created and which do not contain Expires header information. In this case, recently created pages would not be cached, since they would be interpreted by WTE as frequently changed pages, thereby resulting in an apparent failure of the CBR caching function.

11.3 CBR proxy for IMAP and POP3

CBR for IMAP or POP3 proxies the client requests to the appropriate server based on user ID and password. You can configure a cluster of IMAP or POP3 servers as though they were one server, thus allowing support for extremely large numbers of mailboxes from a single site. The content in question is only the IMAP or POP3 login sequence. The CBR proxy is placed in front of a cluster of either IMAP or POP3 servers (not both). The proxy directs the login request to a server that supports the user name and password supplied by the client.

Note: If any individual mailbox is actually installed on more than one of these clustered mailbox servers, then the first one to reply will be used.

11.3.1 Scenario
We used a scenario with one machine running Network Dispatcher, where we created the CBR cluster, and two back end servers running POP3. On these servers, we created individual users (the user created in one POP3 server does not exist on the other POP3 server).

462

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 11-11 shows the environment we used.

9.24.105.60 rs600010

9.24.105.62 rs600037

POP3 servers

Cluster: 9.24.105.87 wses1

Network Dispatcher
9.24.105.83 m238p4yl

Figure 11-11 CBR for POP3 scenario

11.3.2 Configuration
The configuration of CBR for IMAP and POP3 is very similar to the configuration of Dispatcher and CBR for HTTP. You need to add a cluster, ports and servers. If you prefer to use the wizard, you can run it by right-clicking Content Based Routing in the left pane of the GUI window and selecting Start Configuration Wizard, as shown in Figure 11-12.

Chapter 11. Content Based Routing (CBR)

463

Figure 11-12 Starting the CBR configuration wizard

The configuration wizard will guide you through the steps necessary to add the clusters, ports and server. In this section, we describe the configuration steps using the GUI. First, you need to start cbrserver. Run the following command (this is only needed for CBR for IMAP or POP3):
# cbrserver

464

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Important: You need to automatically start cbrserver after a reboot if you are using this in your production environment. Refer to Starting Network Dispatcher after a reboot on page 95 for more information. For Windows systems, since there is no service created for cbrserver, you can: create a service and start it automatically, or add it to the Startup program group For more information, refer to Windows NT or Windows 2000 documentation. Then, you can connect to the host where CBR is running using the GUI. Once you are connected, you can add the cluster by right-clicking Executor in the left pane of the GUI window, as shown in Figure 11-13.

Chapter 11. Content Based Routing (CBR)

465

Figure 11-13 Adding a cluster

A pop-up window will be displayed, as shown in Figure 11-14. We typed in the IP address of our POP3 cluster, 9.24.105.87.

Figure 11-14 Entering the cluster address

466

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

The next step is to add a port to this cluster. Right-click Cluster, then select Add Port.... The pop-up window shown in Figure 11-15 will be displayed. Type in the port number you are using and select the protocol (either POP3 or IMAP). Since we are creating a POP3 cluster, we typed in the port number 110 and selected the protocol POP3.

Figure 11-15 Adding the POP3 port

Now you need to add the servers that belong to this cluster. Right click Port:110, then select Add Server.... The pop-up window shown in Figure 11-16 will be displayed. Type in the IP address of the POP3 server you want to add to this cluster.

Figure 11-16 Adding a server

Add all servers that you want to be part of this cluster. We added two servers: 9.24.105.60 and 9.24.105.62. The complete configuration is shown in Figure 11-17.

Chapter 11. Content Based Routing (CBR)

467

Figure 11-17 CBR for POP3 configuration

If you want, you can also start the Manager and Advisors for this environment.

468

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

11.3.3 Configuration file

The configuration file generated by the GUI after saving the configuration is shown in Figure 11-6. You can use these commands to create your configuration using the command line interface.
Example 11-6 CBR for POP3 configuration file cbrcontrol cluster add 9.24.105.87 cbrcontrol port add 9.24.105.87:110 protocol POP3 cbrcontrol server add 9.24.105.87:110:9.24.105.62 cbrcontrol server add 9.24.105.87:110:9.24.105.60

11.3.4 Tests
We recommend that you test your POP3 servers to make sure they are working properly before you start testing the CBR cluster configuration. We created a user cris in the machine 9.24.105.62. We checked to see if this user had messages in the mailbox (see Example 11-7). When you check for messages in the users mailbox, make sure you do not remove them from the mailbox.
Example 11-7 Checking the user criss mailbox # cd /var/spool/mail # ls -l total 5 -rw-rw---1 cris mail 1217 Apr 06 09:52 cris -rw-rw---1 root mail 535 Mar 01 16:31 root # su - cris [YOU HAVE NEW MAIL] $ mail Mail [5.2 UCB] [AIX 4.1] Type ? for help. "/var/spool/mail/cris": 2messages 2 unread >U 2 root Fri Apr 6 09:25 16/371 "Test 1" U 3 root Fri Apr 6 09:52 16/365 "Test 2" ? q Held 2messages in /var/spool/mail/cris

Chapter 11. Content Based Routing (CBR)

469

Then we set up Netscape Mail to use the CBR cluster as a POP3 server, as shown in Figure 11-18.

Figure 11-18 Netscape Mail configuration for user cris

Using Netscape Mail, we tried to download the messages from the POP3 server. The client opened a connection to the cluster address; CBR communicated with the POP3 that had the user cris on it, and sent back the messages to the client (see Figure 11-19).

470

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Figure 11-19 Netscape Mail showing the messages for user cris

As Figure 11-19 shows, the messages were successfully downloaded from the POP3 server using the CBR cluster.

Chapter 11. Content Based Routing (CBR)

471

472

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Part 5

Part

Appendix

Copyright IBM Corp. 2001

473

474

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Appendix A.

Java source code

This appendix lists source code for the two Java applications used to demonstrate the dynamic caching feature of Web Traffic Express.

Copyright IBM Corp. 2001

475

TimeServlet Java source code

Example 11-8 TimeServlet.java import java.io.*; import javax.servlet.*; import javax.servlet.http.*; public class TimeServlet extends HttpServlet { public void service(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException { java.util.Date d = new java.util.Date(); PrintWriter out = res.getWriter(); out.println("<br><br>"); out.println(d.getHours()+":"+d.getMinutes()+":"+d.getSeconds()); out.println("<BR>This timestamp was calculated by TimeServlet, which is not cacheable, and should change<br>"); } }

CalcServlet Java source code

Example 11-9 CalcServlet.java import java.io.*; import javax.servlet.*; import javax.servlet.http.*; public class CalcServlet extends HttpServlet { public void service(HttpServletRequest req, HttpServletResponse res) throws ServletException, IOException { try { ServletContext servletContext = getServletContext(); java.util.Date d = new java.util.Date(); PrintWriter out = res.getWriter(); out.println(d.getHours()+":"+d.getMinutes()+":"+d.getSeconds()+"\n--Performing Calculation...<br>"); out.println("--This timestamp was calculated by CalcServlet, which is cacheable, and shouldn't change<br>"); servletContext.getRequestDispatcher("/servlet/TimeServlet").include(req,res); int arg1 = Integer.parseInt(req.getParameter("arg1")); int arg2 = Integer.parseInt(req.getParameter("arg2")); //we'll wait for a little while to pretend this calculation takes some effort, //and make the caching more noticable. synchronized (this) { try {

476

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

wait(300); } catch (Exception e) {e.printStackTrace();} } out.println(d.getHours()+":"+d.getMinutes()+":"+d.getSeconds()+"\n--answer = "+arg1*arg2+"<br>"); } catch (Exception e) { e.printStackTrace(); } } }

Appendix A. Java source code

477

478

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Related publications
The publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this redbook.

IBM Redbooks
For information on ordering these publications, see How to get IBM Redbooks on page 480. IBM WebSphere Performance Pack: Load Balancing with IBM SecureWay Network Dispatcher, SG24-5858-00 IBM WebSphere Performance Pack: Caching and Filtering with IBM Web Traffic Express, SG24-5859-00

Other resources
These publications are also relevant as further information sources: WebSphere Edge Server for Multiplatforms Getting Started Version 1.0, SC09-4566 WebSphere Edge Server for Multiplatforms Web Traffic Express Users Guide, GC09-4567 WebSphere Edge Server for Multiplatforms WTE Programming Guide, GC09-4568 WebSphere Edge Server for Multiplatforms WTE Programming Guide, GC09-4568 IBM Network Dispatcher Users Guide Version 3.0 for Multiplatforms, GC31-8496

Referenced Web sites

These Web sites are also relevant as further information sources: http://www-105.ibm.com/developerworks/tools.nsf/dw/java-devkits-byname ?OpenDocument&Count=100 developerWorks: Java technology : Tools and products - Developer kits

Copyright IBM Corp. 2001

479

http://www.ibm.com/developerworks/java/jdk/index.html developerWorks: Java technology: IBM Developer Kits http://www.ibm.com/software/webservers/edgeserver/library.html WebSphere Edge Server for Multiplatforms README Version 1.0.3 http://www.ibm.com/software/webservers/edgeserver/doc/wte_v36/readme.h tml WebSphere Edge Server for Multiplatforms WTE V3.6 Readme Version 1.0.3 http://www.ibm.com/software/webservers/edgeserver/doc/ndv36/readme.txt README for IBM Network Dispatcher V3.6 http://www.ibm.com/software/webservers/edgeserver/library.html WebSphere Edge Server for Multiplatforms Version 1.0.3 Documentation Supplement http://www.ibm.com/software/webservers/edgeserver/library.html Network Dispatcher, V3.6: Scalability, Availability and Load-balancing for TCP/IP Applications http://www.ibm.com/software/webservers/edgeserver/efix-wte.html Caching Proxy component - WebSphere Edge Server 1.0 corrective service http://www.ibm.com/software/webservers/edgeserver/efix-nd.html Load Balancing component - WebSphere Edge Server 1.0.3 corrective service http://www.ibm.com/software/webservers/edgeserver/library.html Websphere Edge Services architecture http://www.iplanet.com/downloads/developer/ iPlanet Downloads Developer Downloads http://www.surfcontrol.com SurfControl - Internet Filtering, Monitoring, Blocking, and Access Control Software http://www.ibm.com/developerworks/linux/ developerWorks: Linux

How to get IBM Redbooks

Search for additional Redbooks or redpieces, view, download, or order hardcopy from the Redbooks Web site:
ibm.com/redbooks

Also download additional materials (code samples or diskette/CD-ROM images) from this Redbooks site.

480

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Redpieces are Redbooks in progress; not all Redbooks become redpieces and sometimes just a few chapters will be published this way. The intent is to get the information out much quicker than the formal publishing process allows.

IBM Redbooks collections

Redbooks are also available on CD-ROMs. Click the CD-ROMs button on the Redbooks Web site for information about all the CD-ROMs offered, as well as updates and formats.

Related publications

481

482

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Special notices
References in this publication to IBM products, programs or services do not imply that IBM intends to make these available in all countries in which IBM operates. Any reference to an IBM product, program, or service is not intended to state or imply that only IBM's product, program, or service may be used. Any functionally equivalent program that does not infringe any of IBM's intellectual property rights may be used instead of the IBM product, program or service. Information in this book was developed in conjunction with use of the equipment specified, and is limited in application to those specific hardware and software products and levels. IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to the IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785. Licensees of this program who wish to have information about it for the purpose of enabling: (i) the exchange of information between independently created programs and other programs (including this one) and (ii) the mutual use of the information which has been exchanged, should contact IBM Corporation, Dept. 600A, Mail Drop 1329, Somers, NY 10589 USA. Such information may be available, subject to appropriate terms and conditions, including in some cases, payment of a fee. The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS. The use of this information or the implementation of any of these techniques is a customer responsibility and depends on the customer's ability to evaluate and integrate them into the customer's operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Customers attempting to adapt these techniques to their own environments do so at their own risk. Any pointers in this publication to external Web sites are provided for convenience only and do not in any manner serve as an endorsement of these Web sites.

Copyright IBM Corp. 2001

483

The following terms are trademarks of other companies: Tivoli, Manage. Anything. Anywhere.,The Power To Manage., Anything. Anywhere.,TME, NetView, Cross-Site, Tivoli Ready, Tivoli Certified, Planet Tivoli, and Tivoli Enterprise are trademarks or registered trademarks of Tivoli Systems Inc., an IBM company, in the United States, other countries, or both. In Denmark, Tivoli is a trademark licensed from Kjbenhavns Sommer - Tivoli A/S. C-bus is a trademark of Corollary, Inc. in the United States and/or other countries. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and/or other countries. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States and/or other countries. PC Direct is a trademark of Ziff Communications Company in the United States and/or other countries and is used by IBM Corporation under license. ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States and/or other countries. UNIX is a registered trademark in the United States and other countries licensed exclusively through The Open Group. SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC. Other company, product, and service names may be trademarks or service marks of others.

484

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Abbreviations and acronyms

ACL AFS AIX ATM API APPN ARP CARP CB CBR CGI CORBA DMZ DNS ECA EJB EJS FDDI FTP GC GEM GRE GUI HACMP HP-UX access control list Andrew File System advanced interactive executive Asynchronous Transfer Mode application programming interface Advanced Peer-to-Peer Network Address Resolution Protocol Cache Array Routing Protocol Component Broker Content Based Routing Common Gateway Interface Common Object Request Broker Architecture demilitarized zone Domain Name System External Cache Adapter Enterprise JavaBeans Enterprise Java Services Fiber Distributed Data Interface File Transfer Protocol garbage collector Global Enterprise Manager Generic Routing Encapsulation graphical user interface High Availability Cluster Multiprocessing Hewlett-Packard UNIX HTML HTTP IBM ICP IEEE IMAP IP ISP ISS ITSO JDK JRE JSP JVM LAN LDAP LPAR MAC Mbps MBps MIB MVS NAP NAT NCSA Hypertext Markup Language Hypertext Transfer Protocol International Business Machines Corporation Internet Caching Protocol Institute of Electrical and Electronics Engineers Internet Mail Access Protocol Internet Protocol Internet Service Provider Interactive Session Support International Technical Support Organization Java Development Kit Java Runtime Environment JavaServer Pages Java Virtual Machine local area network Lightweight Directory Access Protocol Logical partition mode Media Access Control megabits per second megabytes per second management information base multiple virtual storage network access points network address translation National Center for Supercomputing Applications Network Dispatcher

ND

Copyright IBM Corp. 2001

485

NFA NNTP OS/2 OSA OSE PAC PICS POP POP3 RCA RFC RMI RPC RSACI

Non-forwarding address NetNews transfer protocol Operating System/2 Open Systems Adapter open servlet engine proxy automatic configuration Platform for Internet Content Selection points of presence Post Office Protocol 3 Remote Cache Access Request for Comment Remote Method Invocation remote procedure call Recreational Software Advisory Council on the Internet Real Time Streaming Protocol Server Directed Affinity Secure Sockets Layer Simple Mail Transfer Protocol Scalable processor architecture Transmission Control Protocol/Internet Protocol Tivoli Enterprise Console Type of service User Datagram Protocol Universal Resource Identifier Universal Resource Locator, Uniform Resource Locator wide area network Wide Area Network Dispatcher Web Traffic Express workload manager Web Proxy Auto Discovery

WWW W3C XML

World Wide Web World Wide Web Consortium Extensible Markup Language

RTSP SDA SSL SMTP SPARC TCP/IP TEC TOS UDP URI URL WAN WAND WTE WLM WPAD

486

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Index
A
access log exclusions 408, 411 AccessLog 404405 active connections 93 administration components 157 advisor 80, 8889, 9394, 102, 122123, 165168, 172173, 189, 199, 201, 232, 238, 241, 243, 416, 418, 421422, 432, 435, 454, 468 advisor fast-failure detection 10 affinity 11, 231232, 235, 451, 459 affinity address mask 234 affinity record 232 affinity table 232233 ARP 92, 95, 441 asynchronous I/O 248 automatic cache refreshing 355356 automatic proxy configuration 373, 376, 379, 416 autoproxy.pac 375377, 379381 autorun 276, 285 461462 CBR for HTTP 29, 56, 445 CBR proxy for IMAP and POP3 29, 56, 462 configuration file 455, 469 CBR proxy 444, 462 cbrcontrol 448, 454459, 469 cbrkeys 157158 cbrserver 464465 Cell 133, 137138, 143, 146 cell 133, 146 CGI 177, 236, 299, 402 Client-IP 314, 317 cluster 75, 77, 80, 8485, 87, 8990, 95, 98, 101, 206213, 226, 231, 418, 431, 447449, 463, 467 cluster IP address 80, 90, 95, 99, 101, 107108, 114, 120, 126, 164, 197, 200, 214, 417, 420, 449450, 457 collocation 9, 158, 160, 416 Configuration and Administration forms 249, 258, 263, 270, 282, 288292, 295, 297, 307, 309, 312, 316, 325, 327, 329, 345, 352, 354, 356, 360, 369, 376, 382, 384, 386, 392393, 395396, 428, 437, 461 Connections per second on a port 183 Content Based Routing. See CBR cookie 235236, 459461 cookie affinity 7, 235236, 451, 459460 cross port affinity 233234 CyberLists 338, 345, 347 CyberNOT 337, 340341, 344, 351 CyberNot 340 CyberPatrol 6, 14, 248 CyberPatrol database 345, 347 CyberPatrol filtering 336, 338340, 342, 344345, 347, 351 CyberPatrol plug-in 336, 351 CyberYES 337, 340341, 344

B
back-end server 8788, 90, 92, 95, 102, 107, 127, 129131, 158, 167, 174, 180, 183, 198, 201, 231232 bandwidth rules 9

C
cache access log 304 cache agent 355, 357358, 397 cache content management 351 Cache expiration settings 305 Cache Expiry Setting 354 CacheAccessLog 404405 cacheagt 358 CacheByIncomingUrl 461 caching proxy component ix, 45 caching proxy server 45, 22, 288, 292, 315, 327 CalcServlet 361362, 364, 366367, 476 CanMonitor 139 capacity utilization 9 CBR 67, 12, 30, 55, 57, 175178, 235237, 249, 263, 443444, 446452, 455456, 458459,

D
default route 427, 433 default router 426427, 432, 436 delving 355, 358 directives 295297, 300, 304, 307309, 317,

Copyright IBM Corp. 2001

487

323324, 326327, 332334, 338, 342, 351, 356, 364365, 390, 397, 404, 407, 446, 461 not refreshed 295 disable listener 12 dnload.log 351 dynacache 13, 359360, 366 dynacache.xml 359360, 363365 location 363 dynamic cache 359, 361, 365366 dynamic caching 13, 359360, 362, 364

E
ECA 359, 363 environment variable 52, 58 ErrorLog 404405 EventLog 404 Executor 8384, 89, 109110, 115116, 120122, 418, 422, 430, 435, 448, 465 expiration time 354, 365 external cache adapter. See ECA externalCacheGroup 363365 ExternalCacheManager 365

high availability 7, 16, 18, 104110, 112113, 115, 117120, 123, 125128, 158, 206, 208209 backup machine 104107, 110, 113115, 117119, 121, 123127 mutual high availability. See mutual high availability primary machine 105, 107108, 110, 112116, 118119, 121126 High Availability Cluster Multi-Processing 7 high availability scripts 114, 120, 210, 213214, 219, 222 goActive 120121, 213214, 217218 goIdle 120 goInOp 120121, 213, 216, 219 goStandby 120121, 213, 215, 218 htadm 291292, 325 htcformat 249, 263, 275, 301302 HTML header 314

I
IBM AIX Developer Kit, Java 2 Technology Edition 28 IBM Cross Platform Technologies for Windows V2.0 40 IBM Runtime Environment for Linux 55 IBM SecureWay Directory 327 IBM SecuryWay Directory 328 ibmproxy 260261, 284, 294 ibmproxy.conf 289, 294295, 299, 304 adding administrators 291 basic configuration 288, 307, 309 basic user authentication 326 CBR 446448, 461 CyberPatrol 338, 351 dynamic caching 360, 364365 garbage collection 307 host name 296 HTTP header options 316, 318319 LDAP authentication 329330, 332, 334, 336 location of 295 logging 407 PidFile 297 proxy chaining 390 ICP 14, 417 ifconfig 87, 91, 100101, 200, 224, 420 IMAP 444, 462464, 467 inittab 95 InstallShield Wizard 3132, 43, 5859, 252, 260,

F
FILTER_DEBUG_ON 351 FindProxyForURL 374 firewall 7, 1920, 202203, 367368, 385 fixed weights 167168, 170172, 174 flexible-client SOCKS 367369 forward proxy 249250, 256, 262, 268, 275, 280, 384

G
garbage collection 304307 bandwidth 306307 blend 306307 responsetime 306 GenCyberDB 343, 349350 Generic Routing Encapsulation 10 gigabit Ethernet 8 GRE 10 GroupFile 325326, 335

H
HACMP 7 heartbeat 105, 110, 113, 115, 118119, 122126, 149, 220, 230

488

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

265, 271, 276, 284285 Interactive Session Support, See ISS Internet Caching Protocol. See ICP Internet Explorer 41, 309310 IP alias 85, 9091, 95, 100, 108, 114, 120121, 126, 164, 199200, 209210, 214216, 224, 227, 231 IP ranges 13 iptrace 316, 439 ISS 6, 11, 30, 93, 102, 127, 129136, 146, 149 agent 11, 127, 130131 cell 7, 132133, 137138, 141, 143, 146, 149150, 152 daemon 133134, 136, 149 high availability 132, 138 monitor 11, 127, 129131, 136, 138, 146, 150 isscontrol 132, 150 isskeys 157158

manager proportions 89, 9394, 122123, 132 MaxActiveThreads 295, 397398 MaxExpiryTime 365 maximum requests 399 memory caching 249, 262, 275 mutual high availability 206, 208, 213, 220221 backup machine 207, 210 primary machine 208, 210, 212, 214

N
ndadmin 38, 65, 71, 135 ndconfig 92 ndcontrol 94, 102103, 113, 117, 124, 126, 165166, 172, 223, 226, 234, 238240, 422, 435 ndkeys 157158 ndload 11 ndloadstat 11 ndserver 38, 65, 82, 95 Netscape 249, 263, 295, 309 Netscape Communicator 28, 41, 56, 309, 379, 422, 434 Netscape Directory Server 327 Netscape Mail 470 Netscape Navigator 28, 41, 56, 373 netstat 109 Network Dispatcher Advanced features 155 Basic configuration 69 command line 89 configuration wizard 7273 GUI 81 Installation 27 Custom 29, 57 Express 29, 56 Guided 29, 56 Network Interface Card 28, 41, 55 new connections 93 NFA. See non-forwarding address non-advertising address 95 non-forwarding address 8990, 122123, 161, 197, 430 Novel NDS eDirectory 327

J
Java Runtime Environment 28, 40, 57, 249, 263 JavaBean 362 JavaServer Pages. See JSP JSP 13, 359, 365

L
LDAP 13, 330 database 326, 328, 332 directory 326, 330 directory server 327, 330, 336 plug-in 330 plugin 328, 330, 332333, 335336 LDAP plugin 332333, 335 Lightweight Directory Access Protocol. See LDAP load balancing component ix, 4, 6, 18 local area network 7 log archiving 407408 log files 335, 391, 394, 405407 logging Network Dispatcher 237 Web Traffic Express 390 loopback adapter 9699, 101, 200 loopback interface 95, 215216, 450

O M
mailbox 462, 469 Manager 88, 93, 132, 167168, 432, 454, 468 observer 146, 148

Index

489

P
PAC file. See proxy automatic configuration file pac.conf 327, 330333, 336 PAC_DEBUG_LEVEL 335 pacd 330, 332, 335 pacd_restart.bat 332, 335 pacd_restart.sh 332, 335 PasswdFile 325326, 334335 persistent connection 399, 401402, 404 Persistent Connections Timeout 399 PICS 6 Platform for Internet Content Selection. See PICS POP3 444, 462464, 466467, 469471 port 76, 87, 89, 102, 109, 115, 162, 177, 195196, 199, 221, 231, 233234, 251, 257, 269, 281, 289, 418, 426427, 430432, 437438, 447, 454, 467 port specific 93 PreExit 446 Privacy Settings 316317 private key 157158 process ID 261, 285 protection setup 321, 323325 proxy 336, 375, 388 proxy access log 299, 312, 437 proxy automatic configuration file 373377, 380381, 383 location of 379 proxy chaining 384386 Proxy Header Filtering 318319 Proxy server 312 proxy server 56, 13, 16, 1920, 251, 255, 267, 279, 291, 297, 299300, 304, 308310, 334336, 355, 364, 366368, 372373, 375, 377, 382385, 387390, 403404, 415421, 423424, 426, 430, 447, 449 ProxyAccessLog 404405 public key 157158 pure proxy 300, 403

Q
quad-port Ethernet 8 quiesce enhancement 11

123127, 221 Auto 105, 110, 113, 115, 118119, 122126 Manual 105, 110, 123124, 127 Recreational Software Advisory Council on the Internet. See RSACI Redbooks Web Site 480 Contact us xi Referer 314 registry 54 remote administration 29, 56 Remote Cache Access. See RCA remote configuration 156, 158 Remote Method Invocation. See RMI resource type 141143 reverse proxy 249, 251, 262, 275 RMI 156, 158 round-robin 7 route delete 99100 routing table 99100, 165, 438, 441 RSACI 6 rule affinity override 235 rules, how they are evaluated 179 rules, types of 176 Always true 178, 184, 186187, 235 Capacity and bandwidth rules 178 Capacity or bandwidth rules 184 Client IP address 176 Client port 177, 184 Connections per second on a port 176 Content of a request 177 Time of day 176 Total active connections for a port 177, 184 Type of service 178, 184 rules-based load balancing 175 Begin range 184 End range 184 Level to evaluate 185 Level to share available bandwidth 185 One or more server addresses 185, 187 Priority 184, 187 Rule name 184, 187 TOS 185

R
raw disk 301303 RCA 5, 21, 417 reach target 110111, 116, 220221 Recovery strategy 110, 113, 115, 118119,

S
SDA 232 SDA agent 232233 Secure Sockets Layer 12 security policy 13

490

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

Selecting 45 Server Activity Monitor 382383, 390391, 393397 Access statistics 391, 394 Activity statistics 391392, 400 Cache refresh summary 391, 397 Cache statistics 391, 396 Network statistics 391, 393 Proxy access statistics 382383, 391, 395 Server Directed Affinity. See SDA server monitor 104 Server weight field 171 ServerInit 446 servlet 359, 361, 364365 servletcache.xml 359360, 364 SMIT 31, 248, 261 SOCKS configuration file 368, 370, 372, 404 SOCKS server 367370, 372373, 388, 390, 404 SSL 12 SSL Tunneling 299 startsrc 261 stickymask 233234 stickytime 11, 231236, 419420, 451452 stopsrc 260 Summary 48 SurfControl 14, 337338, 345, 347 SurfControl database 336, 338 syslog 406 system metrics 93

Web Traffic Express Advanced features 313 Basic configuration 287 Installation 247 WebSphere Application Server. See WAS WebSphere Edge Server for Multiplatforms 48 WebSphere Performance Pack 4 wide area network Dispatcher 191 local Dispatcher 194195, 197, 199201, 204205 remote Dispatcher 195202, 204206 wide area port number 195, 199 wildcard cluster 425, 430431, 435 wildcard port 431, 438 WTE registry 326, 329 WteAdapter 362364 WteMetaDataGeneratorImpl 362364

X
X-Windows 31, 58

T
thread 177, 392, 397400, 404 timeout 365, 399402 TimeServlet 361362, 367, 476 Tivoli SecureWay Firewall 368 transparent proxy 249, 251, 262, 268, 275, 404, 425429

U
user administration 320 user authentication basic 320 LDAP 326 User-Agent 314, 317320

W
WAS 13, 70, 359365, 372

Index

491

492

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher

(1.0 spine) 0.875<->1.498 460 <-> 788 pages

Back cover

WebSphere Edge Server: Working with Web Traffic Express and Network Dispatcher
Configuration of WebSphere Edge Server Scenarios for Web Traffic Express and Network Dispatcher Details of latest enhancements
As the Internet economy evolves, it is no longer enough to simply establish an online presence to sell your products and services. To compete in this fast-paced, global marketplace, you need to respond rapidly and intelligently to changing application and network demands while reducing the complexity of implementing new e-business solutions. Availability, scalability and performance are critical to ensuring that your e-business runs smoothly and cost efficiently. WebSphere Edge Server for Multiplatforms allows Internet service providers (ISPs) and enterprise customers to confidently host mission-critical Web sites for intranet and Internet e-business applications. WebSphere Edge Server brings together innovative caching, local- and wide-area load balancing and application-aware quality-of-service routing capabilities. The integration of these functions and their wide array of features can help reduce cost for the site owner while providing improved customer satisfaction. This redbook will help you install, tailor and configure the new features and enhancements included in WebSphere Edge Server for Multiplatforms. It is intended for networking personnel and information technology administrators who plan to use WebSphere Edge Server to provide Internet access to end users and distribute Web-accessible content more efficiently.

INTERNATIONAL TECHNICAL SUPPORT ORGANIZATION

BUILDING TECHNICAL INFORMATION BASED ON PRACTICAL EXPERIENCE IBM Redbooks are developed by the IBM International Technical Support Organization. Experts from IBM, Customers and Partners from around the world create timely technical information based on realistic scenarios. Specific recommendations are provided to help you implement IT solutions more effectively in your environment.

For more information: ibm.com/redbooks

SG24-6172-00 ISBN 0738421812