Monitoring Tomcat with JMX

Monitoring Tomcat with JMX

Christopher Schultz
Chief Technology Offcer
Total Child Health, nc!
Christopher Schultz
Chief Technology Offcer
Total Child Health, nc!
* Slides available on the Linux Foundation / ApacheCon2014 web site and at
http://people.apache.o!/"schult#/ApacheCon $A 2014/%o&cat 'onitoin!/
(ava 'ana!e&ent )xtensions
(ava 'ana!e&ent )xtensions

*otocol and A*+ ,o &ana!in! and &onitoin!

Access data via ('- .'beans/

0ead and wite bean attibutes

+nvo1e opeations

0eceive noti,ications

(2' exposes cetain status

%o&cat exposes cetain status

*otocol and A*+ ,o &ana!in! and &onitoin!

Access data via ('- .'beans/

0ead and wite bean attibutes

+nvo1e opeations

0eceive noti,ications

(2' exposes cetain status

%o&cat exposes cetain status

'onitoin! (2'
'onitoin! (2'

Heap status

%otal3 ,ee3 used &e&o4

5aba!e collection

5C pause ti&es

6eap status

%otal3 ,ee3 used &e&o4

5aba!e collection

5C pause ti&es
'onitoin! %o&cat
'onitoin! %o&cat

Status o, connecto

Status o, e7uest8pocesso thead pool

Status o, data souces

0e7uest pe,o&ance

Status o, connecto

Status o, e7uest8pocesso thead pool

Status o, data souces

0e7uest pe,o&ance
('- %ools
('- %ools

9console :(;<=

2isual2' :(;<3 app bundle=

'ost po,iles :e.!. >ou<it3 etc.=

Custo& tools usin! 9avax.&ana!e&ent A*+

9console :(;<=

2isual2' :(;<3 app bundle=

'ost po,iles :e.!. >ou<it3 etc.=

Custo& tools usin! 9avax.&ana!e&ent A*+

'onitoin! (2': 6eap
'onitoin! (2': 6eap
'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
'onitoin! %o&cat: Sessions
'onitoin! %o&cat: Sessions
'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
'onitoin! %o&cat: ;ataSouces
'onitoin! %o&cat: ;ataSouces
'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
'onitoin! >ou Application
'onitoin! >ou Application

'onito Application *ocesses

*e,o&ance 'etics

?n8the8,l4 e8con,i!uation

'onito Application *ocesses

*e,o&ance 'etics

?n8the8,l4 e8con,i!uation
'onitoin! >ou Application
'onitoin! >ou Application

@ite an 'Aean

Ceate an +nte,ace: Foo'Aean

Ceate an +&ple&entation: Foo

Ceate an -'L 'Aean descipto

;eplo4 pac1a!e to %o&cat

*ublish the 'Aean to the 'Aean seve

Bue4 / invo1e as necessa4

* )xa&ple code will be available at
http://people.apache.o!/"schult#/ApacheCon $A 2014/%o&cat 'onitoin!/
)xa&ple 'Aean
)xa&ple 'Aean

Sevlet Filte that captues total e7uest

pocessin! ti&e

%i&esta&p pio to e7uest

%i&esta&p a,te e7uest

Add the delta to a ('-8accessible counte:

0e7uestStats
0e7uestStats 'Aean
0e7uestStats 'Aean

@ite an 'Aean
public interface RequestStatsMBean {
public long getProcessingTime();
public long getRequestCount();
public void resetCounters();

public class RequestStats

implements RequestStatsMBean {
!"""#
public void updateStats(long
timestamp$ ServletRequest request$ long
elapsed) {

%total&lapsedTime"add'nd(et(elapsed);

%requestCount"increment'nd(et();

public long getProcessingTime(){
return %total&lapsedTime"get();

public long getRequestCount() {
return %requestCount"get();

public void resetCounters() {
%total&lapsedTime"set()l);
%requestCount"set()l);

0e7uestStats 'Aean
0e7uestStats 'Aean

@ite an 'Aean descipto

*mbeans+descriptors,
*mbean name-.RequestStats. """,
*operation name-.getProcessingTime.
description-.(ets t/e total number of
milliseconds spent processing requests".
impact-.0123.
returnT4pe-.long. 5,
*operation name-.getRequestCount.
description-.(ets t/e total number
of requests processed".
impact-.0123.
returnT4pe-.long. 5,
*operation
name-.resetCounters.
description-.Resets all
counters".
impact-.'CT031.
returnT4pe-.void. 5,
*5mbean,
*5mbeans+descriptors,
0e7uestStats 'Aean
0e7uestStats 'Aean

Ceate (A0

(ava inte,ace

(ava i&ple&entation

&beans8desciptos.x&l

*ut (A0 into CA%AL+$ACAAS)/lib

Ceate (A0

(ava inte,ace

(ava i&ple&entation

&beans8desciptos.x&l

*ut (A0 into CA%AL+$ACAAS)/lib

0e7uestStats 'Aean
0e7uestStats 'Aean

@ite the Filte

public void init(2ilterConfig config) {
MBeanServer server - getServer();
server"registerMBean(%stats$ ne6
3b7ect1ame(.&8ample9RequestStats-RequestStats$name-. : filter1ame;));

public void do2ilter(""") {
timestamp - elapsed - S4stem"currentTimeMillis();
c/ain"do2ilter(request$ response);
elapsed - S4stem"currentTimeMillis() + elapsed;
%stats"updateStats(timestamp$ request$ elapsed);

0e7uestStats 'Aean
0e7uestStats 'Aean

'ap the Filte

*filter,
*filter+name,servlet+request+stats*5filter+name,
*filter+class,filters"RequestStats2ilter*5filter+class,
*init+param,
*param+name,name*5param+name,
*param+value,servlets*5param+value,
*5init+param,
*5filter,
*filter+mapping,
*filter+name,servlet+request+stats*5filter+name,
*url+pattern,5servlets5;*5url+pattern,
*5filter+mapping,
*filter,*filter+name,7sp+request+stats*5filter+name,*filter+
class,filters"RequestStats2ilter*5filter+class,*init+param,*param+name,name*5param+
name,*param+value,7sps*5param+value,*5init+param,*5filter,
*filter+mapping,*filter+name,7sp+request+stats*5filter+name,*url+pattern,57sp5;*5url+
pattern,*5filter+mapping,

'ap the Filte

*filter,
*filter+name,servlet+request+stats*5filter+name,
*filter+class,filters"RequestStats2ilter*5filter+class,
*init+param,
*param+name,name*5param+name,
*param+value,servlets*5param+value,
*5init+param,
*5filter,
*filter+mapping,
*filter+name,servlet+request+stats*5filter+name,
*url+pattern,5servlets5;*5url+pattern,
*5filter+mapping,
*filter,*filter+name,7sp+request+stats*5filter+name,*filter+
class,filters"RequestStats2ilter*5filter+class,*init+param,*param+name,name*5param+
name,*param+value,7sps*5param+value,*5init+param,*5filter,
*filter+mapping,*filter+name,7sp+request+stats*5filter+name,*url+pattern,57sp5;*5url+
pattern,*5filter+mapping,
0e7uestStats 'Aean
0e7uestStats 'Aean
0e7uestStats 'Aean
0e7uestStats 'Aean
Auto&ated 'onitoin!
Auto&ated 'onitoin!

0e&ote Access

La!e Scale

Constant

0e&ote Access

La!e Scale

Constant
Auto&ated 'onitoin!
Auto&ated 'onitoin!

0e&ote Access

La!e Scale

Constant

$eed &oe toolsD

0e&ote Access

La!e Scale

Constant

$eed &oe toolsD

Auto&ated 'onitoin!
Auto&ated 'onitoin!

$a!ios

Simple

Flexible

@ell8deplo4ed

$o8cost co&&unit4 vesion available

$a!ios

Si&ple

Flexible

@ell8deplo4ed

$o8cost co&&unit4 vesion available

Auto&ated 'onitoin!
Auto&ated 'onitoin!
$a!ios 'onitoin!
$a!ios 'onitoin!

*lu!8in achitectue :i.e. abita4 scipts=

Feel48available ('- plu!8in: chec1C9&x

< "5c/ec=%7m8 +>
service97m89rmi95557ndi5rmi955local/ost9??))57m8rmi@
+3 7ava"lang9t4pe-Memor4 +' 1onAeapMemor4>sage +B used@
+6 CD)))))) +c E)))))))
FMG H'R101( 1onAeapMemor4>sage"used-CD)I)JJ)

*lu!8in achitectue :i.e. abita4 scipts=

Feel48available ('- plu!8in: chec1C9&x

< "5c/ec=%7m8 +>
service97m89rmi95557ndi5rmi955local/ost9??))57m8rmi@
+3 7ava"lang9t4pe-Memor4 +' 1onAeapMemor4>sage +B used@
+6 CD)))))) +c E)))))))
FMG H'R101( 1onAeapMemor4>sage"used-CD)I)JJ)
$a!ios 'onitoin!
$a!ios 'onitoin!

*oble&s with chec1C9&x

Co&plex con,i!uation ,o e&ote ('-

(2' launch ,o eve4 chec1

Couse8!ained authentication options

*oble&s with chec1C9&x

Co&plex con,i!uation ,o e&ote ('-

(2' launch ,o eve4 chec1

Couse8!ained authentication options

$a!ios 'onitoin!
$a!ios 'onitoin!

Altenative ?ption: %o&catEs ('-*ox4Sevlet

('- data available via 6%%*

Can use %o&catEs authentication tools

< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK @
+6 CD)))))) +c E)))))))
FMG CR0T0C'N9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K +
AeapMemor4>sage + =e4 KusedK - ?))JOICPJ

Altenative ?ption: %o&catEs ('-*ox4Sevlet

('- data available via 6%%*

Can use %o&catEs authentication tools

< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK @
+6 CD)))))) +c E)))))))
FMG CR0T0C'N9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K +
AeapMemor4>sage + =e4 KusedK - ?))JOICPJ
* chec1C9&xpox4 can be ,ound at
http://wi1i.apache.o!/to&cat/tools/chec1C9&xpox4.pl
$a!ios 'onitoin!
$a!ios 'onitoin!
('- Co&&and8line %ic1s
('- Co&&and8line %ic1s

Show all lo!!ed8in usena&es

for sessionid in Q6get +3 + K/ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-listSession0
dsK @
S sed +e .s5 5@n5g.
S grep KT!)+D'+Ua+V#@:@(@"";@)@L<K ;@
do 6get +3 W ./ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-getSession't
tributeMps-<sessionid$user. ; done C,5dev5null @
S grep >ser

Show all lo!!ed8in usena&es

for sessionid in Q6get +3 + K/ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-listSession0
dsK @
S sed +e .s5 5@n5g.
S grep KT!)+D'+Ua+V#@:@(@"";@)@L<K ;@
do 6get +3 W ./ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-getSession't
tributeMps-<sessionid$user. ; done C,5dev5null @
S grep >ser
%ac1in! 2alues ?ve %i&e
%ac1in! 2alues ?ve %i&e

So&e &etics ae best obseved as deltas

Session count

0e7uest eo count

0e7uies that 4ou have a histo4 o, data

0e7uies that 4ou consult the histo4 o, that data

chec1C9&xpox4 povides such capabilities

So&e &etics ae best obseved as deltas

Session count

0e7uest eo count

0e7uies that 4ou have a histo4 o, data

0e7uies that 4ou consult the histo4 o, that data

chec1C9&xpox4 povides such capabilities

%ac1in! 2alues ?ve %i&e
%ac1in! 2alues ?ve %i&e
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)CCOJD)P$
delta-!"""#
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ??EJ)X?PP$
delta-??ICOCP)
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)DCXP)IX$
delta-+PIPC)JJ
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)CCOJD)P$
delta-!"""#
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ??EJ)X?PP$
delta-??ICOCP)
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)DCXP)IX$
delta-+PIPC)JJ
%ac1in! 2alues ?ve %i&e
%ac1in! 2alues ?ve %i&e

Session count

%o&cat actuall4 povides this alead4 via 'ana!eEs

sessionCreateRate attibute

0e7uest eos
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-Catalina9t4pe-RequestProcessor$6or=er-./ttp+nio+?CO")")"?+
JC?O.$name-AttpRequest?Matt-errorCountK +6 ? +c ?) ++6rite errors"t8t ++compare
errors"t8t
FMG 3B9 3B + 'ttribute get KCatalina9t4pe-RequestProcessor$6or=er-./ttp+nio+
?CO")")"?+JC?O.$name-AttpRequest?K + errorCount - )$ delta-)

Session count

%o&cat actuall4 povides this alead4 via 'ana!eEs

sessionCreateRate attibute

0e7uest eos
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-Catalina9t4pe-RequestProcessor$6or=er-./ttp+nio+?CO")")"?+
JC?O.$name-AttpRequest?Matt-errorCountK +6 ? +c ?) ++6rite errors"t8t ++compare
errors"t8t
FMG 3B9 3B + 'ttribute get KCatalina9t4pe-RequestProcessor$6or=er-./ttp+nio+
?CO")")"?+JC?O.$name-AttpRequest?K + errorCount - )$ delta-)
;etectin! ?ut?,'e&o4
;etectin! ?ut?,'e&o4

Many sources of OOME

Heap exhaustion

*e&5en exhaustion

6it thead li&it

6it ,ile descipto li&it

'an4 souces o, ??')

6eap exhaustion

*e&5en exhaustion

6it thead li&it

6it ,ile descipto li&it

;etectin! ?ut?,'e&o4
;etectin! ?ut?,'e&o4

%wo t4pes o, heap ??')

?ne thead !eneates lots o, local e,eences

All theads collaboate to !eneate !loball48

eachable ob9ects :e.!. session data=

Fo&e is ecoveable3 latte is not

>ou want to be noti,ied in an4 case

%wo t4pes o, heap ??')

?ne thead !eneates lots o, local e,eences

All theads collaboate to !eneate !loball48

eachable ob9ects :e.!. session data=

Fo&e is ecoveable3 latte is not

>ou want to be noti,ied in an4 case

'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
'e&o4 *ool %hesholds

Choice o, how to detect exceeded8

theshold conditions

*ollin! usin! chec1C9&xpox4

0e!iste a noti,ication listene ,o& (ava

Have that listener take some action

Choice o, how to detect exceeded8

theshold conditions

*ollin! usin! chec1C9&xpox4

0e!iste a noti,ication listene ,o& (ava

6ave that listene ta1e so&e action

;etect ?ut?,'e&o4
;etect ?ut?,'e&o4

'onitoin! 'e&o4 %hesholds

Set theshold on statup

0e!iste a noti,ication listene :callbac1=

@atch .exceeded/ count :poll=

0epot to &onitoin! so,twae :$a!ios=

0epeat ,o each &e&o4 pool 4ou want to watch

6ope the (2' does not ,ail duin! noti,ication

%his is !ettin! idiculous

'onitoin! 'e&o4 %hesholds

Set theshold on statup

0e!iste a noti,ication listene :callbac1=

@atch .exceeded/ count :poll=

0epot to &onitoin! so,twae :$a!ios=

0epeat ,o each &e&o4 pool 4ou want to watch

6ope the (2' does not ,ail duin! noti,ication

%his is !ettin! idiculous

;etectin! ?ut?,'e&o4
;etectin! ?ut?,'e&o4

(2' has an easie wa4

Fse 8--:?n?ut?,'e&o4)o to un a
co&&and on first ??') detected b4 the
(2'

$eed a co&&and to noti,4 $a!ios

(2' has an easie wa4

Fse 8--:?n?ut?,'e&o4)o to un a
co&&and on first ??') detected b4 the
(2'

$eed a co&&and to noti,4 $a!ios

$oti,4 $a!ios on ??')
$oti,4 $a!ios on ??')

Scipt that waps cul

< curl +si @
++data+urlencode Kcmd%t4p-E)K @
++data+urlencode Kcmd%mod-CK @
++data+urlencode ./ost-m4/ost. @
++data+urlencode .service-FYM9Aeap933M&. @
++data+urlencode .plugin%state-C. @
++data+urlencode .plugin%output-33M& CR0T0C'N. @
K/ttps955monitoring+/ost5nagios5cgi+bin5cmd"cgiK

Scipt that waps cul

< curl +si @
++data+urlencode Kcmd%t4p-E)K @
++data+urlencode Kcmd%mod-CK @
++data+urlencode ./ost-m4/ost. @
++data+urlencode .service-FYM9Aeap933M&. @
++data+urlencode .plugin%state-C. @
++data+urlencode .plugin%output-33M& CR0T0C'N. @
K/ttps955monitoring+/ost5nagios5cgi+bin5cmd"cgiK
Scipt can be ,ound at http://wi1i.apache.o!/to&cat/tools/na!ios8send8passive8
chec1.sh
'onitoin! %o&cat with ('-
'onitoin! %o&cat with ('-

('- *ovides 'onitoin! and 'ana!e&ent o, (2's

%o&cat exposes a !eat a&ount o, in,o&ation via ('-

Applications can expose an4thin! to ('- via 'Aeans

(0) ships with tools ,o li!ht ('- inteaction

*actical use o, ('- e7uies so&e additional tools

('- *ovides 'onitoin! and 'ana!e&ent o, (2's

%o&cat exposes a !eat a&ount o, in,o&ation via ('-

Applications can expose an4thin! to ('- via 'Aeans

(0) ships with tools ,o li!ht ('- inteaction

*actical use o, ('- e7uies so&e additional tools

0esouces
0esouces

*esentation Slides
/ttp955people"apac/e"org5Zsc/ultV5'pac/eCon 1' C)?P5Tomcat Monitoring5

$a!ios passive8chec1 scipt

/ttp9556i=i"apac/e"org5tomcat5tools5nagios+send+passive+c/ec="s/

chec1C9&xpox4
/ttp9556i=i"apac/e"org5tomcat5tools5c/ec=%7m8pro84"pl

Special than1s to Chistophe Alunc1 :'Aeans in,o=

/ttp955oss"68net"org5mbeans"/tml

*esentation Slides
/ttp955people"apac/e"org5Zsc/ultV5'pac/eCon 1' C)?P5Tomcat Monitoring5

$a!ios passive8chec1 scipt

/ttp9556i=i"apac/e"org5tomcat5tools5nagios+send+passive+c/ec="s/

chec1C9&xpox4
/ttp9556i=i"apac/e"org5tomcat5tools5c/ec=%7m8pro84"pl

Special than1s to Chistophe Alunc1 :'Aeans in,o=

/ttp955oss"68net"org5mbeans"/tml


Monitoring Tomcat with JMX
Monitoring Tomcat with JMX


Christopher Schultz
Chief Technology Offcer
Total Child Health, nc!
Christopher Schultz
Chief Technology Offcer
Total Child Health, nc!
* Slides available on the Linux Foundation / ApacheCon2014 web site and at
http://people.apache.o!/"schult#/ApacheCon $A 2014/%o&cat 'onitoin!/
+E& essentiall4 a ;ev?ps C%?3 and eve4thin! +E&
pesentin! toda4 has been so&ethin! +Eve had to do
in &4 own wo1 in that e!ad. '4 own &onitoin!
wo1 is ve4 &uch a wo1 in po!ess.
%his is an intoduction to &onitoin! %o&cat and
even (2' pocesses in !eneal. $othin! +E& !oin! to
pesent is paticulal4 eath8shattein! o di,,icult to
undestand. And thatEs !ood newsD
%hee is eall4 no need to conside wh4 &onitoin! is
necessa43 so letEs 9ust 9u&p i!ht in.
+E& essentiall4 a ;ev?ps C%?3 and eve4thin! +E&
pesentin! toda4 has been so&ethin! +Eve had to do
in &4 own wo1 in that e!ad. '4 own &onitoin!
wo1 is ve4 &uch a wo1 in po!ess.
%his is an intoduction to &onitoin! %o&cat and
even (2' pocesses in !eneal. $othin! +E& !oin! to
pesent is paticulal4 eath8shattein! o di,,icult to
undestand. And thatEs !ood newsD
%hee is eall4 no need to conside wh4 &onitoin! is
necessa43 so letEs 9ust 9u&p i!ht in.


(ava 'ana!e&ent )xtensions
(ava 'ana!e&ent )xtensions

*otocol and A*+ ,o &ana!in! and &onitoin!

Access data via ('- .'beans/

0ead and wite bean attibutes

+nvo1e opeations

0eceive noti,ications

(2' exposes cetain status

%o&cat exposes cetain status

*otocol and A*+ ,o &ana!in! and &onitoin!

Access data via ('- .'beans/

0ead and wite bean attibutes

+nvo1e opeations

0eceive noti,ications

(2' exposes cetain status

%o&cat exposes cetain status

'ana!e and &onito (2' pocesses.
)ve4thin! is 'Aeans
0ead/wite attibutes
+nvo1e opeations
0eceive noti,ications
Aoth the (2' and %o&cat expose these t4pes o,
thin!s via ('-.
'ana!e and &onito (2' pocesses.
)ve4thin! is 'Aeans
0ead/wite attibutes
+nvo1e opeations
0eceive noti,ications
Aoth the (2' and %o&cat expose these t4pes o,
thin!s via ('-.


'onitoin! (2'
'onitoin! (2'

Heap status

%otal3 ,ee3 used &e&o4

5aba!e collection

5C pause ti&es

6eap status

%otal3 ,ee3 used &e&o4

5aba!e collection

5C pause ti&es
%he (2' exposes a lot about its intenal state. 6ee
ae so&e o, the &oe inteestin! ite&s.
%he (2' exposes a lot about its intenal state. 6ee
ae so&e o, the &oe inteestin! ite&s.


'onitoin! %o&cat
'onitoin! %o&cat

Status o, connecto

Status o, e7uest8pocesso thead pool

Status o, data souces

0e7uest pe,o&ance

Status o, connecto

Status o, e7uest8pocesso thead pool

Status o, data souces

0e7uest pe,o&ance
%o&cat has a !eat deal o, in,o&ation available as
well. 6eeEs a sa&ple o, whatEs thee.
%o&cat has a !eat deal o, in,o&ation available as
well. 6eeEs a sa&ple o, whatEs thee.


('- %ools
('- %ools

9console :(;<=

2isual2' :(;<3 app bundle=

'ost po,iles :e.!. >ou<it3 etc.=

Custo& tools usin! 9avax.&ana!e&ent A*+

9console :(;<=

2isual2' :(;<3 app bundle=

'ost po,iles :e.!. >ou<it3 etc.=

Custo& tools usin! 9avax.&ana!e&ent A*+

@hile ('- is an A*+ G potocol3 4ou donEt need to
1now o undestand eithe o, the& to bene,it: tools
alead4 exist.
>ou can alwa4s wite 4ou own i, 4ou need
so&ethin! special.
@hile ('- is an A*+ G potocol3 4ou donEt need to
1now o undestand eithe o, the& to bene,it: tools
alead4 exist.
>ou can alwa4s wite 4ou own i, 4ou need
so&ethin! special.


'onitoin! (2': 6eap
'onitoin! (2': 6eap
An exa&ple o, the (2'Es exposue o, the (ava
heapEs usa!e: initial and &axi&u& values ae
available as well as the cuentl48used &easue&ent.
$otice the $on6eap'e&o4Fsa!e attibute which
has not 4et been .expanded/ as the
6eap'e&o4Fsa!e attibute has. Aoth o, these
attibute values ae epesented b4 ob9ects that
contain &ultiple na&e8value pais. %he ob9ect that
stoes these pais also indicates the data t4pe o,
each value and can include desciptive in,o&ation
,o a client as well.
An exa&ple o, the (2'Es exposue o, the (ava
heapEs usa!e: initial and &axi&u& values ae
available as well as the cuentl48used &easue&ent.
$otice the $on6eap'e&o4Fsa!e attibute which
has not 4et been .expanded/ as the
6eap'e&o4Fsa!e attibute has. Aoth o, these
attibute values ae epesented b4 ob9ects that
contain &ultiple na&e8value pais. %he ob9ect that
stoes these pais also indicates the data t4pe o,
each value and can include desciptive in,o&ation
,o a client as well.


'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
+&a!e: sceenshot ,o& 2isual2' o, %o&catEs 'Aean
tee.
%o&cat povides a wealth o, in,o&ation about its
intenal state. 'uch o, this in,o&ation is &eel4
con,i!uation values that ae ead on statup and do
not chan!e ove ti&e.
%hee is3 howeve3 a !eat deal o, eal8ti&e data
available about the sevlet containe and its vaious
co&ponents. +Ell dive into these pactical exa&ples to
de&onstate the ich data that is available.
+E& !oin! to cove these out8o,8ode with espect to
the top8to8botto& ode shown above in ode to
ease8into so&e o, the concepts.
+&a!e: sceenshot ,o& 2isual2' o, %o&catEs 'Aean
tee.
%o&cat povides a wealth o, in,o&ation about its
intenal state. 'uch o, this in,o&ation is &eel4
con,i!uation values that ae ead on statup and do
not chan!e ove ti&e.
%hee is3 howeve3 a !eat deal o, eal8ti&e data
available about the sevlet containe and its vaious
co&ponents. +Ell dive into these pactical exa&ples to
de&onstate the ich data that is available.
+E& !oin! to cove these out8o,8ode with espect to
the top8to8botto& ode shown above in ode to
ease8into so&e o, the concepts.


'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
%o&cat tac1s the pe,o&ance o, e7uests :in
a!!e!ate= ,o each connecto sepaatel4. A
5lobal0e7uest*ocesso exists ,o each connecto
whee 4ou can obtain in,o&ation about the
pe,o&ance o, the e7uests handled b4 that
paticula connecto.
%o&cat tac1s the pe,o&ance o, e7uests :in
a!!e!ate= ,o each connecto sepaatel4. A
5lobal0e7uest*ocesso exists ,o each connecto
whee 4ou can obtain in,o&ation about the
pe,o&ance o, the e7uests handled b4 that
paticula connecto.


'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
6ee is a view o, one o, %o&catEs
5lobal0e7uest*ocessos. + happen to have H
connectos con,i!ued3 and 4ou can tell the& apat
b4 thei na&es which also indicate a lot about the&:
potocol3 inte,ace addess3 and pot nu&be will
uni7uel4 identi,4 an4 connectoEs
5lobal0e7uest*ocesso.
%hese 5lobal0e7uest*ocessos 1eep tac1 o,
&etics about e7uests such as the nu&be o,
e7uests3 the cu&ulative pocessin! ti&e o, those
e7uests3 and the oveall volu&e o, data pocessed.
6ee is a view o, one o, %o&catEs
5lobal0e7uest*ocessos. + happen to have H
connectos con,i!ued3 and 4ou can tell the& apat
b4 thei na&es which also indicate a lot about the&:
potocol3 inte,ace addess3 and pot nu&be will
uni7uel4 identi,4 an4 connectoEs
5lobal0e7uest*ocesso.
%hese 5lobal0e7uest*ocessos 1eep tac1 o,
&etics about e7uests such as the nu&be o,
e7uests3 the cu&ulative pocessin! ti&e o, those
e7uests3 and the oveall volu&e o, data pocessed.


'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
An4 'Aean can suppot opeations that can be
called via the ('- A*+s. %he
5lobal0e7uest*ocesso beans have a sin!le
opeation: esetCountes. %his opeation as 4ou
&i!ht !uess esets all the collected &etics ,o the
5lobal0e7uest*ocesso to #eo.
An4 'Aean can suppot opeations that can be
called via the ('- A*+s. %he
5lobal0e7uest*ocesso beans have a sin!le
opeation: esetCountes. %his opeation as 4ou
&i!ht !uess esets all the collected &etics ,o the
5lobal0e7uest*ocesso to #eo.


'onitoin! %o&cat: 0e7uests
'onitoin! %o&cat: 0e7uests
All values #eoed8outD
0esettin! these countes can be use,ul i, 4ou want to
&onito pe,o&ance data ove ti&e and want to
peiodicall4 eset the state o, the connectoEs &etics.
All values #eoed8outD
0esettin! these countes can be use,ul i, 4ou want to
&onito pe,o&ance data ove ti&e and want to
peiodicall4 eset the state o, the connectoEs &etics.


'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
Sessions ae anothe thin! 4ou &i!ht want to 1eep
tac1 o,: too &an4 sessions can bo!8down a seve
and cause pe,o&ance poble&s. %he eal poble&
is stoin! lots o, data in the session3 o couse3 but
the nu&be o, sessions can be an i&potant data
point in 4ou seve &onitoin! state!4.
Sessions ae anothe thin! 4ou &i!ht want to 1eep
tac1 o,: too &an4 sessions can bo!8down a seve
and cause pe,o&ance poble&s. %he eal poble&
is stoin! lots o, data in the session3 o couse3 but
the nu&be o, sessions can be an i&potant data
point in 4ou seve &onitoin! state!4.


'onitoin! %o&cat: Sessions
'onitoin! %o&cat: Sessions
'ost use,ul attibutes shown hee: activeSessions3
&axActive3 and expiedSessions. ?ne attibute that
is not shown is the sessionCeation0ate3 which !ives
4ou an idea o, how ,ast sessions ae bein! ceated.
%o&cat actuall4 exposes eve4 session in the
containe via 'Aean opeations. >ou can ,etch a list
o, all session ids3 ,etch attibute values ,o& a
paticula session3 and even expie sessions diectl4.
'ost use,ul attibutes shown hee: activeSessions3
&axActive3 and expiedSessions. ?ne attibute that
is not shown is the sessionCeation0ate3 which !ives
4ou an idea o, how ,ast sessions ae bein! ceated.
%o&cat actuall4 exposes eve4 session in the
containe via 'Aean opeations. >ou can ,etch a list
o, all session ids3 ,etch attibute values ,o& a
paticula session3 and even expie sessions diectl4.


'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
A !eat nu&be o, web applications use a elational
database via (;AC. %hose ;ataSouces con,i!ued
via %o&cat :and not diectl4 in the application3 such
as those con,i!ued b4 Spin!3 6ibenate3 etc.= ae
available ,o inspection.
%o&catEs ;ataSouces have a connection pool with
&ini&u& and &axi&u& si#es :nu&bes o,
connections=3 and a &ax+dle settin! which allows the
pool to !ow and shin1 dependin! upon the
de&and.
A !eat nu&be o, web applications use a elational
database via (;AC. %hose ;ataSouces con,i!ued
via %o&cat :and not diectl4 in the application3 such
as those con,i!ued b4 Spin!3 6ibenate3 etc.= ae
available ,o inspection.
%o&catEs ;ataSouces have a connection pool with
&ini&u& and &axi&u& si#es :nu&bes o,
connections=3 and a &ax+dle settin! which allows the
pool to !ow and shin1 dependin! upon the
de&and.


'onitoin! %o&cat: ;ataSouces
'onitoin! %o&cat: ;ataSouces
Speci,icall43 4ou &i!ht want to ta1e a loo1 at the
nu&Active and nu&+dle attibutes: 4ou can see i,
4ou (;AC connection pool is &eetin! the de&and
o, 4ou uses.
$ote that + have &axActiveI1 since this is a test
s4ste&.
Speci,icall43 4ou &i!ht want to ta1e a loo1 at the
nu&Active and nu&+dle attibutes: 4ou can see i,
4ou (;AC connection pool is &eetin! the de&and
o, 4ou uses.
$ote that + have &axActiveI1 since this is a test
s4ste&.


'onitoin! %o&cat
'onitoin! %o&cat

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation

Status o, data souces

Status o, e7uest8
pocesso thead pool

0e7uest pe,o&ance

Session in,o&ation
)ach o, %o&catEs connectos has a thead pool that
is used to actuall4 pocess the e7uests: once a
e7uest aives3 it is dispatched to a thead in the
pool.
%head pools in %o&cat ae called )xecutos and
&a4 be shaed between connectos3 which is wh4
the4 ae teated sepaatel4 ,o& the Connectos
the&selves.
)xecutos ae li1e the (;AC connection8pools ,o&
the pevious exa&ple: the4 have &ini&u& and
&axi&u& si#es3 as well as an idle ta!et to help
&atch esouces to use de&and.
)ach o, %o&catEs connectos has a thead pool that
is used to actuall4 pocess the e7uests: once a
e7uest aives3 it is dispatched to a thead in the
pool.
%head pools in %o&cat ae called )xecutos and
&a4 be shaed between connectos3 which is wh4
the4 ae teated sepaatel4 ,o& the Connectos
the&selves.
)xecutos ae li1e the (;AC connection8pools ,o&
the pevious exa&ple: the4 have &ini&u& and
&axi&u& si#es3 as well as an idle ta!et to help
&atch esouces to use de&and.


'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
>ou can ,ind out the nu&be o, cuentl48active
e7uests :activeCount=3 the total nu&be o, e7uests
pocessed :b4 the executo3 which &a4 not be the
sa&e as the nu&be pocessed b4 an4 !iven
connecto=3 etc.
>ou can ,ind out the nu&be o, cuentl48active
e7uests :activeCount=3 the total nu&be o, e7uests
pocessed :b4 the executo3 which &a4 not be the
sa&e as the nu&be pocessed b4 an4 !iven
connecto=3 etc.


'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
6ee3 +Eve ,ied8up a little ('ete scipt to put so&e
load on the seve. >ou can see that thee ae J
active theads and the pool si#e has 9u&ped ,o& 4
theads to 213 indicatin! that +Eve put 7uite a load on
the pool K elativel4 spea1in!. %he
co&pleted%as1Count is !on!8up da&aticall4.
:+ suspect the eason + donEt have 21 theads bus488
o &oe K i!ht now is because &4 laptop onl4 has L
lo!ical coes3 so eall4 onl4 L theads can be active
at once K that &eans both ('ete and %o&cat. %he
e7uests ae also pocessed so 7uic1l4 that itEs had
to catch a la!e nu&be o, theads actuall4 active.=
6ee3 +Eve ,ied8up a little ('ete scipt to put so&e
load on the seve. >ou can see that thee ae J
active theads and the pool si#e has 9u&ped ,o& 4
theads to 213 indicatin! that +Eve put 7uite a load on
the pool K elativel4 spea1in!. %he
co&pleted%as1Count is !on!8up da&aticall4.
:+ suspect the eason + donEt have 21 theads bus488
o &oe K i!ht now is because &4 laptop onl4 has L
lo!ical coes3 so eall4 onl4 L theads can be active
at once K that &eans both ('ete and %o&cat. %he
e7uests ae also pocessed so 7uic1l4 that itEs had
to catch a la!e nu&be o, theads actuall4 active.=


'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
A,te a bit &oe load3 +Eve been able to captue the
activeCount !ettin! a bit hi!he.
A,te a bit &oe load3 +Eve been able to captue the
activeCount !ettin! a bit hi!he.


'onitoin! %o&cat: %heads
'onitoin! %o&cat: %heads
;onEt want to tac1 the values 4ousel, ove ti&eM $o
poble&: 9ust double8clic1 on an4 nu&eic value and
2isual2' will !aph it ,o 4ou ove ti&e.
;onEt want to tac1 the values 4ousel, ove ti&eM $o
poble&: 9ust double8clic1 on an4 nu&eic value and
2isual2' will !aph it ,o 4ou ove ti&e.


'onitoin! >ou Application
'onitoin! >ou Application

'onito Application *ocesses

*e,o&ance 'etics

?n8the8,l4 e8con,i!uation

'onito Application *ocesses

*e,o&ance 'etics

?n8the8,l4 e8con,i!uation
So3 the (2' and %o&cat expose in,o&ation about
the&selves. %hatEs !eat ,o &onitoin! the state o,
the (2' and the sevlet containe3 but what about
4ou own applicationEs healthM
>ou have caches3 othe data stoes3 co&plex
ob9ects3 and a little bit o, eve4thin! !oin! on inside
4ou own application. 6ow can we pee1 unde those
covesM
So3 the (2' and %o&cat expose in,o&ation about
the&selves. %hatEs !eat ,o &onitoin! the state o,
the (2' and the sevlet containe3 but what about
4ou own applicationEs healthM
>ou have caches3 othe data stoes3 co&plex
ob9ects3 and a little bit o, eve4thin! !oin! on inside
4ou own application. 6ow can we pee1 unde those
covesM


'onitoin! >ou Application
'onitoin! >ou Application

@ite an 'Aean

Ceate an +nte,ace: Foo'Aean

Ceate an +&ple&entation: Foo

Ceate an -'L 'Aean descipto

;eplo4 pac1a!e to %o&cat

*ublish the 'Aean to the 'Aean seve

Bue4 / invo1e as necessa4

* )xa&ple code will be available at
http://people.apache.o!/"schult#/ApacheCon $A 2014/%o&cat 'onitoin!/
A !eat wa4 to do this is to wite 4ou own 'Aean.
%hen 4ou can use all the tools descibed in this
pesentation to tac1 abita4 details about 4ou
application.
0e&e&be that 4ou can also invo1e opeations on
'Aeans3 so 4ou can even chan!e the state and ta1e
whateve actions 4ou ,eel ae wothwhile ,o& a ('-
client.
+tEs eas4 to wite 4ou own 'Aean: 9ust ,ollow the
steps above. +Ell show a si&ple exa&ple in the next
,ew slides.
A !eat wa4 to do this is to wite 4ou own 'Aean.
%hen 4ou can use all the tools descibed in this
pesentation to tac1 abita4 details about 4ou
application.
0e&e&be that 4ou can also invo1e opeations on
'Aeans3 so 4ou can even chan!e the state and ta1e
whateve actions 4ou ,eel ae wothwhile ,o& a ('-
client.
+tEs eas4 to wite 4ou own 'Aean: 9ust ,ollow the
steps above. +Ell show a si&ple exa&ple in the next
,ew slides.


)xa&ple 'Aean
)xa&ple 'Aean

Sevlet Filte that captues total e7uest

pocessin! ti&e

%i&esta&p pio to e7uest

%i&esta&p a,te e7uest

Add the delta to a ('-8accessible counte:

0e7uestStats
%o&cat also povides e7uest8pocessin! &etics on
a pe8sevlet basis. @ant to 1now how the (S*
sevlet is pe,o&in!M $o poble&: %o&cat alead4
tac1s that in,o&ation ,o 4ou.
%he poble& is that itEs not ve4 ,ine8!ained: 4ou !et
&etics ,o& the si&plest index.9sp &ixed8in with
4ou *e,o&Lon!%ansactionAnd*oduce*;F.9sp
nu&bes. %hatEs not paticulal4 convenient.
So3 +E& !oin! to wite a Filte that captues this 1ind
o, data and &a1es it available via ('-. >ou can have
&ultiple instances o, the Filte &apped to di,,eent
F0L pattens3 and 4ouEll !et a sepaate set o, &etics
,o each o, the&.
%o&cat also povides e7uest8pocessin! &etics on
a pe8sevlet basis. @ant to 1now how the (S*
sevlet is pe,o&in!M $o poble&: %o&cat alead4
tac1s that in,o&ation ,o 4ou.
%he poble& is that itEs not ve4 ,ine8!ained: 4ou !et
&etics ,o& the si&plest index.9sp &ixed8in with
4ou *e,o&Lon!%ansactionAnd*oduce*;F.9sp
nu&bes. %hatEs not paticulal4 convenient.
So3 +E& !oin! to wite a Filte that captues this 1ind
o, data and &a1es it available via ('-. >ou can have
&ultiple instances o, the Filte &apped to di,,eent
F0L pattens3 and 4ouEll !et a sepaate set o, &etics
,o each o, the&.


0e7uestStats 'Aean
0e7uestStats 'Aean

@ite an 'Aean
public interface RequestStatsMBean {
public long getProcessingTime();
public long getRequestCount();
public void resetCounters();

public class RequestStats

implements RequestStatsMBean {
!"""#
public void updateStats(long
timestamp$ ServletRequest request$ long
elapsed) {

%total&lapsedTime"add'nd(et(elapsed);

%requestCount"increment'nd(et();

public long getProcessingTime(){
return %total&lapsedTime"get();

public long getRequestCount() {
return %requestCount"get();

public void resetCounters() {
%total&lapsedTime"set()l);
%requestCount"set()l);

Fo %o&catEs 'Aean seve i&ple&entation3 4ou

have to wite an inte,ace as well as a concete class.
$o supises in the code3 hee.
$ote that +E& usin! Ato&icLon! ob9ects :declaations
not shown ,o bevit4= because the4 ae bein! used
in a &ulti8theaded context and need to e&ain
theadsa,e.
Fo %o&catEs 'Aean seve i&ple&entation3 4ou
have to wite an inte,ace as well as a concete class.
$o supises in the code3 hee.
$ote that +E& usin! Ato&icLon! ob9ects :declaations
not shown ,o bevit4= because the4 ae bein! used
in a &ulti8theaded context and need to e&ain
theadsa,e.


0e7uestStats 'Aean
0e7uestStats 'Aean

@ite an 'Aean descipto

*mbeans+descriptors,
*mbean name-.RequestStats. """,
*operation name-.getProcessingTime.
description-.(ets t/e total number of
milliseconds spent processing requests".
impact-.0123.
returnT4pe-.long. 5,
*operation name-.getRequestCount.
description-.(ets t/e total number
of requests processed".
impact-.0123.
returnT4pe-.long. 5,
*operation
name-.resetCounters.
description-.Resets all
counters".
impact-.'CT031.
returnT4pe-.void. 5,
*5mbean,
*5mbeans+descriptors,
%o&catEs docu&entation states that 4ou &ust ceate
an &beans8desciptos.x&l ,ile and place it in the
sa&e pac1a!e as 4ou 'Aean inte,ace3 but + have
,ound that it is not actuall4 a e7uie&ent.
Aut3 itEs a !ood idea to wite the descipto because it
docu&ents what 4ou attibutes &ean and what 4ou
and opeations do. ('- clients can ead this
in,o&ation and pesent it to the use. ;ocu&entation
is alwa4s nice.
:+ was unable to !et %o&cat to ead &4 &beans8
desciptos.x&l ,ile ,o so&e eason. )al48on in &4
wo13 + ecall it wo1in!3 but it stopped wo1in! at
so&e point and + wasnEt able to discove the cause.=
%o&catEs docu&entation states that 4ou &ust ceate
an &beans8desciptos.x&l ,ile and place it in the
sa&e pac1a!e as 4ou 'Aean inte,ace3 but + have
,ound that it is not actuall4 a e7uie&ent.
Aut3 itEs a !ood idea to wite the descipto because it
docu&ents what 4ou attibutes &ean and what 4ou
and opeations do. ('- clients can ead this
in,o&ation and pesent it to the use. ;ocu&entation
is alwa4s nice.
:+ was unable to !et %o&cat to ead &4 &beans8
desciptos.x&l ,ile ,o so&e eason. )al48on in &4
wo13 + ecall it wo1in!3 but it stopped wo1in! at
so&e point and + wasnEt able to discove the cause.=


0e7uestStats 'Aean
0e7uestStats 'Aean

Ceate (A0

(ava inte,ace

(ava i&ple&entation

&beans8desciptos.x&l

*ut (A0 into CA%AL+$ACAAS)/lib

Ceate (A0

(ava inte,ace

(ava i&ple&entation

&beans8desciptos.x&l

*ut (A0 into CA%AL+$ACAAS)/lib

*ac1a!e8up the 'Aean and put it into %o&catEs lib
diecto4. $ote that the bean must be placed8into the
containeEs lib diecto4 and not with 4ou web
application3 othewise 4ou is1 a pinned8ClassLoade
&e&o4 lea1 duin! edeplo4&ent.
+ believe %o&cat e7uies that 4ou 'Aean be in the
lib/ diecto4 an4wa43 do 4ou &a4 not actuall4 have a
choice.
*ac1a!e8up the 'Aean and put it into %o&catEs lib
diecto4. $ote that the bean must be placed8into the
containeEs lib diecto4 and not with 4ou web
application3 othewise 4ou is1 a pinned8ClassLoade
&e&o4 lea1 duin! edeplo4&ent.
+ believe %o&cat e7uies that 4ou 'Aean be in the
lib/ diecto4 an4wa43 do 4ou &a4 not actuall4 have a
choice.


0e7uestStats 'Aean
0e7uestStats 'Aean

@ite the Filte

public void init(2ilterConfig config) {
MBeanServer server - getServer();
server"registerMBean(%stats$ ne6
3b7ect1ame(.&8ample9RequestStats-RequestStats$name-. : filter1ame;));

public void do2ilter(""") {
timestamp - elapsed - S4stem"currentTimeMillis();
c/ain"do2ilter(request$ response);
elapsed - S4stem"currentTimeMillis() + elapsed;
%stats"updateStats(timestamp$ request$ elapsed);

$ow3 we need to wite the Filte that will actuall4
captue the data and publish the 'Aean to the
seve.
%he init &ethod hee e!istes the 'bean :Cstats=3
and the doFilte &ethod 9ust ti&es e7uests as the4
pass8thou!h3 then updates the stats on the bean.
$ow3 we need to wite the Filte that will actuall4
captue the data and publish the 'Aean to the
seve.
%he init &ethod hee e!istes the 'bean :Cstats=3
and the doFilte &ethod 9ust ti&es e7uests as the4
pass8thou!h3 then updates the stats on the bean.


0e7uestStats 'Aean
0e7uestStats 'Aean

'ap the Filte

*filter,
*filter+name,servlet+request+stats*5filter+name,
*filter+class,filters"RequestStats2ilter*5filter+class,
*init+param,
*param+name,name*5param+name,
*param+value,servlets*5param+value,
*5init+param,
*5filter,
*filter+mapping,
*filter+name,servlet+request+stats*5filter+name,
*url+pattern,5servlets5;*5url+pattern,
*5filter+mapping,
*filter,*filter+name,7sp+request+stats*5filter+name,*filter+
class,filters"RequestStats2ilter*5filter+class,*init+param,*param+name,name*5param+
name,*param+value,7sps*5param+value,*5init+param,*5filter,
*filter+mapping,*filter+name,7sp+request+stats*5filter+name,*url+pattern,57sp5;*5url+
pattern,*5filter+mapping,

'ap the Filte

*filter,
*filter+name,servlet+request+stats*5filter+name,
*filter+class,filters"RequestStats2ilter*5filter+class,
*init+param,
*param+name,name*5param+name,
*param+value,servlets*5param+value,
*5init+param,
*5filter,
*filter+mapping,
*filter+name,servlet+request+stats*5filter+name,
*url+pattern,5servlets5;*5url+pattern,
*5filter+mapping,
*filter,*filter+name,7sp+request+stats*5filter+name,*filter+
class,filters"RequestStats2ilter*5filter+class,*init+param,*param+name,name*5param+
name,*param+value,7sps*5param+value,*5init+param,*5filter,
*filter+mapping,*filter+name,7sp+request+stats*5filter+name,*url+pattern,57sp5;*5url+
pattern,*5filter+mapping,
LetEs &ap two instances o, the Filte to two di,,eent
F0L pattens to see what thin!s loo1 li1e.
LetEs &ap two instances o, the Filte to two di,,eent
F0L pattens to see what thin!s loo1 li1e.


0e7uestStats 'Aean
0e7uestStats 'Aean
Chec1 it: (S*s and sevlets have sepaate stats. +Eve
put a bit o, ('ete load on the seve to !et so&e
nu&bes.
Chec1 it: (S*s and sevlets have sepaate stats. +Eve
put a bit o, ('ete load on the seve to !et so&e
nu&bes.


0e7uestStats 'Aean
0e7uestStats 'Aean
@e can also eset countes3 9ust li1e with the built8in
%o&cat 'Aeans.
@e can also eset countes3 9ust li1e with the built8in
%o&cat 'Aeans.


Auto&ated 'onitoin!
Auto&ated 'onitoin!

0e&ote Access

La!e Scale

Constant

0e&ote Access

La!e Scale

Constant
All the exa&ples thus ,a have used 2isual2' which
is a 5F+ inte,ace. @hile thatEs ,un ,o inspectin! a
sin!le seve and &a4be doin! so&e scoutin! ,o
inteestin! data available3 itEs not !oin! to wo1 in the
eal wold o, poduction &onitoin!.
All the exa&ples thus ,a have used 2isual2' which
is a 5F+ inte,ace. @hile thatEs ,un ,o inspectin! a
sin!le seve and &a4be doin! so&e scoutin! ,o
inteestin! data available3 itEs not !oin! to wo1 in the
eal wold o, poduction &onitoin!.


Auto&ated 'onitoin!
Auto&ated 'onitoin!

0e&ote Access

La!e Scale

Constant

$eed &oe toolsD

0e&ote Access

La!e Scale

Constant

$eed &oe toolsD

Auto&ated 'onitoin!
Auto&ated 'onitoin!

$a!ios

Simple

Flexible

@ell8deplo4ed

$o8cost co&&unit4 vesion available

$a!ios

Si&ple

Flexible

@ell8deplo4ed

$o8cost co&&unit4 vesion available

LetEs use $a!ios: a widel48deplo4ed &onitoin!
s4ste&.
LetEs use $a!ios: a widel48deplo4ed &onitoin!
s4ste&.


Auto&ated 'onitoin!
Auto&ated 'onitoin!
%he ASF uses $a!ios and %o&cat exposes data via
('-. LetEs see how we can &a4 the two.
%he ASF uses $a!ios and %o&cat exposes data via
('-. LetEs see how we can &a4 the two.


$a!ios 'onitoin!
$a!ios 'onitoin!

*lu!8in achitectue :i.e. abita4 scipts=

Feel48available ('- plu!8in: chec1C9&x

< "5c/ec=%7m8 +>
service97m89rmi95557ndi5rmi955local/ost9??))57m8rmi@
+3 7ava"lang9t4pe-Memor4 +' 1onAeapMemor4>sage +B used@
+6 CD)))))) +c E)))))))
FMG H'R101( 1onAeapMemor4>sage"used-CD)I)JJ)

*lu!8in achitectue :i.e. abita4 scipts=

Feel48available ('- plu!8in: chec1C9&x

< "5c/ec=%7m8 +>
service97m89rmi95557ndi5rmi955local/ost9??))57m8rmi@
+3 7ava"lang9t4pe-Memor4 +' 1onAeapMemor4>sage +B used@
+6 CD)))))) +c E)))))))
FMG H'R101( 1onAeapMemor4>sage"used-CD)I)JJ)
$a!ios suppots plu!8ins and theeEs one ,o ,etchin!
data via ('-: chec1C9&x: i, 4ou 1now the ob9ectEs
na&e3 4ou can !et data ,o& the co&&and8line.
$a!ios suppots plu!8ins and theeEs one ,o ,etchin!
data via ('-: chec1C9&x: i, 4ou 1now the ob9ectEs
na&e3 4ou can !et data ,o& the co&&and8line.


$a!ios 'onitoin!
$a!ios 'onitoin!

*oble&s with chec1C9&x

Co&plex con,i!uation ,o e&ote ('-

(2' launch ,o eve4 chec1

Couse8!ained authentication options

*oble&s with chec1C9&x

Co&plex con,i!uation ,o e&ote ('-

(2' launch ,o eve4 chec1

Couse8!ained authentication options

%hee ae so&e caveats with chec1C9&x. %hin1 about
how &an4 values 4ou &i!ht want to &onito:
spinnin!8up 14 (2's eve4 &inute &i!ht 9ust be
consideed a waste o, s4ste& esouces.
%hee ae so&e caveats with chec1C9&x. %hin1 about
how &an4 values 4ou &i!ht want to &onito:
spinnin!8up 14 (2's eve4 &inute &i!ht 9ust be
consideed a waste o, s4ste& esouces.


$a!ios 'onitoin!
$a!ios 'onitoin!

Altenative ?ption: %o&catEs ('-*ox4Sevlet

('- data available via 6%%*

Can use %o&catEs authentication tools

< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK @
+6 CD)))))) +c E)))))))
FMG CR0T0C'N9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K +
AeapMemor4>sage + =e4 KusedK - ?))JOICPJ

Altenative ?ption: %o&catEs ('-*ox4Sevlet

('- data available via 6%%*

Can use %o&catEs authentication tools

< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK @
+6 CD)))))) +c E)))))))
FMG CR0T0C'N9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K +
AeapMemor4>sage + =e4 KusedK - ?))JOICPJ
* chec1C9&xpox4 can be ,ound at
http://wi1i.apache.o!/to&cat/tools/chec1C9&xpox4.pl
%o&cat has ('-*ox4Sevlet.
chec1C9&xpox4 is a little *el scipt + wote to ,etch
data ,o& ('-*ox4Sevlet and povide $a!ios8
,iendl4 output.
Sa&e basic ,eatues o, chec1C9&x except that (ava
and the ('- potocol aenEt actuall4 used: we use
%o&catEs 6%%*8tp8('- pox4 instead.
%o&cat has ('-*ox4Sevlet.
chec1C9&xpox4 is a little *el scipt + wote to ,etch
data ,o& ('-*ox4Sevlet and povide $a!ios8
,iendl4 output.
Sa&e basic ,eatues o, chec1C9&x except that (ava
and the ('- potocol aenEt actuall4 used: we use
%o&catEs 6%%*8tp8('- pox4 instead.


$a!ios 'onitoin!
$a!ios 'onitoin!
6eeEs a !lance at so&e values sa&pled in a
poduction settin!. @eEll tal1 about the ??') one
late.
6eeEs a !lance at so&e values sa&pled in a
poduction settin!. @eEll tal1 about the ??') one
late.


('- Co&&and8line %ic1s
('- Co&&and8line %ic1s

Show all lo!!ed8in usena&es

for sessionid in Q6get +3 + K/ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-listSession0
dsK @
S sed +e .s5 5@n5g.
S grep KT!)+D'+Ua+V#@:@(@"";@)@L<K ;@
do 6get +3 W ./ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-getSession't
tributeMps-<sessionid$user. ; done C,5dev5null @
S grep >ser

Show all lo!!ed8in usena&es

for sessionid in Q6get +3 + K/ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-listSession0
dsK @
S sed +e .s5 5@n5g.
S grep KT!)+D'+Ua+V#@:@(@"";@)@L<K ;@
do 6get +3 W ./ttp955user9p6dR/ost5manager57m8pro84L
invo=e-Catalina9t4pe-Manager$conte8t-5m4app$/ost-local/ostMop-getSession't
tributeMps-<sessionid$user. ; done C,5dev5null @
S grep >ser
@e stoe a .use/ bean in ou sessions3 and so we
can use so&e co&&and8line tic1s &ixed with data
,o& chec1C9&xpox4 to list all the cuentl4 lo!!ed8in
uses.
@e can use si&ila tic1s to expie all sessions that
donEt epesent a lo!!ed8in use.
@e stoe a .use/ bean in ou sessions3 and so we
can use so&e co&&and8line tic1s &ixed with data
,o& chec1C9&xpox4 to list all the cuentl4 lo!!ed8in
uses.
@e can use si&ila tic1s to expie all sessions that
donEt epesent a lo!!ed8in use.


%ac1in! 2alues ?ve %i&e
%ac1in! 2alues ?ve %i&e

So&e &etics ae best obseved as deltas

Session count

0e7uest eo count

0e7uies that 4ou have a histo4 o, data

0e7uies that 4ou consult the histo4 o, that data

chec1C9&xpox4 povides such capabilities

So&e &etics ae best obseved as deltas

Session count

0e7uest eo count

0e7uies that 4ou have a histo4 o, data

0e7uies that 4ou consult the histo4 o, that data

chec1C9&xpox4 povides such capabilities

@hat about data whose ate8o,8chan!e is &oe
i&potant than its cuent valueM
chec1C9&xpox4 can stoe the pevious value
etieved and then co&pae duin! the next
invocation.
@hat about data whose ate8o,8chan!e is &oe
i&potant than its cuent valueM
chec1C9&xpox4 can stoe the pevious value
etieved and then co&pae duin! the next
invocation.


%ac1in! 2alues ?ve %i&e
%ac1in! 2alues ?ve %i&e
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)CCOJD)P$
delta-!"""#
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ??EJ)X?PP$
delta-??ICOCP)
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)DCXP)IX$
delta-+PIPC)JJ
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)CCOJD)P$
delta-!"""#
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ??EJ)X?PP$
delta-??ICOCP)
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-7ava"lang9t4pe-Memor4Matt-AeapMemor4>sageM=e4-usedK +6 EEIIPPEC +c I)EE?XPJ ++6rite number"out
++compare number"out
FMG 3B9 3B + 'ttribute get K7ava"lang9t4pe-Memor4K + AeapMemor4>sage + =e4 KusedK - ?)DCXP)IX$
delta-+PIPC)JJ
LetEs watch heap &e&o4 usa!e ove a ,ew
invocations.
LetEs watch heap &e&o4 usa!e ove a ,ew
invocations.


%ac1in! 2alues ?ve %i&e
%ac1in! 2alues ?ve %i&e

Session count

%o&cat actuall4 povides this alead4 via 'ana!eEs

sessionCreateRate attibute

0e7uest eos
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-Catalina9t4pe-RequestProcessor$6or=er-./ttp+nio+?CO")")"?+
JC?O.$name-AttpRequest?Matt-errorCountK +6 ? +c ?) ++6rite errors"t8t ++compare
errors"t8t
FMG 3B9 3B + 'ttribute get KCatalina9t4pe-RequestProcessor$6or=er-./ttp+nio+
?CO")")"?+JC?O.$name-AttpRequest?K + errorCount - )$ delta-)

Session count

%o&cat actuall4 povides this alead4 via 'ana!eEs

sessionCreateRate attibute

0e7uest eos
< "5c/ec=%7m8pro84 +> K/ttp955local/ost5manager57m8pro84L
get-Catalina9t4pe-RequestProcessor$6or=er-./ttp+nio+?CO")")"?+
JC?O.$name-AttpRequest?Matt-errorCountK +6 ? +c ?) ++6rite errors"t8t ++compare
errors"t8t
FMG 3B9 3B + 'ttribute get KCatalina9t4pe-RequestProcessor$6or=er-./ttp+nio+
?CO")")"?+JC?O.$name-AttpRequest?K + errorCount - )$ delta-)
%hee ae lots o, data whose ates o, chan!e ae
&oe i&potant than thei cuent values. Session
count and eo count ae a&on! the&.
%hee ae lots o, data whose ates o, chan!e ae
&oe i&potant than thei cuent values. Session
count and eo count ae a&on! the&.


;etectin! ?ut?,'e&o4
;etectin! ?ut?,'e&o4

Many sources of OOME

Heap exhaustion

*e&5en exhaustion

6it thead li&it

6it ,ile descipto li&it

'an4 souces o, ??')

6eap exhaustion

*e&5en exhaustion

6it thead li&it

6it ,ile descipto li&it

LetEs tal1 about ?ut?,'e&o4)os. ?, all
&onitoin! 7uestions +Eve head about (ava web
applications3 this one is alwa4s the ,ist: how can +
!et noti,ied about an ??')M
LetEs tal1 about ?ut?,'e&o4)os. ?, all
&onitoin! 7uestions +Eve head about (ava web
applications3 this one is alwa4s the ,ist: how can +
!et noti,ied about an ??')M


;etectin! ?ut?,'e&o4
;etectin! ?ut?,'e&o4

%wo t4pes o, heap ??')

?ne thead !eneates lots o, local e,eences

All theads collaboate to !eneate !loball48

eachable ob9ects :e.!. session data=

Fo&e is ecoveable3 latte is not

>ou want to be noti,ied in an4 case

%wo t4pes o, heap ??')

?ne thead !eneates lots o, local e,eences

All theads collaboate to !eneate !loball48

eachable ob9ects :e.!. session data=

Fo&e is ecoveable3 latte is not

>ou want to be noti,ied in an4 case

LetEs ,ocus on heap ??') ,o a &o&ent.
LetEs ,ocus on heap ??') ,o a &o&ent.


'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
)ach &e&o4 pool in the (2' has an 'Aean to
epesent it. 6eeEs the *e&5en &e&o4 pool. >ou
can see the cuent usa!e and thee ae a nu&be o,
.theshold/ values that 4ou can set.
@heneve the &e&o4 usa!e exceeds the theshold
value3 the (2' ince&ents the
Fsa!e%hesholdCount value and also publishes a
notification to all inteested listenes.
)ach &e&o4 pool in the (2' has an 'Aean to
epesent it. 6eeEs the *e&5en &e&o4 pool. >ou
can see the cuent usa!e and thee ae a nu&be o,
.theshold/ values that 4ou can set.
@heneve the &e&o4 usa!e exceeds the theshold
value3 the (2' ince&ents the
Fsa!e%hesholdCount value and also publishes a
notification to all inteested listenes.


'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
6eeEs the ?ld :tenued= 5eneation with its usa!e
expanded so 4ou can see the individual values. +Eve
also set a theshold o, ou!hl4 11N 'iA3 which + 1now
is too low o, a theshold: weEll exceed this be,oe the
5C 1ic1s8in.
LetEs e8un &4 ('ete load test ,o& ealie 9ust to
chew8thou!h so&e heap &e&o4 and see i, we can
bea1 the theshold.
6eeEs the ?ld :tenued= 5eneation with its usa!e
expanded so 4ou can see the individual values. +Eve
also set a theshold o, ou!hl4 11N 'iA3 which + 1now
is too low o, a theshold: weEll exceed this be,oe the
5C 1ic1s8in.
LetEs e8un &4 ('ete load test ,o& ealie 9ust to
chew8thou!h so&e heap &e&o4 and see i, we can
bea1 the theshold.


'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
%hee: the Fsa!e%hesholdCount value is now 2 :up
,o& #eo=. $ote that the Fsa!e%heshold)xceeded
value is false even thou!h we have cleal4 bo1en
that theshold. %he .)xceeded/ attibute value will
onl4 be true while the theshold is still being
exceeded. +tEs not a one8wa4 tip: once the &e&o4
usa!e ,alls below the theshold3 that value will !o
bac1 to false.
>ou can see hee that the cuent usa!e is about
100'iA3 less than the 11N'iA theshold we set.
%hee: the Fsa!e%hesholdCount value is now 2 :up
,o& #eo=. $ote that the Fsa!e%heshold)xceeded
value is false even thou!h we have cleal4 bo1en
that theshold. %he .)xceeded/ attibute value will
onl4 be true while the theshold is still being
exceeded. +tEs not a one8wa4 tip: once the &e&o4
usa!e ,alls below the theshold3 that value will !o
bac1 to false.
>ou can see hee that the cuent usa!e is about
100'iA3 less than the 11N'iA theshold we set.


'e&o4 *ool %hesholds
'e&o4 *ool %hesholds
6e43 loo1 at thatD @e can !et a ,ail4 detailed :tust
&e= noti,ication about the &e&o4 theshold
condition. Cool.
6e43 loo1 at thatD @e can !et a ,ail4 detailed :tust
&e= noti,ication about the &e&o4 theshold
condition. Cool.


'e&o4 *ool %hesholds
'e&o4 *ool %hesholds

Choice o, how to detect exceeded8

theshold conditions

*ollin! usin! chec1C9&xpox4

0e!iste a noti,ication listene ,o& (ava

Have that listener take some action

Choice o, how to detect exceeded8

theshold conditions

*ollin! usin! chec1C9&xpox4

0e!iste a noti,ication listene ,o& (ava

6ave that listene ta1e so&e action

As usual3 we have so&e options.
*ollin! doesnEt see& li1e a !eat idea. @hat about
these noti,icationsM
As usual3 we have so&e options.
*ollin! doesnEt see& li1e a !eat idea. @hat about
these noti,icationsM


;etect ?ut?,'e&o4
;etect ?ut?,'e&o4

'onitoin! 'e&o4 %hesholds

Set theshold on statup

0e!iste a noti,ication listene :callbac1=

@atch .exceeded/ count :poll=

0epot to &onitoin! so,twae :$a!ios=

0epeat ,o each &e&o4 pool 4ou want to watch

6ope the (2' does not ,ail duin! noti,ication

%his is !ettin! idiculous

'onitoin! 'e&o4 %hesholds

Set theshold on statup

0e!iste a noti,ication listene :callbac1=

@atch .exceeded/ count :poll=

0epot to &onitoin! so,twae :$a!ios=

0epeat ,o each &e&o4 pool 4ou want to watch

6ope the (2' does not ,ail duin! noti,ication

%his is !ettin! idiculous

5eatD All 4ou have to do is O wait. %hee &ust be a
bette wa4. ?ne that is less ,a!ile. %his stu,, is
supposed to be eas4.
5eatD All 4ou have to do is O wait. %hee &ust be a
bette wa4. ?ne that is less ,a!ile. %his stu,, is
supposed to be eas4.


;etectin! ?ut?,'e&o4
;etectin! ?ut?,'e&o4

(2' has an easie wa4

Fse 8--:?n?ut?,'e&o4)o to un a
co&&and on first ??') detected b4 the
(2'

$eed a co&&and to noti,4 $a!ios

(2' has an easie wa4

Fse 8--:?n?ut?,'e&o4)o to un a
co&&and on first ??') detected b4 the
(2'

$eed a co&&and to noti,4 $a!ios

Fo& &4 ,ield eseach and anecdotal evidence3
8--:?n?ut?,'e&o4)o see&s to be the &ost
eliable wa4 to !et noti,ications o, ??')s.
%hee is one poble&: 4ou onl4 !et noti,ied o, the first
??') detected3 so i, 4ou want to !et anothe
noti,ication3 4ouEe !oin! to have to bounce the (2'.
Fo& &4 ,ield eseach and anecdotal evidence3
8--:?n?ut?,'e&o4)o see&s to be the &ost
eliable wa4 to !et noti,ications o, ??')s.
%hee is one poble&: 4ou onl4 !et noti,ied o, the first
??') detected3 so i, 4ou want to !et anothe
noti,ication3 4ouEe !oin! to have to bounce the (2'.


$oti,4 $a!ios on ??')
$oti,4 $a!ios on ??')

Scipt that waps cul

< curl +si @
++data+urlencode Kcmd%t4p-E)K @
++data+urlencode Kcmd%mod-CK @
++data+urlencode ./ost-m4/ost. @
++data+urlencode .service-FYM9Aeap933M&. @
++data+urlencode .plugin%state-C. @
++data+urlencode .plugin%output-33M& CR0T0C'N. @
K/ttps955monitoring+/ost5nagios5cgi+bin5cmd"cgiK

Scipt that waps cul

< curl +si @
++data+urlencode Kcmd%t4p-E)K @
++data+urlencode Kcmd%mod-CK @
++data+urlencode ./ost-m4/ost. @
++data+urlencode .service-FYM9Aeap933M&. @
++data+urlencode .plugin%state-C. @
++data+urlencode .plugin%output-33M& CR0T0C'N. @
K/ttps955monitoring+/ost5nagios5cgi+bin5cmd"cgiK
Scipt can be ,ound at http://wi1i.apache.o!/to&cat/tools/na!ios8send8passive8
chec1.sh
6ee is a cul co&&and that can be used to po1e a
passive8chec1 into $a!ios. %o encapsulate the
co&&and3 as well as to pevent the 6%%*
authentication in,o&ation ,o& appeain! in a ps
listin!3 weEll wap this co&&and in a scipt.
$ote that 4ouEll pobabl4 want to use a $a!ios
sevice con,i!ued ,o onl4 .passive/ chec1s since no
active chec1s ae eall4 possible. Also3 disable .,lap
detection/ othewise the sevice will i&&ediatel4
appea to be .,lappin!/ when 4ou epot an ??') to
$a!ios K +E& not sue wh4 K and 4ou wonEt !et an
actual noti,ication because $a!ios thin1s itEs doin!
4ou a ,avo.
6ee is a cul co&&and that can be used to po1e a
passive8chec1 into $a!ios. %o encapsulate the
co&&and3 as well as to pevent the 6%%*
authentication in,o&ation ,o& appeain! in a ps
listin!3 weEll wap this co&&and in a scipt.
$ote that 4ouEll pobabl4 want to use a $a!ios
sevice con,i!ued ,o onl4 .passive/ chec1s since no
active chec1s ae eall4 possible. Also3 disable .,lap
detection/ othewise the sevice will i&&ediatel4
appea to be .,lappin!/ when 4ou epot an ??') to
$a!ios K +E& not sue wh4 K and 4ou wonEt !et an
actual noti,ication because $a!ios thin1s itEs doin!
4ou a ,avo.


'onitoin! %o&cat with ('-
'onitoin! %o&cat with ('-

('- *ovides 'onitoin! and 'ana!e&ent o, (2's

%o&cat exposes a !eat a&ount o, in,o&ation via ('-

Applications can expose an4thin! to ('- via 'Aeans

(0) ships with tools ,o li!ht ('- inteaction

*actical use o, ('- e7uies so&e additional tools

('- *ovides 'onitoin! and 'ana!e&ent o, (2's

%o&cat exposes a !eat a&ount o, in,o&ation via ('-

Applications can expose an4thin! to ('- via 'Aeans

(0) ships with tools ,o li!ht ('- inteaction

*actical use o, ('- e7uies so&e additional tools

Su&&a4.
Su&&a4.


0esouces
0esouces

*esentation Slides
/ttp955people"apac/e"org5Zsc/ultV5'pac/eCon 1' C)?P5Tomcat Monitoring5

$a!ios passive8chec1 scipt

/ttp9556i=i"apac/e"org5tomcat5tools5nagios+send+passive+c/ec="s/

chec1C9&xpox4
/ttp9556i=i"apac/e"org5tomcat5tools5c/ec=%7m8pro84"pl

Special than1s to Chistophe Alunc1 :'Aeans in,o=

/ttp955oss"68net"org5mbeans"/tml

*esentation Slides
/ttp955people"apac/e"org5Zsc/ultV5'pac/eCon 1' C)?P5Tomcat Monitoring5

$a!ios passive8chec1 scipt

/ttp9556i=i"apac/e"org5tomcat5tools5nagios+send+passive+c/ec="s/

chec1C9&xpox4
/ttp9556i=i"apac/e"org5tomcat5tools5c/ec=%7m8pro84"pl

Special than1s to Chistophe Alunc1 :'Aeans in,o=

/ttp955oss"68net"org5mbeans"/tml
0esouces.
BuestionsM
0esouces.
BuestionsM