IBM Software Group

Michael S. Patton
[email protected]
WebSphere Support Technical Exchange
WebSphere Message Broker Top Ten
Problems
A Level 2 Perspective
IBM Software Group
WebSphere

Support Technical Exchange

2
Agenda
1) Failing Deploys
2) Toolkit is slow or terminates all together
3) DFE Memory Leak
4) Debugger (RAC) not working (Remote Agent Controller)
5) Broker and RAD (Rational Application Developer)
6) Message Flow Hangs
7) Publish and Subscribe
8) MRM and Validation
9) eSupport
10) Noteworthy APARS and Useful Websites
IBM Software Group
WebSphere

Support Technical Exchange

3
Failing Deploys
Three possible scenarios
No response at all
Negative response
Error on deploy
Common Deploy Error messages
BIP2080, BIP2066, BIP2087, & BIP1536
(See additional notes in the Addendum)
IBM Software Group
WebSphere

Support Technical Exchange

4
Failing Deploy No Response
Check Event Viewer / message logs
Other messages give a clue as to what happened.
Check for MQ FDC errors
MQ Series could not send the deploy message for some
reason
No responses should still result in a timeout response
IBM Software Group
WebSphere

Support Technical Exchange

5
Failing Deploy No Response
Infrastructure problems
Large deploy message?
MAXMSGLENGTH large enough?
MQ Objects
Any errors with MQ Series
Check MQ logs.
MQ Return codes propagated through BIP2070.
Database error codes propagated through BIP2322
(See additional notes in the Addendum)
IBM Software Group
WebSphere

Support Technical Exchange

6
Failing Deploy Negative Response
BIP1536E: Configuration Manager unable to register for
internal subscriptions on the broker
The deploy was processed
The configmgr simply cannot receive updates from
broker
Thus Toolkit will not reflect the current status of the
broker
BIP7150E: User id not authorized to put to queue on
queue manager
The configmgr service id must be defined on broker box
and must have permission to put to transmit queue on
broker queue manager to send messages to the
configmgr QM
IBM Software Group
WebSphere

Support Technical Exchange

7
Failing Deploy Negative Response
BIP2080E, BIP2066E
Configuration timeout and ConfigurationDelayTimeout
Are they set high enough?
Mqsichangebroker <brokername> -g <value> -k <value>
-g = how much time to allow for the deploy
-k = how much time to wait before starting the g timer.
Complex Flows tale time to quiesce
Deploy to EG with large number of flows takes time
Messages currently in mid-stream will either finish or rollback
prior to processing a a new deploy
IBM Software Group
WebSphere

Support Technical Exchange

8
Failing Deploy Hints and Tips
Minimize the scope of the deploy
Deploy smaller groups of message flows and/or sets
Thus increased deploy performance
Tune configuration timeout and configuration delay timeout
Remove the flow from the broker and deploy a fresh copy
Back out any updates that were applied since the last deploy
Stop the broker, clear all SYSTEM.BROKER.*.* queues,
restart the broker and deploy again.
IBM Software Group
WebSphere

Support Technical Exchange

9
Failing Deploy Hints and Tips
Set the environment variable MQSI_SHOW_DEPLOY
Works on Windows and Unix
Inserts a BIP8099I into the message log
Broker restart needed for variable to be picked up
Helps to isolate the problem
Check the message logs for any other MQ, DB or OS errors
that may provide a clue
Questions on Deploys?
IBM Software Group
WebSphere

Support Technical Exchange

10
Toolkit is slow or terminates
Common Problems
Saving a project is slow
Create a bar file is slow
Toolkit is gorging on CPU
JVM Exceptions
Debugging
Saving a project or Create a bar file or Toolkit gorging on CPU
Message Flow and ESQL validation is memory intensive
Disable this validation in preferences
Window Preferences ESQL and Mapping Validation
Toolkit shuts down due to JVM exception
Inspect the <install dir>/workspace/.metadata/.log for exception
message
IBM Software Group
WebSphere

Support Technical Exchange

11
Toolkit is slow or terminates
Debugging Issues
Large projects and debugging complex flows / testing
large messages in Toolkit are resource intensive
Toolkit (mqsistudio.exe) creates a JVM with default
maximum heap size
The default is the real memory on the system
Increase this value by starting the toolkit from the
command line
CD to the <Install Dir>\eclipse directory
Mqsistudio vmargs Xmx<value>M
<value> = number of Megabytes
IBM Software Group
WebSphere

Support Technical Exchange

12
DFE Memory Leak
DFE is expected to eventually reach a plateau
DFE memory consumption will increase to accommodate the
demands of the flows it contains
Size and type of message
Complexity of the flow
DFE is multithreaded, Flows are threads within
DFE obtains memory from OS and hands it out to threads
DFE cannot return memory to OS
Threads can return memory to the DFE when a flow
ends
Memory is returned to the OS when the DFE ends
IBM Software Group
WebSphere

Support Technical Exchange

13
DFE Memory Leak
A memory leak may be simply the DFE has not reached a plateau
IF DFE shuts down due to memory exhaustion, try giving it
more memory
AIX Alter the DataFlowEngines executable
ldedit -b maxdata:0x40000000 DataFlowEngine
4 segments = 1 GB
http://publib.boulder.ibm.com/infocenter/pseries/index.jsp?topic
=/com.ibm.aix.doc/cmds/aixcmds3/ldedit.htm
Env Var: MQSI_THREAD_STACK_SIZE= <value in bytes>
Increase the JVM Heap size for the broker
mqsichangeproperties BROKER_NAME -e EG_NAME -o
ComIbmJVMManager -n jvmMinHeapSize -v 16777216
IBM Software Group
WebSphere

Support Technical Exchange

14
Debugger not working
Common problems:
List of Execution Groups do not show up
List of Message Flows do not show up.
Agent Controller uses default port 10002 for communication
Check port status with netstat
Port must have status of Listen
Is there a firewall?
Default port on RAC server can be changed
Edit the file <Rac Install Dir>/config/serviceconfig.xml
Edit also the toolkit Window Preferences FlowDebug
IBM Software Group
WebSphere

Support Technical Exchange

15
Debugger not working
Agent controller must be started before the broker
In configuration file serviceconfig.xml, change element <Allow
host=LOCAL> to <Allow host=All>
RAC writes its log message to
<RAC_Inst_Dir>/config/servicelog.log
Logging level and detail can be modified in the serviceconfig.xml
file as well
Default is LoggingLevel=WARNING and
LoggingDetail=LOW
Change to LoggingLevel=DEBUG and
LoggingDetail=HIGH
Check to see that installed RAC is correct for CSD
(See additional notes in the Addendum)
IBM Software Group
WebSphere

Support Technical Exchange

16
Broker and RAD (Rational App Dev)
WBIMB Version 6
Toolkit V6.0 installs RAD version 6.0.0.1
Toolkit V6.0.0.1 installs RAD version 6.0.1.1
Usually best to let the toolkit install the RAD
Compatibility errors to install Toolkit V6.0 on RAD 6.0.1.1
Install will fail
IBM Software Group
WebSphere

Support Technical Exchange

17
Message Flow Hangs
Messages are not picked up from the input queue
Message is stuck in the flow
Exception Handling
Did you wire a catch or failure terminal?
Is there a Back Out Queue defined (MQ Series)
Has the DLQ for the QM been set?
Aggregate Reply Exception Handling
Are the unknown, timeout, catch or fail terminals wired?
Has the MQMD Header been built?
A message will loop if it cannot be put to a queue.
IBM Software Group
WebSphere

Support Technical Exchange

18
Publish and Subscribe
Collectives (Interbroker communications)
Communication is TCPIP except for the initial handshake (via
MQ)
If a new broker is added to the collective, or a previous removed
broker is re-added, then you must restart all the brokers, and
issue a complete topology deploy.
This forces the brokers to share the initial data for comm via MQ
Subscription RegOpt VarUserId
Allows any other user to delete/alter the subscription
Does not indicate multiple owners for a subscription
Subscriptions are QM:Q:CorrelationId
How to register multiple topics on the same queue
APAR IC48412
IBM Software Group
WebSphere

Support Technical Exchange

19
Publish and Subscribe
Subscription messages sent between cloned brokers are not
persistent
Slight chance that subscription messages shared between
cloned brokers will be lost if MQ Series is shut down
APAR IY74763 will send subscription requests persistent.
Leading slash (/) in Topic will fail
Using a leading slash in the Topic will cause the EG to
continuously respawn
Also occurs with double slash in the Topic
APAR IY81324
IBM Software Group
WebSphere

Support Technical Exchange

20
MRM and Validation
Nillable Attributes on XML Elements not removed
Results in a validation failure
XML nil=true acts as a placeholder for repeats or arrays
Should simple imply that particular elements value is NULL
The schema allows the value to be null, but the attribute
nil=true will cause a validation error because the element
does not have this as an attribute in the model
Solution: Use NULLXMLSchema in the Physical Format
Importing a C Header file with a reserved C/C++ keyword
Using a keyword as a structure name will fail when creating a
message definition from a C Header file.
Cannot use Reserved Keywords as structure or variable
names
IBM Software Group
WebSphere

Support Technical Exchange

21
MRM and Validation
XML Parser performance is slow when parsing tag with large
number of entity references.
The XML Doc contains many entity references ie. &lt;
Stop the XML parser from parsing the entities
Enclose in CDATA prior to submitting the msg to the flow
Process msg as blob to remove the entities
XMLNS identical Namespaces, different prefixes
Two NS Prefixes point to the same URL
Xmlns:NS1=www.ibm.com and xmlns:NS2=www.ibm.com
The separate prefixes will not be maintained
The broker does works with expanded URLs, not prefixes
The URLs are functionally equivalent
IBM Software Group
WebSphere

Support Technical Exchange

22
Support
New support initiative geared towards self help
Single interface for known, published problems, PTFs,
APARs and Support Packs
http://www-306.ibm.com/software/integration/
wbimessagebroker/support/
Information from multiple sources is updated daily
IBM Software Group
WebSphere

Support Technical Exchange

23
Support Website
Product
Search
Engine
Download
PTFs and
Support Packs
IBM Software Group
WebSphere

Support Technical Exchange

24
Noteworth APARs and useful Websites
IC48712 Changing a variable in the toolkit while debugging
causes the debugger to disconnect (V6.0)
JR23282 BOs generated by Ariba Adapter do not match the
schema generated by the same framework (All)
IC48399 Main Map is not using schema of submap (V5.0)
APAR IC48389 Toolkit interpreting different types as the
same type (V5.0)
APAR IC48373 Debug status of Message Flow not preserved
during deploy (V6.0)
(See additional notes in the Addendum)
IBM Software Group
WebSphere

Support Technical Exchange

25
Noteworth APARs and useful Websites
WMQ Family, WBI Brokers homepage
http://www-306.ibm.com/software/integration/wmq/
http://www-306.ibm.com/software/integration/wbimessagebroker/
WMQ Support Site
including Books and Manuals, Lifecycle, Support Packs, Supported Software
http://www-306.ibm.com/software/integration /wbimessagebroker/support/
Red Books
http://www.redbooks.ibm.com
News Groups
News://news.software.ibm.com/ibm.software.websphere.mqintegrator/
http://www.mqseries.net/
(See additional notes in the Addendum)
IBM Software Group
WebSphere

Support Technical Exchange

26
Additional WebSphere Product Resources
Discover the latest trends in WebSphere Technology and implementation,
participate in technically-focused briefings, webcasts and podcasts at:
www.ibm.com/developerworks/websphere/community/
Learn about other upcoming webcasts, conferences and events:
www.ibm.com/software/websphere/events_1.html
Join the Global WebSphere User Group Community: www.websphere.org
Access key product show-me demos and tutorials by visiting IBM Education
Assistant: ibm.com/software/info/education/assistant
Learn about the Electronic Service Request (ESR) tool for submitting
problems electronically:
www.ibm.com/software/support/viewlet/probsub/ESR_Overview_viewlet_swf
.html
Sign up to receive weekly technical My support emails:
www.ibm.com/software/support/einfo.html
IBM Software Group
WebSphere

Support Technical Exchange

27
Questions and Answers
IBM Software Group
WebSphere

Support Technical Exchange

28
Addendum
IBM Software Group
WebSphere

Support Technical Exchange

29
Notes (Failing Deploys):
BIP 2080 : Execution Group <insert_0> failed to return an
internal configuration <insert_1> message response within a
<insert_2> second time period.
BIP 2066 : Broker <insert_0> (UUID <insert_1>) was unable
to retrieve an internal configuration response message for
execution group '<insert_2>' within the <insert_3> second
Configuration Timeout.
BIP 2087 : Broker <insert_0> was unable to process the
internal configuration message.
BIP 1536 : The Configuration Manager was unable to register
for internal subscriptions with broker <insert_0>.
IBM Software Group
WebSphere

Support Technical Exchange

30
Notes (No Response):
BIP 2070 : A problem was detected with WebSphere MQ
while issuing <insert_0> for WebSphere MQ queue
'<insert_1>', WebSphere MQ queue manager '<insert_2>'.
MQCC=<insert_3>, MQRC=<insert_4>.
BIP2322 : Database error: SQL State '<insert_0>'; Native
Error Code '<insert_1>'; Error Text '<insert_2>'.
IBM Software Group
WebSphere

Support Technical Exchange

31
Notes (Debugger not working):
RAC log: What we usually see is :
error=53, which on AIX means : Unable to create a socket So RAC
is unable to open a socket to the XP client usually it is because of
firewall, filter or anything on the network
IBM Software Group
WebSphere

Support Technical Exchange

32
Notes (Support Website):
You should start your search with minimal search terms and
special keywords. Then, to reduce the number of item
returned, add more specialized keywords. Search terms are
not case sensitive Special words like OR and NOT must be
capitalized. For example
mqsi* wbi* R500 JVM "deploy failure" NOT toolkit
IBM Software Group
WebSphere

Support Technical Exchange

33
Notes (Additional APARS)
APAR IC48712
A breakpoint is set between two nodes in the Toolkit debugger. When the variables associated with the message
are displayed, the user wishes to change the value of a variable. The variable is right clicked and the option to
change the variable is selected . OK is now selected (after the variable is changed or it can be left unchanged)
and the Toolkit disconnects.
APAR JR23282
Ariba Buyer does not have verbs for child objects. For this reason, the adapter was not designed to populate the
verb value for child objects. The xsd generated by the adapter framework though requires verbs for all objects.
APAR IC48399
A user has their main map in one broker schema and submaps in another broker schema (which is not the default
schema). When creating a mapping, the user opts to call an existing submap. The tool generates the submap
call for the user, but the call doesn't include the broker schema of the submap. This results in an unresolved
function call. The code for generating the submap call has been modified so that it no longer assumes that
submaps always reside in the same broker schema as the caller.
IBM Software Group
WebSphere

Support Technical Exchange

34
Notes (Additional APARS)
APAR IC48389
The source and target are defined in two separate message sets. Both source and target have
their own global types. The type of the source and type of the target have the same type name
but in different namespace. In both message sets, there is no prefix defined for the
namespace of the types. In these cases the tool thought the two types were identical, it
therefore created a deep copy from source to target. The two types are actually different, and
a submap call should be created. The code used to determine if two types are the same has
been modified so that namespace prefix, or the lack of it, are taken into consideration.
APAR IC48373
The debug status of a message flow in the toolkit is not preserved when the flow is deployed
again. This has now been corrected and the debug status is now correctly passed on to the
database.