Introduction to MQ

Sam Goulden
IBM MQ L3 Service

MQ Technical Conference v2.0.1.8

Agenda
¡ Messaging
„ What is messaging and why use it?
„ What does MQ give you?

¡ Fundamentals of IBM MQ
„ Messaging models
„ Key components
„ Messaging applications
„ MQ Environments
„ Security
„ Reliability and availability
„ Administration
„ MQ Advanced

MQ Technical Conference v2.0.1.8

What is messaging?
¡ It connects your applications!

From the simplest pairs

of applications…

...to the most complex

¡ and breaks the tight coupling… business processes.

MQ Technical Conference v2.0.1.8

 Why use it?
¡ Extended reach

¡ Reliability

¡ Scalability

¡ Flexibility

¡ Provides for simplification of application development

„ Ubiquity
„ Easy to change and scale
„ Focus on the business logic

¡ Important regardless of the initial scale of deployment

MQ Technical Conference v2.0.1.8

 Direct communication between applications

Request an action
App B
App A

Wait for a response

¡ Issues with this ‘synchronous’ approach

„Both applications A and B must always be available for A to continue
„A cannot do anything whilst B is processing A’s request
„What is B fails whilst A is waiting for it to complete?
„What is B needs to handle a high workload of different priority requests?

MQ Technical Conference v2.0.1.8

Fragility of tight coupling
As systems become more Maximum number of
tightly coupled, their connections goes up with
reliance on each other the square of
increases the number of systems

The cost of a failure of a Scaling systems

process increases independently
to respond to requirements
becomes unmanageable

A process was originally designed for one, well-defined, ¡

purpose...it then needed to change to meet new requirements

Being able to respond rapidly to internal and external

challenges by rapidly modifying existing services gives a
competitive advantage

MQ Technical Conference v2.0.1.8

 Adding flexibility with Messaging
Send request Perform action
App B
App A

App B

¡ A ‘queue’ is placed between the two applications

„Allows A to continue immediately, without waiting for B
„Allows B is efficiently process a queue of work
„Overcomes availability of B versus A – “store and forward” of messages

MQ Technical Conference v2.0.1.8

 What if I NEED a response?
Send request Perform action

App B
App A
Process response Send response

¡ Using messaging still adds value!

„Process the response whenever it becomes available
—No need for A to be idle whilst the request is performed
„Application B processes its workload efficiently and can handle spikes in
load
„Application, network and infrastructure failures are handled

MQ Technical Conference v2.0.1.8

 The power of events

App

¡ Not all information is distributed on a one-to-one basis

¡ Think about streams of information

„ Regularly updated information - such as stock prices or sensor data
„ Business events - such as ‘new customer’ or ‘purchase’

¡ Publish / Subscribe messaging is the solution!

„The owner of the information simply publishes it on a topic
„Anybody who is interested simply subscribes to the topic

MQ Technical Conference v2.0.1.8

Messaging Models

MQ Technical Conference v2.0.1.8

Point-to-point
Message consumer

Message producer Message consumer

Message consumer

MQ Technical Conference v2.0.1.8

Point-to-point
Message consumer

Message producer Message consumer

Message consumer

Publish/subscribe Subscription Message consumer

Topic
Message producer Subscription Message consumer

Subscription Message consumer

MQ Technical Conference v2.0.1.8

Durable publish/subscribe in action
¡ Durable subscriptions result in published messages being retained when the subscriber is not
connected to the messaging provider.

Subscriber 1

ra ble
Message
Du
Publisher
Topic

No n
-du
rab Subscriber 2
le

MQ Technical Conference v2.0.1.8

IBM MQ

MQ Technical Conference v2.0.1.8

IBM MQ timeline
MQSeries WebSphere MQ IBM MQ
MQ Light developer toolkit
and Bluemix service MQ on
Multi-instance Multiple cluster
QMGR XMIT queue MQ Light IBM Cloud
JMS 2.0 channels
Common System Pattern
criteria Hypervisors Application Pattern
Eclipse V9.1
XML .NET HTTP Integrated

V9
AJAX Messaging
REST Managed

V8
Web 2.0 File Transfer
JMS JMS 1.1
Mobile SSL RCMS
New LTS
SOAP Release:
Multi- V7.5 Aug 2018
MQI
Assured platform V7.0.1
V7.1
delivery
Pub/sub V7 IBM MQ
V6 Appliance First LTS
IBM MQ IBM MQ RELEASE:
V5.3 Advanced June 2016
V5.2 IBM MQ Advanced
V5 V5.1 Managed for Two delivery
V2 V2.1
V1.1 File Developers models:
IBM IBM MQ
IBM MQ MQTT IBM MQ Transfer Advanced 1) Long Term
IBM IBM
Everyplace Low Message Service
MQSeries MessageSight
Latency Security 2) Continuous
1990s 2000s Delivery
2010s
MQ Technical Conference v2.0.1.8
What MQ adds to messaging
Enterprise Messaging
Reliability Scalability Ubiquity Security
Assured message High performance Breadth of support for Data encryption and
delivery “Once and solution platforms and integrity
once only” environments
Incremental growth of End use authentication
Resiliency and high applications and Multiple application and authorisation
availability of the infrastructure environments and APIs
infrastructure to suit many styles Audit trails for
configuration and data
Continued support and flows
interoperability of
systems for over twenty
years

MQ Technical Conference v2.0.1.8

 Anatomy of an MQ system
System 1 System 2
¡ Applications
„ Applications use MQ clients to connect to an MQ queue manager
„ Applications can connect to queue managers either on the same system (BINDINGS App A
mode) or remotely over a network (CLIENT mode)

¡ Queue Managers QMgr QMgr

„ A queue manager is a runtime that hosts messaging resources such as queues and
their messages
„ A queue manager manages the flow and storage of messages Channel
„ Each queue manager runs on a single system
„ Multiple queue managers can be connected together using channels and messages
routed between them

¡ Queues App B
„ Queues are a named resource where messages sent to by applications, stored by the
queue manager and retrieved by applications

¡ Messages System 3
„ Are just chunks of data
„ Applications build messages to send and receive

¡ Channels
„ Channels define a way for one queue manager to connect to another queue manager
„ Channels can be manually configured or dynamically created as and when needed
using MQ Clusters

MQ Technical Conference v2.0.1.8

IBM MQ
IBM MQ Architecture

MQ Technical Conference v2.0.1.8

 Distributed Architectures z/OS @ MQ 9.1

¡ Used for connectivity of heterogeneous HP @ MQ 8.0.0.1

systems
QMgr

QMgr

¡ “Store and Forward” system to account for

network outages
Windows @ MQ 7.5

Linux @ MQ 8.0.0.5
¡ This is the ‘original’ deployment pattern
for MQ QMgr
AIX @ MQ 7.1
QMgr

¡ Queue managers will interoperate with QMgr

other queue managers and clients at any

other version of MQ

MQ Technical Conference v2.0.1.8

 MQ hub architecture
¡ A ‘hub’ of systems running queue
managers on a standard deployment QMgr

¡ Applications connect as clients from

remote systems
QMgr

¡ This pattern has gained popularity as

networks improve and administration
QMgr

costs go up

MQ Technical Conference v2.0.1.8

 Connecting queue managers together
¡ Channels connect queue managers together, allowing messages to flow
QMgr
QMgr

between them

¡ Two options:
QMgr

QMgr

„ Manual configuration of channels

— Each channel relationship must be defined on both ends
QMgr

— Additional resource also need to be defined

(transmission queues and remote queues)

QMgr
„ MQ clusters QMgr

— Once queue managers join a cluster (a pair of special

channels must be defined) they can route messages to any
other clustered resource in the cluster without requiring
further, per queue manager, configuration.
QMgr
— As queue manager networks grow, clusters become a benefit QMgr

— Clusters also enable workload balancing and availability routing

of messages QMgr

MQ Technical Conference v2.0.1.8

IBM MQ
Applications

MQ Technical Conference v2.0.1.8

MQ APIs – How do I connect my apps to my queue
manager?

¡ MQI
¡ JMS
¡ MQ Light API
¡ MQTT
¡ REST API Messaging (point to point only!)

MQ Technical Conference v2.0.1.8

MQ APIs - MQI (MQ Interface)
¡ C, COBOL, Java

¡ MQ’s proprietary API offering full access to MQ’s capabilities

QM1 MQCONN(QM)
…
MQOPEN(Q)
…
APP.Q
MQPUT(Q)
…
MQGET(Q)
…
MQCLOSE(Q)
…
MQDISC(QM)

MQ Technical Conference v2.0.1.8

 MQ APIs - Java Message Service (JMS/XMS)
¡ JMS is part of the JEE specification.
„ Fully supported in application servers such as WSAS, Liberty, WebLogic and more

¡ Simplifies programming for Java developers

¡ No MQ coding knowledge needed!

¡ XMS syntactically the same as JMS V1.1 but for C, C++ and C#

// Lookup the MQ specific objects in JNDI

QM1 Context jndiContext
ConnectionFactory cf
= new InitialContext();
= (ConnectionFactory) jndiContext.lookup("jms/QM1");
Destination dest = (Destination) jndiContext.lookup("jms/APP.Q");

// Establish a connection with the queue manager & create JMS objects
APP.Q JMSContext context
JMSConsumer consumer
= cf.createContext();
= context.createConsumer(dest);

// Get a message
Message msg = consumer.receive();
MQ Technical Conference v2.0.1.8
MQ APIs - MQ Light
¡ AMQP based API

¡ Node.js, Java, Ruby

¡ Connects cloud applications to MQ!

# Receive:
var mqlight = require('mqlight');
var recvClient = mqlight.createClient({service: 'amqp://localhost'});
recvClient.on('started', function() {
recvClient.subscribe('news/technology');
recvClient.on('message', function(data, delivery) {
console.log(data);
});
});

MQ Technical Conference v2.0.1.8

 MQ APIs - MQ Telemetry (MQTT)
● Product extension supports connectivity for smart devices to the
enterprise Enterprise
Oil rig

● Utilises the open standard MQTT protocol Vehicle

• a lightweight, public, low bandwidth

messaging protocol for scenarios where
enterprise messaging clients are too big or
bandwidth intensive. Sensor
e.g.
RFID Retail
Store

● Java, C and JavaScript libraries provided,

but you can “roll your own” that implement Pervasive
Device Medical
the MQTT v3 spec
Petrol
Forecourt Smartphones

MQ Technical Conference v2.0.1.8

 REST Messaging
• The new HTTP server support in MQ 9.0.x provides the App
App
platform for a properly integrated REST API solution App POST …/qmgr/QM1/queue/Q1/message
DELETE…/qmgr/QM1/queue/Q1/message

HTTPS
• Allowing applications to put and get messages from a
queue without installing any MQ software locally

REST API
• Ideal for environments with native REST support,
such as common JavaScript libraries including QM1
NodeJS, and AngularJS

Existing channels
MQ App
App
App
• Can only be used for point to point messaging Q1 MQPUT(Q1)
MQGET(Q1)

• For full functionality and resiliency an MQ client

should still be used

MQ Technical Conference v2.0.1.8

Messaging APIs
• All interoperate with each other!
– Any application can receive messages from any other application

.NET
MQTT

QMgr

QMgr
JMS

COBOL
QMgr

MQ Technical Conference v2.0.1.8

¡ LearnMQ ¡ Totally new to MQ? ¡ Step-by-step guide ¡ Tutorials on building
Learn the basics to getting up and your applications
running with MQ

¡ Finding it hard to
get developers
started with MQ?

¡ Point them to:

developer.ibm.com/
¡

messaging/learn-
mq

MQ Technical Conference v2.0.1.8

 “Once and once only delivery”
¡ Message persistence
„ Persistent messages
— Stored to disk
— Queued messages are recovered following a server failure
– No matter what the failure, as long as the disks are intact, so will your messages be
„ Non-persistent messages
— Kept in memory as much as possible (better performance)
— Queued messages are lost in the event of a server failure or restart

¡ Transactions
„ Multiple messaging operations can be coordinated as a transaction
„ Messaging applications are often updating other resources based on messages
— E.g. Receive a message, insert the data to a database
„ MQ applications can coordinate messaging operations with other transactional resources
— A queue manager can be an XA transaction coordinator
— Or coordinated externally, for example a JEE application server such as WebSphere Application Server
„ Available in MQI, JMS and XMS APIs

¡ Combining persistent messages with transactions gives you

once and once only delivery of messages from an application’s point of view

MQ Technical Conference v2.0.1.8

 - Non Persistent
Transactional Messaging - Persistent

Request Queue

Message producer Message consumer

Reply Queue

Persistent Queues

MQ Technical Conference v2.0.1.8

Transactional Messaging

Request Queue

Message producer Message consumer

Reply Queue

Persistent Queues

Transaction
Combining persistent messages with transactions gives you
once and once only delivery of messages from an application’s point of view
MQ Technical Conference v2.0.1.8
IBM MQ
Environments
On-premise & Cloud

MQ Technical Conference v2.0.1.8

¡ Run IBM MQ in ¡ On-premise, ¡ Run it yourself in ¡ Let IBM host it for
any location or software and the any cloud, public or you with its new
cloud exactly as MQ Appliance private managed MQ
service in IBM Cloud
you
need it

AWS
AWS
AWSAzure NEW
IBM Z
Linux AWS
AIX
Windows
Solaris
HPE IBMi
Appliance …
IBM Cloud Private

Private cloud

MQ Technical Conference v2.0.1.8

¡ MQ in ¡ MQ has been ¡ More recently it has ¡ And now MQ
supporting Docker been demonstrating Advanced is
Containers containers since how to get the most available as a fully
2015 with images on from containers supported product
Docker Hub and using Kubernetes with IBM Cloud
Docker Store and Private, a
sample setups on Kubernetes-based
Github solution from IBM

MQ Technical Conference v2.0.1.8

¡ MQ on IBM Cloud
Provision queue managers directly into IBM
Cloud
IBM owns the infrastructure and the
responsibility to keep the systems up to date
and running
The customer owns the configuration and the
monitoring of the messaging

Try the service for free at:

console.bluemix.net/catalog/services/mq

Hosted on

MQ Technical Conference v2.0.1.8

IBM MQ
Security

MQ Technical Conference v2.0.1.8

Security provided on Client to Queue Manager connections
Channel Authentication
(BLOCKADDR)

SSL/TLS

Channel Authentication
(ADDR/USER/SSL Map)

Security Exit

MQRC_NONE
Connection Authentication
Or

Channel Authentication
MQRC_NOT_AUTHORIZED (BLOCKUSER)

Authorization

MQ Technical Conference v2.0.1.8

IBM MQ
High Availibility

MQ Technical Conference v2.0.1.8

IBM MQ HA capabilities ¡ Support for HA
clusters and
¡ Client connectivity

network storage • Automatic

reconnection
Come to my other talks! ¡ Multi-instance
Benefitting from the MQ Appliance queue managers • CCDTs
Room: Zebrawood (Windows, Linux,
• Pre-connect exit
Time: Monday, 14:30PM or UNIX)
Wednesday 09:45AM
¡ IBM MQ Appliance

¡ Replicated Data Queue Managers (Linux)

¡ Queue-sharing groups (z/OS)

¡ Support for cloud orchestration frameworks

e.g. Kubernetes, Docker Swarm, Apache
Mesos

MQ Technical Conference v2.0.1.8

IBM MQ
Administration

MQ Technical Conference v2.0.1.8

Administration and ¡ Command line ¡ Tivoli and third-
party tooling
monitoring Come to my other talks! ¡ Scripting

¡ Programmatic APIs
Administering MQ, The MQ Console
and the MQ REST API! ¡ REST API
Room: Zebrawood
Time: Tuesday, 8:30AM or
Wednesday 13:00PM

¡ Web console ¡ GUI tooling

GSE Nordic Conference / © 2018 IBM Corporation MQ Technical Conference v2.0.1.8

IBM MQ
MQ Advanced

MQ Technical Conference v2.0.1.8

 MQ Advanced Message Security
¡ Secures application data even before it is passed to MQ
¡ Upgrade from base MQ
App A
„ No changes to existing applications or network required

AMS Client
MQ standard security: App B Interceptor
• Industry standard TLS channels (256-bit)
• Certified for Common Criteria
• Authentication is based on Operating System identifier of local
process
• Message data can be encrypted in transport but not when it resides in
the queues AMS Interceptor

MQ
MQ Advanced Message Security adds: Queue Manager
• Authentication policies are based on certificates associated with
each application
• Message data is protected end-to-end – including when it resides in
queues
• Much finer granularity in security policies
• No changes needed to applications or queues

MQ Technical Conference v2.0.1.8

MQ Managed File Transfer

MQMFT Eclipse Tooling

Servers
MQMFT MQMFT MQMFT
Coordination MQ MQ MQ
Queue Manager

MQMFT MQMFT MQMFT MQMFT

MQMFT Eclipse Tooling

Clients

MQ Technical Conference v2.0.1.8

Where to go now?

MQ Technical Conference v2.0.1.8

¡ LearnMQ ¡ Totally new to MQ? ¡ Step-by-step guide ¡ Tutorials on building
Learn the basics to getting up and your applications
running with MQ

¡ Finding it hard to
get developers
started with MQ?

¡ Point them to:

developer.ibm.com/
¡

messaging/learn-
mq

MQ Technical Conference v2.0.1.8

Where do I get more information?
IBM MQ Knowledge Center
http://www.ibm.com/software/integration/wmq/library/

IBM Messaging developerWorks

developer.ibm.com/messaging

Youtube
https://www.youtube.com/user/IBMmessagingMedia

MQ Technical Conference v2.0.1.8

Copyright and Trademarks
© IBM Corporation 2017. All Rights Reserved.

IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business
Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be
trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at
“Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml.

MQ Technical Conference v2.0.1.8

 Thanks for listening

Questions?

Sam Goulden IBM UK, IBM MQ L3 Service

[email protected]

MQ Technical Conference v2.0.1.8