Communication Patterns Using Data-Centric Publish/Subscribe

Sumant Tambe, Ph.D.
Senior Software Research Engineer
Real-Time Innovations, Inc.
www.rti.com

Agenda

• Communication Models
• Data Distribution Service Standard
• Data-Centricity 101
– Demo
• Communication Patterns
– Request/Reply
– Guaranteed Delivery

10/11/2012 2

Middleware Communication Models

10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 3

RPC Vs. Pub-Sub/Messaging/Data-Distribution
• RPC (WebServices, CORBA, DCOM) offer a remote method abstraction
– Familiar OO programming model
– Results in a tightly-coupled system
• Forces synchronous invocations
• Imposes global object model
• Limited QoS(appearance of local method call)
• Lack robustness: cascading points of failure
– Typically built on top of TCP:
• Impacts scalability and time-determinism
– Best-suited to smaller, closely-coupled systems
• Pub-Sub (Messaging Data-Distribution) offer a queue-based and/or
replicated-data model
– Subsystems are decoupled in time, space, and synchronization
• Contracts established by verifying QoS compatibility
– Supports a variety of transports including multicast UDP
– Better suited for high-performance and real-time

Queue Vs. Pub/Sub
• Queue
– Message sent to Queue
– Multiple readers can read from the queue
– Each message is delivered to ONLY one reader
• Readers “affect each other”
– Apps:
• Job Scheduling, Load Balancing, Collaboration

• Pub/Sub
– Message sent to Topic
– Multiple readers can subscribe to Topic with or
without filters
– Each message delivered to ALL subscribers that
pass filter
• Readers are decoupled
– Apps:
• Notifications, Information Distribution


Pub/Sub Vs. Data-Distribution

• Pub-Sub
– Only messages no concept of data
– Each message is interpreted without context
– Messages must be delivered FIFO or according to
some “priority” attribute
– No Caching of data
– Simple QoS: filters, durability, lifespan
• Data-Distribution
– Messages represent update to data-objects
– Data-Objects identify by a key Data Bus
– Middleware maintains state of each object
– Objects are cached. Applications can read at
leisure
– Smart QoS
• Ownership, History (per key), Deadline
– Subsumes Pub-Sub

DDS: Standards-based Integration Infrastructure for
Critical Applications

Streaming
Sensors Events
Data

Real-Time Enterprise
Actuators
Applications Applications

© 2009 Real-Time Innovations, Inc.

Systems that interact with the Real World

• Must adapt to changing environment
• Cannot stop processing the information
• Live within world-imposed timing

Beyond traditional interpretation of real-time


Real-Time Data Distribution


Family of Specifications

2008 2009 2010 2010 2012 2013
UML Profile DDS for DDS DDS-STD-C++ DDS Security RPC over DDS
for DDS Lw CCM X-Types DDS-JAVA5

App 2004 App App
DDS Spec

DDS 2006 DDS DDS
Implementation DDS Implementation Implementation
Interoperablity

Network / TCP / UDP / IP

© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 10

Broad Adoption

• Vendor independent
Cross-vendor portability

– API for portability
– Wire protocol for interoperability
• Multiple implementations
– 10 of API DDS API

– 8 support RTPS
• Heterogeneous Middleware
– C, C++, Java, .NET (C#, C++/CLI)
– Linux, Windows, VxWorks, other DDS Real-Time
embedded & realtime Publish-Subscribe
Wire Protocol (RTPS)
• Loosely coupled

Cross-vendor interoperability

© 2009 Real-Time Innovations, Inc. 11

DDS adopted by key programs
• European Air Traffic Control
– DDS used to interoperate ATC centers
• UK Generic Vehicle Architecture
– Mandates DDS for vehicle comm.
– Mandates DDS-RTPS for interop.
• DISR (DoD Information Technology
Standards Registry)
– Mandates DDS for Pub-Sub API and Interop.
• US Navy Open Architecture
– Mandates DDS for Pub-Sub
• SPAWAR NESI
– Mandates DDS for Pub-Sub SOA © 2010 Real-Time Innovations, Inc.

Evolution of DataBus
Data-centricity basics

Everyday Example: Schedule Meeting via Emails

Alternative Process #1 (message-centric):
1. Email: “Meeting Monday at 10:00.”
2. Email: “Here’s dial-in info for meeting…”
3. Email: “Meeting moved to Tuesday”

4. You: “Where do I have to be? When?”
5. You: (sifting through email messages…)

14

Everyday Example: Schedule Meeting Using a Calendar

Alternative Process #2:
1. Calendar: (add meeting Monday at 10:00)
2. Calendar: (add dial-in info)
3. Calendar: (move meeting to Tuesday)

4. You: “Where do I have to be? When?”
5. You: (check calendar. Contains
consolidated-state)
The difference is state!
The infrastructure consolidates changes and maintains it
15

What’s the Difference? State.

• Objects have identity and “State” (“data”) is a
attributes
– The meeting will run 1:00– snapshot of those
2:00 in the conference room. attributes and
– My friend’s phone number is characteristics.
555-1234 his email is…
– The car is blue and is
traveling north from
Sunnyvale at 65 mph. If the infrastructure
• …whether they exist in maintains the state,
the real world, in the the application does
computer, or both not need to re-
• …whether or not we construct it…
observe or acknowledge
them

Why is it better to have the (data-centric)
middleware manage the state?

• Reconstructing the state of an object is hard
– Must infer based on all previous messages
– Maintaining all these messages is expensive
– Each app makes these inferences  duplicate effort
• Reconstructing state is not robust
– Many copies of state  may be different  bugs
vs. Uniform operations on state  fewer bugs
– Any missing change compromises integrity
• State awareness results in better performance
– Middleware can be smart about what to send and when
• Data-type awareness simplifies programming
– Middleware supports direct definition and instantiation of the data-
model

DDS Communication Model


Demo: Publish-Subscribe


DDS Real-Time Quality of Service (QoS) Attributes
QoS Policy QoS Policy
DURABILITY USER DATA

User QoS
HISTORY TOPIC DATA
Volatility

READER DATA LIFECYCLE GROUP DATA

WRITER DATA LIFECYCLE PARTITION

Presentation
LIFESPAN PRESENTATION
Infrastructure

ENTITY FACTORY DESTINATION ORDER

RESOURCE LIMITS OWNERSHIP

Redundancy
RELIABILITY OWNERSHIP STRENGTH
Delivery

TIME BASED FILTER LIVELINESS

Transport
DEADLINE LATENCY BUDGET

CONTENT FILTERS TRANSPORT PRIORITY

Communication Patterns
Beyond DDS


Why Patterns

• From app programming POV
– Patterns solve specific problems nicely
• From platform or systems POV
– Handling each pattern in independent ad hoc ways
results in brittle, poorly performing systems
• Solution: Build platform in terms of
fundamental patterns
– Derive high-level patterns from basic patterns;
expose those to apps

© 2012 RTI 22

Communication Pattern
Request/Reply

Introduction to Request/Reply
• Publish/Subscribe (Pub/Sub)
– Fundamental pattern in RTI Connext
– Pub/Sub shines where there is
• Push model
• Unidirectional stream of data
• One-to-many (multicast)
• Request/Reply
– Client-server model built on top of pub/sub
– Request/Reply shines where there is
• Pull model
• Bidirectional communication
• A need for remote code execution
– E.g., Get/set configuration, send commands


Request-Reply

Request
Request Topic
Requestor (Foo) Replier

Reply
Topic Reply
(Bar)

COMPANY CONFIDENTIAL

Correlation
Correlation
Message ID

1 3 2 1

Requests Replier

1

1 2 3
Requestor

Replies
Correlation ID

How Does Correlation Work
Requester App Replier App

sample_id1

Data VGUID1 VSN1 Foo Data
Foo Write Writer Reader
(VGUID1)

Part of Foo
Processing
SampleInfo VSN1 VGUID1
Bar

Read
Data Data
Bar
Reader Writer
VGUID2 VSN2 VGUID1 VSN1 Bar
(VGUID2)
sample_id-id2 related_sample_id1
Index Condition
“@related_sample_id.VSN == VSN1”

Content-based Filter
“@related_sample_id.VGUID == VGUID1”
Filtering Implementation details

10/11/2012 © 2012 RTI RTI • COMPANY CONFIDENTIAL
© 2012 • COMPANY CONFIDENTIAL 27 27

Single-Request Multiple-Reply

Request
Requestor Replier

1 2 3

Replies
Sequence ID

Multiple Repliers
Reply
Replier A

Requester Replier B

Request

Replier C


Replier “Hello, World!” (Server-side)
using namespace connext;

DDS::DomainParticipantFactory factory = DDS::DomainParticipantFactory::get_instance();

DDS::DomainParticipant * participant = factory->create_participant(domain_id,
DDS::PARTICIPANT_QOS_DEFAULT,
NULL, DDS::STATUS_MASK_NONE);

std::auto_otr<Replier<Foo, Bar> > replier (new Replier<Foo, Bar>(participant, "TestService"));

Sample<Foo> request;
// Receive one request
bool received = replier->receive_request(request, MAX_WAIT);

if (!received) {
std::cout << "Requests not received" << std::endl;
return false;
}

// A WriteSample automatically initializes and finalizes
// the data using the TypeSupport (in this case BarTypeSupport)
WriteSample<Bar> reply;
if (request.info().valid_data) {
sprintf(reply.data().message, "Reply for %s", request.data().message);
// Send a reply for that request
replier->send_reply(reply, request.identity());
// Note: a replier can send more than one reply for the same request
}


Requester “Hello, World!” (Client-side)
using namespace connext;

// Create a DomainParticipant
DDS::DomainParticipant * participant = ...;

// Create a Requester
std::auto_ptr<Requester<Foo, Bar> > requester
(new Requester<Foo, Bar>(participant, "TestService"));

// Send request
WriteSample<Foo> request;
strcpy(request.data().message, "A Request");
requester->send_request(request);

if (requester->wait_for_replies(2, MAX_WAIT, request.identity())) {
LoanedSamples<Bar> replies = requester->take_replies(request.identity());

// If there are no replies, the container is empty
for(LoanedSamples<Bar>::const_iterator it = replies.begin();
it != replies.end();
++it) {
if (it->info().valid_data) {
std::cout << "Received reply: " << it->data().message << std::endl;
}
}
}

Implementing Request/Reply Pattern
Steps With Vanilla DDS API With BIGPINE API

Define request and reply types Manual Manual
Create Participant Manual Manual
Pollute application data model with a Manual Not necessary
correlation-id
Create publisher/subscriber Manual Automatic
Create datareader/datawriter Manual Automatic
Register request/reply types Manual Automatic
Create request/reply topics Manual Automatic
Setup CFT to filter unwanted replies Manual Automatic
Correlation of request/reply samples Manual Semi-Automatic
Memory management of samples Manual Automatic
Lifecycle management of entities Manual Automatic
Language friendly (idiomatic API) Partially Yes
Scalable Depends on impl Yes

Salient Features of Connext Request/Reply

• New API, built on top of DDS
• Data-centric
• Scalable
• Simple to use, abstracts many DDS details
• Idiomatic, language-friendly API (C++, Java, and
C#)
• It’s not RPC, though can be the foundation for an
RPC solution


Scalable Request-Reply
Requester 1

Reply 1

Replier Requester 2

Replies Reply 2

…
Thousands of clients! Requester N

Reply N

Combining Patterns
Subscriber
Wire Tap

Request
Request Topic
Requestor Replier

Reply
Topic Reply

Subscriber

Communication Pattern
Guaranteed Delivery

The Guaranteed Delivery Problem
• Delivery of samples from a Producer to the
Intended Consumers such that the delivery is
robust to multiple kind of failures
• Guaranteed delivery does not imply complete
certainty  Continuum of Delivery Guarantees

© 2012 RTI • COMPANY CONFIDENTIAL

The Guaranteed Delivery Scenario
• Producer ‘P’ publish a Sample ‘S’ that should
be received by Consumer ‘C’
Consumer process failure
Producer process failure (the process crashes after the sample is
(the producer process crashes after the
received by the middleware and before it is
sample has been written)
processed by the application)
Sample ‘S’
Producer ‘P’ Consumer ‘C’
(DataWriter) (DataReader)
Network and transport failures
(packets are lost due to temporary
network failures or Late joiner consumer
transport/middleware buffer overflows) (the consumer is not running when
the sample is written)


Guaranteed Delivery
App-level ack

Publisher Subscriber

Message Message

Durable
Subscriber

Message

Message

Disk © 2011 Real-Time Innovations, Inc.

DDS Durability QoS Primer
DDS Durability controls data availability with respect to
late joining datareaders
• VOLATILE
– No need to keep data samples for late joiners
• TRANSIENT_LOCAL
– Data instances available for late joiners
– Data does not survive beyond datawriter
• TRANSIENT
– Data is not tied to the lifecycle of the datawriter.
– A separate service (persistence service) keep data in memory
• PERSISTENT
– Data is kept on permanent storage, so that they can outlive a
system session

Beyond DDS: Required Subscriptions

• DataWriter can be configured with:
– a list of required subscriptions
• Required subscriptions are named subscriptions identified by a
role name
• DataWriter must store a sample until both:
– Acknowledged by all active reliable readers
– Acknowledged by all required subscriptions
• DataReader identifies itself:
– As serving a required subscription
– Uniquely via a virtual GUID

Beyond DDS: Durable Subscriptions
• Concept
– It is a required subscription with durability >= TRANSIENT
– Features that allows to keep data until received by durable
subscriptions for which readers may or may not exist at
any given time
– Data is not lost even if the DataWriter crashes
• Benefits and Use Cases
• In combination with other features durable subscriptions
provides guaranteed messaging
• Requires persistence service
– To persist data
– To persist existence of durable subscriptions and their stats

Beyond DDS: Collaborative DataWriters
• Concept:
– Multiple DataWriters can relay samples from a common
stream (data source). The DataReaders reconstruct the
original stream and deliver to the application the complete
set of samples

• Benefits & Use cases:
– High availability and fault tolerance
• Multiple DataWriters publishing the same samples
– Load balancing and data partition
• Multiple DataWriters publishing different sets of samples from the
same source
– Scalability
• Partition across different machines and cores

Delivery Models: Best-Effort to Guaranteed!
Use Case Features in RTI Connext Messaging 5.0.0
Periodic data from live producers Best effort reliability
Event data from live producers Reliable reliability
Last value cache from live producers (entire Transient local durability +
state of the datawriter cache) Reliable reliability
Durable last value cache (datawriter may Persistent durability +
come and go) Reliable reliability +
Persistence Service
Guaranteed messaging from live producers Transient local durability +
Reliable reliability +
Required subscriptions +
Application level ACK
Durable Guaranteed messaging Persistent durability +
Reliable reliability +
Collaborative DataWriters +
Persistence Service +
Durable subscriptions +
Application level ACK


References

• Real-Time Innovations, Inc.
– www.rti.com
• RTI Connext DDS
– www.rti.com/products/dds
• RTI Connext Request-Reply API
– Just Google!
• OMG DDS Specifications
– www.omg.org/technology/documents/dds_spec_catalog.htm


Thank you


Communication Patterns Using Data-Centric Publish/Subscribe

More Related Content

What's hot (20)

Similar to Communication Patterns Using Data-Centric Publish/Subscribe (20)

More from Real-Time Innovations (RTI) (20)

Recently uploaded (20)

Communication Patterns Using Data-Centric Publish/Subscribe