The Java Microservice Library

Java microservice development for high-speed services
Java Microservice lib
QBit
high-speed, in-memory,
services for REST, WebSocket
and JSON

What is QBit?
❖ Fast queue based service engine!
❖ Services sit behind high-speed queues!
❖ Uses micro-batching and mechanical sympathy to create high speed REST
and WebSocket microservice!
❖ Programming model looks like Spring MVC, but enables high-speed async
model like Akka, Vertx, Reactor, but easier to program!
❖ Uses Consul or Etcd for clustering, health, cloud conﬁg, elasticity !
❖ Has event bus that can use Kafka, 0Q, etc.!
❖ Uses JSON for wire protocol, works easily with any language over HTTP or
WebSocket

Why QBit?
❖ Just a lib, does not take over the world!
❖ Works with Guice or Spring!
❖ Just POJOs written like Java developers write POJOs no steep
learning curve!
❖ FAST!!
❖ Easier than reactive, worker pools, sharded services and queued
services!
❖ Uses Consul for service discovery, WebSocket for high-speed
messaging and REST

Architects / Developers of QBit
❖ Wrote high throughput global services for large high-trafﬁc clients!
❖ F5 crushing load!!
❖ Implemented systems that took other systems 10x to 100x as many servers and 20x
#of developers that could handle only 1/10th of the trafﬁc !
❖ Wrote JSON parser that is 4x faster than mainstream JSON parsers!
❖ Been there, done that for 20+ years. 100+ years building high-speed, reliable systems
combined!
❖ Consulting with airlines, plane manufacturers, banks, commodity trading, electronic
manufacturing, and other trading systems!
❖ Wrote services that powers some of the busiest sites on the Internet!
❖ Wrote fastest disk batcher for JVM, high-speed, distributed k/v store, in-memory db

QBit Speed - FAST!
❖ 200 M in-proc messages a second!
❖ 10M to 100M in-proc events per second!
❖ 700K to 1,000,000 RPC calls!
❖ Equates to 1.4M to 2M messages TPS!
❖ Using JSON! One Service Thread! One WebSocket!!
❖ Not uncommon to use a fraction of CPU for 2x to 10x to
100x the same performance as a traditional microservice

It all starts with a fast queue

Speed First

Service Discovery / Health
❖ Integration with Consul for
health and event bus wiring
of peers!
❖ Cloud conﬁg!
❖ Peer discovery!
❖ Health Checks!
❖ Serf/Dead man’s switch,
gossip, custom

Ease of use and simplicity too

What is QBit again?
❖ Java microservice development for high-speed services!
❖ JSON!
❖ REST!
❖ WebSocket!
❖ Services look like Spring MVC REST / JAX-RS !
❖ look like Spring Boot, Drop-wizard but work like Spring reactor, Akka, Go,
ErLang, etc. !
❖ Embraces microservices!
❖ Clients can be Web, Mobile, any language that has JSON lib, HTTP lib can use
QBit services

QBit Services
❖ WebSocket RPC, HTTP REST, and in-proc messaging!
❖ In-proc, sits behind an actor style queue, gets notiﬁed of
queue empty, queue limit, queue init, queue start batch,
etc.!
❖ Allows reduction of sending IO or calls to CPU
intensive services in batches based on queue events!
❖ Batching calls increases system throughput through
the system 10x, 100x, 1000x.

Key Concepts
❖ `!
❖ Queue !
❖ A thread managing a queue. !
❖ supports batching. !
❖ queue callbacks for empty, reachedLimit,
startedBatch, idle, etc.!
❖ ServiceQueue!
❖ POJO (plain old Java object) behind a
queue!
❖ receive method calls and events !
❖ thread safe!
❖ queue callback methods!
❖ Java idioms looks like Spring MVC REST,
JAX-RS!
❖ ServiceBundle !
❖ Many POJOs behind one response queue
and many receive queues.
❖ ServiceServer !
❖ Exposes services to REST/JSON and
WebSocket/JSON communication!
❖ EventBus !
❖ send a lot of messages to services !
❖ may be loosely coupled,!
❖ integration with others event buses
(Kafka, STOMP, etc.)!
❖ Metrics/Stats: track in-ﬂow, and outﬂow
of messages, other metrics for
performance, high-speed time-series
database with REST interface!
❖ Meta-Data-Service: Publish track all
REST JSON schemas for easy integration
and use by third parties

Principles of QBit
❖ Web ﬁrst (JSON, HTTP, WebSocket)!
❖ Integration via high-speed event bus (Kafka, STOMP,
0MQ, ActiveMQ, etc.)!
❖ Embrace Mechanical Sympathy and Queue Theory to
optimize thread hand-off, IO, and core cross talk!
❖ Less is more (be the 95% solution leave the long tail
alone)!
❖ Fun and easy to use!

QBit Service Example
Domain Object
Service
WebSocket HTTP support

QBit Example
CURL-ABLE
JSON Output

Service REST support
❖ Supports Request params, URI params, JSON POST,
HTTP GET, etc.!
❖ Customize URIs using annotation!
❖ Reasonable Defaults (annotations not needed)!
❖ Methods available via annotations

QBit versus….

QBit versus Spring XXX
❖ Spring what? DI? MVC? Boot?!
❖ QBit Works easily with DI and Spring Boot!
❖ Competes against Spring MVC but has better thread
model for high-speed services, but not all the “features”.!
❖ Spring MVC does a lot. QBit only does Microservices
only! So only JSON, REST, and WebSocket!
❖ QBit Enable writing in-memory, high-speed services that
can handle 200K TPS to 2M TPS per second single node

Akka vs. QBit
❖ Akka uses actor model!
❖ QBit uses queuing actor-like model + active objects model!
❖ Akka focuses on Queue handlers!
❖ QBit focuses on Services that look / act like Java services in
Spring REST, JAX-RS, Java EE, and other frameworks!
❖ QBit is a Java centric model, Akka programming model is similar
but foreign to most Java developers!
❖ QBit center of gravity is Microservices = REST, JSON, WebSocket

Akka and QBit similarities/differences
❖ Akka Typed Actor a lot like QBit Service!
❖ QBit Services can be exposed via REST/JSON or
WebSocket/JSON!
❖ Both use queue events (empty, init, limit, etc.) to optimize
output to IO!
❖ Both are fast!
❖ QBit is library and you can use / compose queues,
services as you see ﬁt (you could use QBit in a Spring app)

Node.js, Vertx
❖ Node.js and Vertx both use a Reactor style design pattern based on event bus.
(QBit is reactive but using active objects not reactor.)!
❖ At ﬁrst glance looks a lot like Akka or QBit!
❖ Less granularity on bus, QBit bus speed is a lot faster!
❖ Less control of when queue is idle, empty, etc. QBit has Akka like tuning which is
critical to optimize around back pressure.!
❖ Vertx/Node.js are harder to optimize. Harder to compose other handlers. !
❖ Great for IO and Vertx can be combined with QBit to provide missing pieces for
Speed!
❖ Not bad. Not even close to what QBit can do!
❖ QBit works well with Vertx

How does it do?
❖ Faster!
❖ Easier to use!
❖ Focuses just on Micorservices!
❖ Tolerant reader, promiscuous writers!
❖ Uses RAFT and Consul for service discovery

Event Bus Overview

Event Bus
❖ QBit has a high-speed event bus modeled loosely on Vertx style
event bus!
❖ Very fast. !
❖ In-proc but can be integrated with others!
❖ Service methods can be invoked by calls (remote and local behind a
queue) !
❖ Service methods can be invoked by listening to a event bus
channels!
❖ You can subscribe to channels or you can consume channels or both

Composable Event Bus
❖ EventBus can be composed!
❖ You can use the system event bus for in-proc service to service
messaging!
❖ You can remote replicate events to other servers!
❖ You can tap into an event bus, and/or post events to a service
directly for integration with 3rd party systems like Kafka, 0MQ,
etc.!
❖ You can use Consul integration to ﬁnd service peers and
compose event bus that allows peers to communicate

Event Bus is an Integration point
❖ Event bus is an integration point!
❖ Can be easily integrated with Camel, ActiveMQ, JMS,
Kafka, etc.!
❖ Events come in on same queue as method call!
❖ Event Bus can do remote broadcast and can discover
peers via Consul or Etcd

QBit Event Bus

Private event bus versus System event bus
❖ Private event bus!
❖ https://github.com/advantageous/qbit/wiki/%5BRough-Cut%5D-Working-
with-private-event-bus-for-inproc-microservices!
❖ Only your module uses it!
❖ System event bus!
❖ https://github.com/advantageous/qbit/wiki/%5BRough-Cut%5D-Working-
with-event-bus-for-QBit-the-microservice-engine!
❖ Used by default if you don’t deﬁne your own!
❖ Event buses are composable!
❖ events are intrinsic to Services and event buses can be wired to services and
other event buses

Simple Example
Java 8 Lambda
Annotation Based
Send
Consumer
Subscriber

More involved Example
❖ Advantage of using the event bus with QBit services is the events come into
same queue that handles method calls so events method calls are thread
safe!
❖ Event Bus is very fast. Expect speeds up to 10M to 100M messages a second.!
❖ New example: set of services that handles when a new employees is hired!
❖ Add the employee to payroll system, !
❖ Enroll employee into beneﬁts system!
❖ Invite employee to our community outreach program!
❖ Four services but the ﬁrst service will not know about other services

Channels and Employee

EmployeeHiringService

Service that listen

Getting a client proxy to a service

Services are just POJOs

Wire in POJOs services

Create a client proxy from service

Add a new Service Listener?

New Service

Event Manager
❖ You can use a private event bus instead of a system
event bus!
❖ You can use strongly typed proxy to the event bus to
abstract channel names and provide a strongly typed
bus

Strongly Type Interface

Using strongly typed interface to event bus

Using Strongly typed Event Proxy

Using Strongly Typed Event Proxy-WIRING

Working with callbacks
https://github.com/advantageous/qbit/wiki/%5BRough-Cut%5D-QBit-Microservice-Lib-
Working-With-CallBacks

Callbacks

Recommendation service callback

ServiceQueue Proxy for service

Recommendation service

Handling a callback

Lambdas are great until you can’t read your code anymore
Lambda calling Lambda, calling
Lambda calling Lambda.!
!
Stop using Lambda if your code
starts to look like Perl threw up on
Lisp.!
!
One, to three levels ok.!
!
Five or six, can you break this out
into a method call or two.

Don’t forget to flush
Auto ﬂushing is available
too!

But gee what if we have to load the user data?
❖ The ﬁrst rule is don’t block!
❖ Services can call other services!
❖ When a service might do IO then it should take a
callback instead of have a return (generally speaking)

Use Callback instead of blocking

If user is loaded, just call callback

If you need to load the user, do it async…

Doing something later…, but not too much later

Async procrastination a good thing…

When to handle callbacks?
When your queue is
empty, !
when you hit your
queue limit,!
and !
perhaps at the start !
of a new message
batch?!
!
Handle callbacks
before handling new
method calls so
clients don’t wait
around with open
connections

Workers for IO, Sharded Services for CPU
❖ QBit allows worker pools for IO bound workers!
❖ QBit allows sharded for CPU bound workers that work
on in-memory data!
❖ Or just regular workers!
❖ Queue system can detect when CPU bound services are
busy and auto-increase queue batch size

The Java Microservice Library

Recommended

More Related Content

What's hot (20)

Similar to The Java Microservice Library (20)

More from Rick Hightower (17)

Recently uploaded (20)

The Java Microservice Library