0% found this document useful (0 votes)
11 views12 pages

optiq_technical_note_mdg_snapshot_20170131

The Euronext Optiq Snapshot Service provides a recovery functionality for clients to synchronize with real-time market data after packet loss or late starts. Snapshots are sent periodically and contain an image of market data messages, allowing clients to discard outdated messages and continue processing in real-time. The service is designed to support various connection types and is essential for maintaining data integrity in trading operations.

Uploaded by

Arnab De
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
11 views12 pages

optiq_technical_note_mdg_snapshot_20170131

The Euronext Optiq Snapshot Service provides a recovery functionality for clients to synchronize with real-time market data after packet loss or late starts. Snapshots are sent periodically and contain an image of market data messages, allowing clients to discard outdated messages and continue processing in real-time. The service is designed to support various connection types and is essential for maintaining data integrity in trading operations.

Uploaded by

Arnab De
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

EURONEXT OPTIQ TECHNICAL NOTES

INTRODUCTION TO THE SNAPSHOT SERVICE

Scope and audience: This technical note intends to provide general


information about the Snapshot Service for Optiq MDG. Describing
the Snapshot kinematics, this document aims to familiarize
developers how to implement this recovery functionality in their
feed handler to recover from packet loss, or in case of an intraday/
late start.

SUMMARY
The Optiq MDG Snapshot service is a recovery functionality provided by the exchange in order to allow
customers to recover from packet loss or in case of an intraday/late start. Providing an image of the
market data and the last Market Data Sequence Number used to build the Snapshot image allows
customers to synchronize with the real time data and continue/start processing messages in real time.

Snapshot messages will be provided on dedicated Snapshot channels, alongside the real-time channels
and the messages will be compressed using LZ4.

A snapshot contains an image of all the market data message types for all instruments that are part
of the real-time channel and the messages are sent with the Rebroadcast indicator field set to 1.

The snapshots are sent out periodically and the minimum time interval between 2 snapshots is
2 seconds.
1. THE SNAPSHOT SEQUENCE EXPLAINED
A snapshot sequence contains multiple channels that when combined provide the necessary
information for our customers to align with the real time MDG feed. The following snapshot channels
are part of a sequence:

ƒƒ Full Order Book Channel

ƒƒ BBO Channel

ƒƒ Reference Data and Full Trade Information Channel

ƒƒ Reference Data and Index Package Channels

Both “Start Of Snapshot” and “End Of Snapshot” messages contain the last “Market Data Sequence
Number” of the last real-time message taken into account by the snapshot. Both these messages are
not compressed.

In order to use the Snapshot, the first step is to queue messages on the real-time channel from the
moment you start receiving the snapshot data.

Clients can use the Start Of Snapshot or End Of Snapshot messages to retrieve the last “Market Data
Sequence Number” used to build the snapshot that is being sent out and can use it to synchronize with
the real time channel by discarding from the real-time queued data those messages with a “Market
Data Sequence Number” equal or smaller than the last “Market Data Sequence Number” received in
the Start or End Of Snapshot messages and continue in real time channel processing the next Market
Data Sequence Numbers.

It is important to note that since the Market Data Sequence Number does not necessarily increment
by 1, within a real time channel, the sequence number in the start or end snapshot messages
might belong to another channel, and was in fact not actually lost. As a reminder, the only criteria to
determine packet loss is by using the Packet Sequence Number which increments monotonically with 1
and is unique per channel ID.
2. SNAPSHOT SEQUENCE FOR CASH:

Full Order Book Full Order Book


Trade and
Market Update Order Update
Referential channel
channel channel

Start Of Snapshot
Market Status Change

Market Update messages with:


1) 1 - Best Bid and 2 - Best Offer
2) 3 - New Bid and 5 - Updated Bid and
58 - New Bid With Liquidity Provider and 60 - Updated Bid With Liquidity Provider
3) 4 - New Offer and 6 - Updated Offer and
59 - New Offer With Liquidity Provider and 61 - Updated Offer With Liquidity Provider
4) 16 - New Bid RLP and 18 - Updated Bid RLP
5) 17 - New Offer RLP and 19 - Updated Offer RLP
6) 20 - New Bid SI and 22 - Updated Bid SI
7) 21 - New Offer SI and 23 - Updated Offer SI
8) 63 - Low Static Collar and 64 - High Static Collar
9) 15 - Low Dynamic Collar and 14 - High Dynamic Collar

End Of Snapshot

Start Of Snapshot
Market Status Change

2 Seconds Order Update messages for each active orders with:


5 - Retransmission of all orders for the given instrument

Market Update messages with:


1) 1 - Best Bid and 2 - Best Offer
2) 63 - Low Static Collar and 64 - High Static Collar
3) 15 - Low Dynamic Collar and 14 - High Dynamic Collar
End Of Snapshot

Start Of Snapshot
Timetable
Full Trade Information messages
Price Update
Real Time Index (for BdL)
Index Summary (for BdL)
Statistics
Exchange Announcement
End Of Snapshot
Cash is composed of: Bourse de Luxembourg (BdL), Equities, Funds and Fixed Income.

1 Reference Data represents: all instruments characteristics, scheduled phases and market administration
messages.

2 This message will not provide: New Bid (3)/New Offer (4), Updated Bid (5) /Updated Offer (6), New Bid With
Liquidity Provider (58)/New Offer With Liquidity Provider (59), Updated Bid With Liquidity Provider(60)/ Updated
Offer With Liquidity Provider (61), New Bid RLP (Retail Liquidity Provider) (16)/ New Offer RLP (Retail Liquidity
Provider) (17), Updated Bid RLP Retail Liquidity Provider) (18)/ Updated Offer RLP (Retail Liquidity Provider) (19),
New Bid SI (20)/ New Offer SI (21) and Updated Bid SI (22)/ Updated Offer SI (23).

3 This message will provide only: New Bid RLP (Retail Liquidity Provider) (16)/ New Offer RLP (Retail Liquidity
Provider) (17), Updated Bid RLP Retail Liquidity Provider) (18)/ Updated Offer RLP (Retail Liquidity Provider) (19)
or Clear-Book (254).

4 This message will provide only: New Bid SI (20)/ New Offer SI (21), Updated Bid SI (22)/ Updated Offer SI (23), SI
Trade (47) or Clear-Book (254).
SNAPSHOT SEQUENCE FOR DERIVATIVES:

Full Order Book


Trade and
Market Update BBO
Referential channel
channel

Start Of Snapshot
Market Status Change

Market Update messages with:


1) 1 - Best Bid or 8 - Implied Bid and 2 - Best Offer or 9 - Implied Offer
2) 3 - New Bid and 5 - Updated Bid
3) 4 - New Offer and 6 - Updated Offer

End Of Snapshot

Start Of Snapshot
Market Status Change
2 Seconds

Market Update messages with:


1 - Best Bid or 8 - Implied Bid and 2 - Best Offer or 9 - Implied Offer

End Of Snapshot

Start Of Snapshot
Intraday Outright or Strategy Standing data
Full Trade Information messages
Price Update
Statistics
Exchange Announcement
End Of Snapshot
Derivatives is composed of: Options, Futures, Warrants & Certificates.

1 Reference Data represents: all instruments characteristics, scheduled phases and market administration
messages.

2 This message will not provide: New Bid (3)/New Offer (4), Updated Bid (5) /Updated Offer (6), New Bid With
Liquidity Provider (58)/New Offer With Liquidity Provider (59), Updated Bid With Liquidity Provider(60)/ Updated
Offer With Liquidity Provider (61), New Bid RLP (Retail Liquidity Provider) (16)/ New Offer RLP (Retail Liquidity
Provider) (17), Updated Bid RLP Retail Liquidity Provider) (18)/ Updated Offer RLP (Retail Liquidity Provider)
(19), New Bid SI (20)/ New Offer SI (21) and Updated Bid SI (22)/ Updated Offer SI (23).
3. SNAPSHOT SEQUENCE FOR INDICES:

Indices

Start Of Snapshot
Real Time Index
2 Seconds Index Summary
Statistics
End Of Snapshot

Snapshot for Indices


4. SEQUENCE NUMBERS
The MDG feed contains two sequence numbers:

ƒƒ The Packet Sequence Number (PSN)

The Packet Sequence Number (PSN) is part of the packet header and should be used for UDP gap
detection and packet ordering. Each channel has its own PSN sequence.

ƒƒ The Market Data Sequence Number

Aggregators are MDG internal components that are dealing with a set of channels. The Market Data
Sequence Numbers are managed at the aggregator level. Each one of them has its own sequence,
starting from 0 and incrementing by step of 1 along the day. Since clients may listen to only a subset
of the channels managed by one aggregator, they won’t see all the Market Data Sequence Numbers
in the messages they get from the channels they listen to. Therefore on one channel the Market
Data Sequence Numbers will increment all along the day but not necessarily by step of 1.

Packet Sequence Numbers are used for detecting packet loss. The Snapshot Channels have their own
PSNs that are not the same as the PSNs on the Real Time Channels.

The same applies for the Market data Sequence Numbers.

5. GAP DETECTION AND LINE ARBITRATION


The Packet Sequence Number (please see Market Data Packet Header) should be used to detect gaps
in the transmission of packets.

Line A Line B

PSN PSN

101 101

102 102

103 103

104 104
Dropped packet can
Gap detected 105 be recovered from
Line B channel
106 106

107 107

Using this method, a lost packet can be recovered from the second line. In case of packet loss on both
lines, then the snapshot mechanism should be used.

UDP packets can potentially arrive unordered and potentially sent twice. As such, systems should be
able to reorder the packets and detect duplicate packets.

Please be noted that the real-time is carried over line A and B.

The snapshot feed is also carried over line A and B, therefore line arbitrage is also possible on the
snapshot multicast feeds.
6. SNAPSHOT APPLIED
Customers that connect intraday (late start) have only one viable way to use the snapshot:

“Hop On” (connect) to the Snapshot Channel corresponding with the Real Time Channel the Packet loss
occurred, while starting to queue the Real Time messages.

The customer clears the order books for all instruments on that channel.

The Start Of Snapshot (2101) message indicates the start of a Snapshot Sequence.

The image provided in this sequence contains everything to rebuild the order book up to the last
Market Data Sequence Number that is in the end snapshot messages.

End Of Snapshot (2102) indicates the end of a Snapshot Sequence.

Clients then discard the real-time queued messages with a “Market Data Sequence Number” less
than or equal to the last “Market Data Sequence Number” of the snapshot.

The customer is in sync with MDG

“Hop Off” (disconnect) from the Snapshot Channel.

Customers that experience packet loss (however unlikely (on both A and B feed) can use the same
preferred method:

“Hop On” (connect) to the Snapshot Channel corresponding with the Real Time Channel the Packet loss
occurred, while starting to queue the Real Time messages.

The customer clears the order books for all instruments on that channel.

The Start Of Snapshot (2101) message indicates the start of a Snapshot Sequence.

The image provided in this sequence contains everything to rebuild the order book up to the last
Market Data Sequence Number that is in the end snapshot messages.

End Of Snapshot (2102) indicates the end of a Snapshot Sequence.

Clients then discard the real-time queued messages with a “Market Data Sequence Number” less
than or equal to the last “Market Data Sequence Number” of the snapshot.

The customer is in sync with MDG

“Hop Off” (disconnect) from the Snapshot Channel.


OR:

Use the snapshot to correct those instruments impacted - message type by message type. This method
is not supported by Euronext and there is no documentation available on this topic. The reason behind
this is that the snapshot provides an image and does not provide all messages sent within a certain
frame. Please note, this follows not the same logic as the XDP’s Refresh Service. This is why members
are discouraged to build a solution that only correct the impacted parts.

Regardless if one has a 10 Gb, 1Gb or 100Mb connection clients use the same dedicated Snapshot data.

Summarized:
7. Q&A ON THE SNAPSHOT SERVICE
Q: What is the Snapshot Service?
Snapshot is a service provided by the exchange in order to provide at a given time of the day an
image of the market data. The Snapshot Service allows to resynchronize with the real-time data
and provides key market data depending on the nature and content of the corresponding real-
time feed.

Q: When do I use it?


In case of packet loss in real-time feed that can’t be resolved by line arbitration.

Q: Is it similar to the Refresh Service on XDP?


No, on XDP a RFS is initiated by the customer, with MDG dedicated Snapshot Channels provide
Snapshot Sequences continuously in cycles. XDP required setting up a TCP/IP connection to
request a RFS. MDG only requires clients to “listen” to the snapshot multicast.

Q: What are the bandwidth needs for subscribing to the Snapshot Channel?
Everyone connects to the same snapshot channel (for a subset of realtimes channels) regardless
their connection bandwidth, the service is developed in such a way all connection types are
supported and use the same data broadcast.

Q: Are Snapshot broadcasts compressed?


Yes they are, but the Begin and End Of Snapshot messages only when they can be.

Q: Compared to RFS on XDP what is the impact on performance.


The expectation is a significant improvement.

Q: Will every snapshot contain all the data since start of the day ?
No. The snapshot will only provide an image of all the messages needed to build the book as it is
at the time of the snapshot sending.

Q: Why do I use the Packet Sequence Numbers for detecting potential data gaps? Can’t I just look at
the Market Data Sequence Numbers?
No, you can’t as the logic behind the MDSNs is such that missing MDSN are not per definition
identifying a gap. PSN gaps, however is an indicator of potential missing data and an action is
required.

Q: Does a Snapshot image contain all needed referential data to rebuild the order book?
Snapshot provides only Intraday standing data, for the full list of instruments (full standing data)
you should use the File Service.

Q: Does a member need a single snapshot channel for the entire Cash market for example or one
snapshot channel per Cash Asset Class?
The repartition is as following: Per Asset Class – Per Sub Asset – Per Partition – Per channel
Type.

Q: When will the Snapshot Service be available?


Snapshot is not available for testing yet. It should be in Bundle 3 (early February 17).
Customer focus CONTACT DETAILS
For additional information, please
The migration onto the new Optiq platform has been designed with reach out to your Optiq Programme
customers in mind. Technical relationship managers from the Euronext client readiness team
Client Readiness team will be available to meet with clients to review the
technical specifications, facilitate impact assessment, hold development Veronica Taylor
workshops, and provide regular updates on the migration timeline and Tel: +44 (0) 20 7076 0931
status. [email protected]

Jack Cohecha
Tel: +33 (1) 70 48 25 40
[email protected]

Optiq Support Desk


Tel: +33 (1) 70 48 25 55
[email protected]

Disclaimers
This document contains information which is confidential and of value to Euronext. The information and materials contained in this document are provided ‘as is’ and Euronext
does not warrant the accuracy, adequacy or completeness and expressly disclaims liability for any errors or omissions or changes enabled to be made for any reason included
correction, update and upgrade purpose. This document contains links to certain Internet Websites developed, sponsored or maintained by third parties unaffiliated with
Euronext. The content you view therein is not provided or controlled by Euronext. Euronext is not responsible for that content, nor has it developed, checked for accuracy or
otherwise reviewed the content or privacy policy of any such third party Website. This document is not intended to impose any legal obligation on Euronext. This document
and any contents thereof, as well as any prior or subsequent information exchanged with Euronext in relation to the subject matter of this document, are confidential and are
for the sole attention of the intended recipient. Except as described below, all proprietary rights and interest in or connected with this publication shall vest in Euronext. No
part of it may be redistributed or reproduced without the prior written permission of Euronext. Portions of this publication may contain materials or information copyrighted,
trademarked or otherwise owned by a third party. No permission to use these third party materials should be inferred from this publication. Implementation of the project may
be subject to regulatory approval. Euronext refers to Euronext N.V. and its affiliates. Information regarding trademarks and intellectual property rights of Euronext is located at
https://www.euronext.com/terms-use.

© 2017 Euronext N.V. - All rights reserved.

01/17

You might also like