Realtime Technical Specification CM Segment
Realtime Technical Specification CM Segment
Version: 1.25
Date: 14 June 2024
COPYRIGHT NOTICE
2
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Revision History
Version 1.25 Adding 6 new fields in BOD - Master Information 14 June 2024
3
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Table of Contents
1 Introduction .............................................................................................. 6
2 Packet Format ........................................................................................... 8
2.1 Data Types ........................................................................................... 9
2.2 Diagrammatic Representation of Packet Format ........................................ 10
3 Session Messages ..................................................................................... 11
3.1 Heartbeat Message (Sent by server) ....................................................... 11
4 Sequenced Data Message (Sent by server) ................................................... 12
4.1 BOD - Master Information ..................................................................... 12
4.2 Online - Market Status Message ............................................................. 15
4.3 Online – Touchline Market Update........................................................... 16
4.4 Online – 5 Depth Market Update ............................................................. 18
4.5 Online – 20 Depth Market update ........................................................... 21
4.6 Online - Call Auction Market Security Update ............................................ 25
4.6.1 Call Auction Market (Call Auction 1) ..................................................... 25
4.6.2 Reserved Market (Call Auction 2) ......................................................... 25
4.6.3 Call Auction 1 & 2 (Touchline Market Update) ........................................ 25
4.6.4 Call Auction 1 & 2 (5 Depth Market Update) .......................................... 28
4.7 Online - Broadcast Message ................................................................... 32
4.8 EOD – Market Statistics (Bhavcopy) ........................................................ 33
4.9 EOD – Master Addition/Modification/Deletion ............................................ 34
4.10 EOD – Corporate Action Update ............................................................ 35
4.11 BOD & EOD Checksum Information ....................................................... 36
4.12 EOD – End of Feed Information ............................................................ 37
5 Steps for Decompressing the Data Packets ................................................... 38
5.1 LZO Algorithm Details ........................................................................... 38
5.2 Files required for LZO algorithm ............................................................. 38
5.3 Decompression steps ............................................................................ 38
6 Checksum Calculation Algorithm ................................................................. 40
7 Notes ..................................................................................................... 41
7.1 Normal Market Session ......................................................................... 41
7.2 Auction Market Session ......................................................................... 41
7.3 Pre-Open Session................................................................................. 41
7.4 Call Auction Session 1........................................................................... 42
4
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
5
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
1 Introduction
NSE Data & Analytics Ltd. disseminates NSEIL’s real time broadcast data to
various information agencies. It provides 6 different types of data products viz.
The real time data and corporate data is a packet broadcast available for
dissemination through feed, whereas the snapshot data, end of day data and
historical data is available in the form of files. All these data products come
under in Infofeed application.
6
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
This document explains the NSE – Market Feed (CM Level 1, Level 2 & Level
3) products. Through this product on a real-time basis all the NSE’s market
update information is disseminated.
The information agencies connect to the Market Feed Server through Leased
Lines. These leased lines are terminated on Infofeed Router and their data
specific pneumonic calls are forwarded to Infofeed server.
The feed consists of series of sequenced and unsequenced variable length
compressed messages. The compression algorithm used over here is LZO –
Compression.
7
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
2 Packet Format
typedef struct
{
CHAR cCompOrNot;
SHORT nDataSize;
SHORT iNoOfPackets;
}ST_COMP_BATCH_HEADER;
typedef struct
{
SHORT iCode;
SHORT iLen;
LONG lSeqNo;
} ST_INFO_HEADER;
typedef struct
{
.
.
}ST_DATA_INFO;
typedef struct
{
SHORT iCheckSum;
CHAR cEOT;
} ST_INFO_TRAILER;
typedef struct
{
ST_INFO_HEADER stInfoHdr;
ST_DATA_INFO stDataInfo;
ST_INFO_TRAILER stInfoTrailer;
.
}ST_DATA_PACKET;
8
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
All the packets received from server consist of compress batch header.
Compress batch header gives the information about the data packet
compressed or not, number of packets in the following data packet and the
total size of data packet. Client needs to decompress the data packet using
LZO decompression algorithm. After decompression each data packet consists
of ST_INFO_HEADER, which has the iCode field to identify the type of the
packet. Using iCode field, data info packet is mapped to the respective data
packet.
9
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Compressed Header
10
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
3 Session Messages
11
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Sequenced data messages will be sent by server and will contain the actual market
data.
These packets are sent at the beginning of each trading day before market open.
This feed contains the information about the securities valid in the CM Market for
trading.
An International
Securities Identification
ISIN Number CHAR [12] Character
Number (ISIN) uniquely
identifies a security
‘Y’ = Deleted
Is Deleted CHAR [1] Character
‘N’ = Not Deleted
12
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Value can be 0 or 1
0 = T+0 will be the
settlement Period for
Settlement cycle SHORT Numeric that security
1 = T+1 will be the
settlement Period for
that security
Security Description CHAR [30] Character Security Name
13
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
ST_SECURITY_ELIGIBILITY_PER_MARKET
‘1’ = Open
Security Status CHAR [1] Character
‘0’ = Suspended
14
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
This message is sent by the server, whenever the market status changes.
15
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
NSE securities update information for pre-open and normal market is sent through
this Message.
16
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Total Traded
CHAR [12] Character Volume traded today
Quantity (TTQ)
‘S’ = Suspended
Security Status CHAR [1] Character
‘ ‘ = Non-suspended
Open price of the
security for the day.
In pre-open session the
Opening Price CHAR [10] Character indicative open price is
sent if security is
available in pre- open
session.
High price of the
High Price CHAR [10] Character
security for the day
Low price of the
Low Price CHAR [10] Character
security for the day
Close price of the
security.
During the day previous
day’s close price is sent.
Close Price CHAR [10] Character
After market close
current day’s close
price is calculated and
sent through this field
Weighted average price
Average Trade Price CHAR [10] Character of the security
i.e. value / quantity
Security traded value
Total Turnover CHAR [25] Character i.e. Average Trade
Price * TTQ
Online Index CHAR [8] Character NIFTY 50 index value
INFO TRAILER
Refer to section
Checksum SHORT Numeric
checksum calculation
End Of Trailer CHAR [1] ‘\r’ Carriage Return
17
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
NSE securities update information for pre-open and normal market is sent through
this Message.
‘N’ = Normal
‘O’ = Odd Lot
Market Type CHAR [1] Character
‘S’ = Spot
‘A’ = Auction
18
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
19
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
INFO TRAILER
Refer to section
Checksum SHORT Numeric
checksum calculation
20
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
These packets contain the latest order and trade information of securities up to the
order book depth of 20. These packets are sent during normal market hours. These
packets would not be sent during the Pre-Open session.
The Online Index field in this packet indicates the value of the Nifty 50 when the
trade occurs.
‘N’ = Normal
‘O’ = Odd Lot
Market Type CHAR [1] Character
‘S’ = Spot
’A’ = Auction
No. of seconds from
Timestamp CHAR [11] Character 01-01-1970 00:00:00
(DD-MM-YYYY
HH:MM:SS)
21
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
‘S’ = Suspended
Security Status CHAR [1] Character
‘ ‘ = Non-suspended
22
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
23
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
MARKET_DEPTH_BUY_ORDER_INFO
MARKET_DEPTH_SELL_ORDER_INFO
24
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Two new market types, Call Auction and Reserved (for future use) are introduced in
the capital market trading system. These markets securities update information is
sent through these messages.
SME (Small & Medium Enterprise) securities call auction session is conducted in
this market type. For detailed explanation please refer to section Call Auction 1.
IPO, Relisting & illiquid securities call auction session is conducted in this market
type. For detailed explanation please refer to section Call Auction 2.
25
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
‘S’ = Suspended
Security Status CHAR [1] Character
‘ ‘ = Non-suspended
This field contains the
indicative opening
price (IOP) of a
security for order
Opening Price CHAR [10] Character
collection period
session and Final Open
Price of a security in
matching period
26
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
27
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
28
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
29
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
30
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
MARKET_DEPTH_AUCTION_BUY_ORDER_INFO[5]
Field Name Data Type Value Remark
MARKET_DEPTH_AUCTION_SELL_ORDER_INFO[5]
Best 5 buy side’s
outstanding orders price,
Best Buy-Order quantity & BBMM flag
CHAR [10] Character
price information.
Refer to section BBMM Flag
In case of Call Auction 1,
best 4 buy side’s
outstanding orders price,
Best Buy-Order quantity & BBMM flag
CHAR [12] Character
Quantity information is sent.
In the 5th price, quantity and
BBMM flag fields buy side’s
ATO orders information is
sent.
Buy BBMM Flag CHAR [1] Character In case of Call auction 2,
best 5 buy side’s
outstanding orders
information is sent
MARKET_DEPTH_AUCTION_SELL_ORDER_INFO[5]
Field Name Data Type Value Remark
MARKET_DEPTH_AUCTION_SELL_ORDER_INFO[5]
Best 5 sell side’s
outstanding orders price,
Best Sell-Order quantity & BBMM flag
CHAR [10] Character
price information.
Refer to section BBMM Flag
In case of Call Auction 1, the
best 4 sell side’s outstanding
orders price, quantity &
Best Sell-Order BBMM flag information is
CHAR [12] Character
Quantity sent.
In the 5th price, quantity and
BBMM flag fields sell side’s
ATO orders information is
sent.
Sell BBMM Flag CHAR [1] Character
In case of Call auction 2,
best 5 sell side’s outstanding
orders information is sent
31
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
These packets consist of the messages broadcast during the Trading time containing
information such as changes in the price bands of particular script and market-
related information.
32
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
The end of day status of the securities is sent through these messages. After market
close, this information is disseminated to client as the “End of Day” (EOD) feed.
34
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
After market close, this information is disseminated to client as the “End of Day”
(EOD) feed.
35
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
This message gives the information about the number of messages (i.e. count)
sent for each BOD & EOD message. This message will be sent multiple times in
a day. (i.e. After complete dissemination of any BOD/ EOD messages this
message will be sent.)
36
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Refer to section
checksum calculation
Checksum SHORT Numeric Checksum is not
calculated, so it is
sent as 0(Zero)
End Of Trailer CHAR [1] ‘\r’ Carriage Return
This end of the packet indicates that all the parts of EOD feed have been
completed. Only once this message is sent through the Feed. After receiving this
message clients can stop their application i.e. no new update information will be
disseminated from the server.
37
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
The LZO stands for Lempel Ziv Oberhaumer. It is a data compression library which
is suitable for data Decompression in real-time. This means it favors speed over
compression ratio.
LZO is written in ANSI C. Both the source code and the compressed data format
are designed to be portable across platforms. This algorithm is freely available on
the internet (URL: https://www.oberhumer.com/opensource/lzo/). It is made
available by free software foundation. The algorithm is tested on various
operating systems like UNIX and Red Hat Linux.
38
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
Each output data packet contains the INFO HEADER, after mapping the output
decompressed buffer to INFO HEADER find out the data packet and according
to it map the output buffer to respective data packet.
Algorithm:
ST_NIFO_HEADER *pstInfoHeader;
switch (pstInfoHeader->iCode)
{
case CB: //Broadcast Message
{
ST_INDEX_DATA *stIndexData = (ST_INDEX_DATA *)cOutputBuf;
.
.
cOutputBuf = cOutputBuf +
sizeof(ST_INDEX_DATA); break;
}
}
}
39
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
#define DC1 17
#define DC3 19
#define CR 13
#define LF 10
#define POLY 0x1021
// End of defines
unsigned check_sum (cData, iLength)
char *cData ;
int iLength;
{
unsigned uAccum = 0;
unsigned uData;
unsigned char ucChk[2];
int i,j;
for (i=0;i<iLength;i++)
{
uData = *(cData+i);
uData <<= 8;
for(j=8; j>0 ;j--)
{
if((uData^uAccum)&0x8000)
uAccum=(uAccum<<1)^POLY;
/* SHIFT AND SUBTRACT POLY */
else
uAccum<<=1;
uData<<=1;
}
}
ucChk[0] = uAccum>>8;
if (ucChk[0] == DC1 || ucChk[0] == DC3 || ucChk[0] == CR || ucChk[0] == LF )
ucChk[0] -= 1;
ucChk[1] = uAccum&0xFF;
if (ucChk[1] == DC1 || ucChk[1] == DC3 || ucChk[1] == CR || ucChk[1] == LF )
ucChk[1] -= 1;
uAccum = ucChk[1];
uAccum = (uAccum<<8) + ucChk[0];
return(uAccum);
}
40
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
7 Notes
All orders which are of regular lot size or multiples thereof are traded in the
Normal Market. Normal market consists of various book types of wherein orders
are segregated as Regular lot orders, Special Term orders, Negotiated Trade
Orders and Stop Loss orders depending on their order attributes.
In the Auction Market, auctions are initiated by the Exchange on behalf of trading
members for settlement related reasons. There are 3 participants in this market.
• Initiator - the party who initiates the auction process is called an initiator.
• Competitor - the party who enters orders on the same side as the initiator.
• Solicitor - the party who enters orders on the opposite side as of the initiator.
In the auction market the Open price and the Last Traded Price would be zero till
the auction ends and the auction price is calculated by the system. Since Auction
in any scrip is done at a fixed price the High Price, Low Price, Closing Price and
Index values is zero for all scrips traded in the Auction Market.
Pre-open session will be conducted for the Normal Market segment. The session
will be conducted before the normal market start time. Exchange may decide to
allow all or selective securities in pre-open session. During the Pre-open session,
only order entry, orders modification and order cancellation will be allowed. Once
the pre-open session ends, no order activity will be allowed and the final open
price (i.e. equilibrium price based on accumulated buy and sell orders) will be
computed. Pre-open orders will be matched at this final open price resulting in
trade execution. Pre-open orders that could not participate in the pre-open
matching for reasons such as a demand-supply gap, order price worse than the
equilibrium price etc. shall be carried forward to the normal market. The time
priority of such orders shall be retained.
In the above context NSE – Market Feed (Level 1, Level 2 & Level 3) product
sends messages in the following sequence.
1. Pre-open session start (PO) – market type ‘N’
2. Security Update Information (PN) – Indicative open price in open price field
3. Pre-open session end (PC) - market type ‘N’
41
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
4. Security Update Information (PN) – Derived final open price in open price
field and current security information
5. Normal Market open (CO) - market type ‘N’
6. Security Update Information (CN) – With current security
7. Normal Market Close (CC) - market type ‘N’
SME (small and medium enterprises) securities call auction is done through this
session. It is similar to the pre-open session. Multiple sessions of this can be held
on a trading day. Market type for this session is ‘C’
In the above context NSE – Market Feed (Level 1, Level 2 & Level 3) product
sends messages in the following sequence in one call auction session.
1. Pre-open session start (PO) – market type ‘C’
2. Call Auction Security Update Information (SN) – Indicative open price in open
price field.
3. Pre-open session end (PC) - market type ‘C’
4. Security Update Information (SN) – Derived final open price in open price field
and current security information
IPO, Relisting, and illiquid securities call auction is done through this session. It
is similar to the pre-open session. Multiple sessions of this can be held in a trading
day. The market type for this session is ‘G’. IPO/Relisted securities get transferred
to normal market session after deriving the open.
In the above context NSE – Market Feed (Level 1, Level 2 & Level 3) product
sends messages in the following sequence in one call auction session.
1. Pre-open session start (PO) – market type ‘G’
Call Auction Security Update Information (SN) – Indicative open price in open
price field.
2. Pre-open session end (PC) - market type ‘G’
Security Update Information (SN) – Derived final open price in open price field
and current security information
In call auction session 1 & 2 the buy back and market maker orders are allowed.
To identify the buy back or market maker orders BBMM flag is sent in
42
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
the SN messages. For the probable values of BBMM flag refer the table given
below
1. BuyBBMMOrderExists: Buy Back or Market Maker order exists at buy side but
not in top five price points.
2. SellBBMMOrderExists: Buy Back or Market Maker order exist at sell side but
not in top five price points.
3. Sell BBMM Flag: Buy Back or Market Maker order at that price point.
4. Buy BBMM Flag: Buy Back or Market Maker order at that price point.
E.g. If Buy Back and Market Maker orders exist at particular price point then
the above fields will contain ‘3’.
43
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
8 Acronyms Used
44
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
9 FAQs
1) For Sequenced Data Messages, why do fields contain datatype as short, but contain
value is specified as character?
Data sent by server contains number, which is the ASCII value of the field and at
client’s end it needs to be converted from ASCII value into character.
The list of market depth is organized by price levels, and it is updated in real-time
to reflect current activity where:
• Level 1 provides the best Bid and best Ask price.
• Level 2 offers up to the best 5 Bids and Asks prices.
• Level 3 market depth offers a view of the best 20 Bids and Asks prices.
45
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
5) What structures are available for level 1, level 2 and level 3 feeds?
6) How do we interpret Security Eligibility per Market in CT message type and Master
Data security.txt file?
The security.txt file does not populate the market type. It contains 12 fields as
depicted below:
|6|1|2|0|2|0|2|1|1|0|1|0|
There are 12 consecutive fields which indicate Security status and Eligibility for 6
Markets.
In these 12 fields the set of the first 2 fields represents the Normal market and
the set of next 2 fields represent the odd lot market and so on.
46
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
47
Market Feed - CM (Level 1, Level 2 & Level 3) Specification
10 Support Information
Name Email Contact Number
Business & Technical [email protected] +91-22-26598385
Support
48