Java Tech & Tools | Beyond the Data Grid: Coherence, Normalisation, Joins and Linear Scalability | Ben Stopford
Download as PPTX, PDF2 likes1,356 views
2011-11-02 | 02:25 PM - 03:15 PM In 2009 RBS set out to build a single store of trade and risk data that all applications in the bank could access simultaniously. This talk discusses a number of novel techniques that were developed as part of this work. Based on Oracle Coherence the ODC departs from the trend set by most caching solutions by holding its data in a normalised form making it both memory efficient and easy to change. However it does this in a novel way that supports most arbitrary queries without the usual problems associated with distributed joins. We'll be discussing these patterns as well as others that allow linear scalability, fault tolerance and millisecond latencies.
More Related Content
Viewers also liked (18)
Similar to Java Tech & Tools | Beyond the Data Grid: Coherence, Normalisation, Joins and Linear Scalability | Ben Stopford (20)
More from JAX London (20)
Recently uploaded (20)
Java Tech & Tools | Beyond the Data Grid: Coherence, Normalisation, Joins and Linear Scalability | Ben Stopford
- 6. The lay of the land: The main architectural constructs in the database industry
- 8. Shared Disk
- 11. ms μs ns ps 1MB Disk/Network 1MB Main Memory 0.000,000,000,000 Cross Continental Main Memory L1 Cache Ref Round Trip Ref Cross Network Round L2 Cache Ref Trip * L1 ref is about 2 clock cycles or 0.7ns. This is the time it takes light to travel 20cm
- 22. Taken from “OLTP Through the Looking Glass, and What We Found There” Harizopoulos et al
- 24. In-Memory TimesTen, Database HSQL, KD B Shared Vertica, Gre Nothing enplumb Regular Distributed Exasol, Database In-Memory VoltDB, Hana Oracle, Sybase, MySql NoSQL Mongo, Cass andra Data Grid Coherence, Teracotta
- 25. Distributed Architecture Simplify the Contract Stick to RAM
- 26. 450 processes 2TB of RAM Oracle Coherenc e Messaging (Topic Based) as a system of record (persistence)
- 27. Access Layer Jav Jav a a clie clie nt API nt API Query Layer Transaction Data Layer s Mtms Cashflows Persistence Layer
- 28. Indexing Partitioning Replication
- 30. But your storage is limited by the memory on a node
- 31. Associating data in different partitions implies moving it. Scalable storage, bandwidth and processing
- 36. Trade r Part Version 1 y Trad e Trade r Part Version 2 y Trad e Trade r Part Version 3 y Trad e Trade r Part Version 4 y Trad …and you need e versioning to do MVCC
- 37. Trad Trade Part e r y Part Trade Trad y r e Part y Trade r Trad e Part y
- 38. So we want to hold entities separately (normalised) to alleviate concerns around consistency and space usage
- 39. Independently Data is Versioned Trade Singleton r Part y Trad e Trad Trade Part e r y
- 40. Trade r Part y Trad e Trad Trade Part e r y
- 44. This is what using Snowflake Schemas and the Connected Replication pattern is all about!
- 47. Crosscuttin g Keys Common Keys
- 48. Replicated Trader Party Trade Partitioned
- 57. Use a Key Assignment Trade Policy MTMs (e.g. KeyAssociation s in Coherence) Common Key
- 58. Replicated Trader Party Trade Partitioned
- 59. Query Layer Trader Party Trade Transactions Data Layer Mtms Cashflows Fact Storage (Partitioned)
- 60. Dimension s (repliacte) Transactions Facts Mtms Cashflows (distribute/ partition) Fact Storage (Partitioned)
- 61. Facts: =>Big =>Distribut e Dimensions =>Small => Replicate
- 62. We use a variant on a Snowflake Schema to partition big entities that can be related via a partitioning key and replicate small stuff who’s keys can’t map to our partitioning key.
- 64. Select Transaction, MTM, RefrenceData From MTM, Transaction, Ref Where Cost Centre = ‘CC1’
- 65. Get Get Get Get Get Get Get Cost Ledger Source Transa MTMs Legs Cost Center Books Books c-tions Center s s Network Time
- 66. Get Get Get Get Get Get Get Cost Ledger Source Transac MTMs Legs Cost Centers Books Books -tions Centers Network
- 68. Select Transaction, MTM, ReferenceData From MTM, Transaction, Ref Where Cost Centre = ‘CC1’ Join Dimensions in Query Layer Transactions Mtms Cashflows Partitioned
- 69. Select Transaction, MTM, ReferenceData From MTM, Transaction, Ref Where Cost Centre = ‘CC1’ Join Dimensions in Query Layer Transactions Join Facts Mtms acrossCashflows cluster Partitioned
- 71. Select Transaction, MTM, ReferenceData From MTM, Transaction, Ref Where Cost Centre = ‘CC1’ Join Join Dimensions Dimensions in Query Layer in Query Layer Transactions Join FactsMtms across Cashflows cluster Partitioned
- 73. Jav Replicated a clie Partitioned nt API Dimensions Facts We never have to do a distributed join!
- 74. So all the big stuff is held partitioned And we can join without shipping keys around and having intermediate results
- 75. Trade r Part y Trad e Trad Trade Part e r y
- 76. Trade r Part Version 1 y Trad e Trade r Part Version 2 y Trad e Trade r Part Version 3 y Trad e Trade r Part Version 4 y Trad e
- 77. Trad Trade Part e r y Part Trade Trad y r e Part y Trade r Trad e Part y
- 83. Facts This is a dimension • It has a different key to the Facts. Dimensions • And it’s BIG
- 91. So we only replicate ‘Connected’ or ‘Used’ dimensions
- 92. Processing Layer Dimension Caches (Replicated) Transactions Data Layer As new Facts are added Mtms relevant Dimensions that they reference are moved Cashflows to processing layer caches Fact Storage (Partitioned)
- 94. Query Layer Save Trade (With connected dimension Caches) Data Layer Cache Trad (All Normalised) Store e Partitioned Trigger Cache Party Sourc Ccy Alias e Book
- 95. Query Layer (With connected dimension Caches) Data Layer Trad (All Normalised) e Party Sourc Ccy Alias e Book
- 96. Query Layer (With connected dimension Caches) Data Layer Trad (All Normalised) e Party Sourc Ccy Alias e Book Party Ledge rBook
- 97. ‘Connected Replication’ A simple pattern which recurses through the foreign keys in the domain model, ensuring only ‘Connected’ dimensions are replicated
- 98. With ‘Connected Replication’ only 1/10th of the data needs to be replicated (on average).
- 100. Jav a Java schema clie nt API Java ‘Stored Procedures’ and ‘Triggers’
- 102. Query with more than twenty joins conditions: 2GB per min / 250Mb/s 3ms latency (per client)
- 105. Partitioned Storage
Editor's Notes
- #49: Big data sets are held distributed and only joined on the grid to collocated objects.Small data sets are held in replicated caches so they can be joined in process (only ‘active’ data is held)
- #59: Big data sets are held distributed and only joined on the grid to collocated objects.Small data sets are held in replicated caches so they can be joined in process (only ‘active’ data is held)