JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
Download as PPTX, PDF0 likes216 views
By leveraging memory-mapped files, Speedment and the Chronicle Engine supports large Java maps that easily can exceed the size of your server’s RAM.Because the Java maps are mapped onto files, these maps can be shared instantly between several microservice JVMs and new microservice instances can be added, removed, or restarted very quickly. Data can be retrieved with predictable ultralow latency for a wide range of operations. The solution can be synchronized with an underlying database so that your in-memory maps will be consistently “alive.” The mapped files can be tens of terabytes, which has been done in real-world deployment cases, and a large number of micro services can share these maps simultaneously. Learn more in this session.
![4. API Java 8 Stream API
REST API
users.stream()
.filter(AGE.greaterThen(42))
.forEach(someOperation());
https://mysite/user/
?filter=[{"property":”age",
”operator":”gt","value":42}]](https://image.slidesharecdn.com/javaone2016microservicesfinal-160922165618/85/JavaOne2016-Microservices-Terabytes-in-Microseconds-CON4516-39-320.jpg)
![4. API Java 8 Stream API
REST API
users.stream()
.filter(AGE.greaterThen(42))
.skip(10)
.limit(100)
.sorted(NAME.comparator())
.forEach(someOperation());
https://mysite/user/
?filter=[{"property":”age",
”operator":”gt","value":42}]
&skip=10
&limit=100
&sort=[{"property":”name", ”direction":”ASC”}]](https://image.slidesharecdn.com/javaone2016microservicesfinal-160922165618/85/JavaOne2016-Microservices-Terabytes-in-Microseconds-CON4516-40-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (16)
Ad
Similar to JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516] (20)
Ad
More from Speedment, Inc. (12)
Recently uploaded (20)
JavaOne2016 - Microservices: Terabytes in Microseconds [CON4516]
- 1. Microservices: Terabytes in Microseconds Peter Lawrey Higher FrequencyTrading Ltd Per Minborg Speedment, Inc.
- 2. About Per 3
- 3. About Peter
- 4. Microservices • Microservices bring together best practices from a variety of areas • Most likely you are already using some of these best practices
- 5. Microservices • It sounds like marketing hype • It all sounds pretty familiar • It just a rebranding of stuff we already do • There is room for improvement in what we do • There are some tools, and ideas we could apply to our systems without changing too much
- 6. • Simple component based design • Asynchronous messaging • Automated, dynamic deployment of services • Service private data sets • Transparent messaging • Teams can develop independently based on well defined contracts Microservices
- 7. Terabytes in Microseconds Solution Speed Data volume GB TB EB us ms s min hour days
- 8. Speedment Chronicle Terabytes in Microseconds Solution Maps the data in a key-value store Synchronizes SQL Data into an in- JVM-memory data store Maps the data in Column key stores Provides application APIs Terabytes 1. Synchronization 2. Key Value Store 3. Column Key Stores 4. API Microservice 3 Microservice 2 Microservice
- 9. 1. Synchronization • Synchronizes SQL Data with the in-JVM-memory solution: • Initially • Over time • Periodically • Reactively
- 10. 1. Synchronization Periodically • Dumps are reloaded periodically • All data elements are reloaded • Data remains unchanged between reloads • System restart is just a reload • MVCC so that several dumps can be active at the same time Reactively • Changed data is captured in the Database • Changed data events are pushed into memory • Events are grouped in transactions • Cache updates are persisted • Data changes all the time • System restart, replay the missed events or
- 11. 1. Synchronization * Periodically Reactively Poll Comparison Max Data Age Dump period Replication Latency- ms Eviction time Lookup Performance Consistently Instant Consistently Instant 20% very slow Consistency Eventually Consistent Eventually Consistent Inconsistent – stale data Database Cache Update Load ~ Total Size / Dump ~ Rate of Change ~ Cache size * Eviction Restart Complete Reload Down time*update rate - >10% of down time Heat-up time
- 12. 1. Synchronization Properties • Detect changes in a database • Buffer the changes • Can replay the changes later on • Preserve order • Preserve transactions • See data as it was persisted • Detect changes from any source JVM Column Oriented CQRS In-JVM- Memory
- 13. 1. Synchronization How is data synchronized? • Periodic MVCC reload • Transaction log harvesting • Event Sourcing and CQRS
- 14. 2. Key Value Store Makes it possible to • Having Maps that can exceed the server’s RAM by up to 40x • Opening up the path to mammoth JVMs with tens of terabytes • Data persistence inside JVM provides fast restart • When you have multiple instances of a service they can share the data with a single copy in memory
- 15. 32 bit Operating System (31-bit heap)
- 16. Compress Oops in Java 7 (35-bit) • Using the default of –XX:+UseCompressedOops • In a 64-bit JVM, it can use “compressed” memory references • This allows the heap to be up to 32 GB without the overhead of 64-bit object references. The Oracle/OpenJDK JVM still uses 64-bit class references by default • As all object must be 8-byte aligned, the lower 3 bits of the address are always 000 and don’t need to be stored. This allows the heap to reference 4 billion * 8-bytes or 32 GB • Uses 32-bit references
- 17. Compressed Oops with 8 Byte Alignment
- 18. Compress Oops in Java 8 (36 bits) • Using the default of –XX:+UseCompressedOops –XX:ObjectAlignmentInBytes=16 • In a 64-bit JVM, it can use “compressed” memory references • This allows the heap to be up to 64 GB without the overhead of 64-bit object references. The Oracle/OpenJDK JVM still uses 64-bit class references by default • As all object must be 8 or 16-byte aligned, the lower 3 or 4 bits of the address are always zeros and don’t need to be stored. This allows the heap to reference 4 billion * 16-bytes or 64 GB • Uses 32-bit references
- 19. 64-bit References in Java (100+ GB) • A small but significant overhead on main memory use • Reduces the efficiency of CPU caches as less objects can fit in • Can address up to the limit of the free memory. Limited to main memory • GC pauses become a real concern and can take tens of second or many minutes • For larger heap sizes, using the G1 collector may be a good choice.
- 20. NUMA Regions (~40 bits) • Large machine are limited in how large a single bank of memory can be. This varies based on the architecture • Ivy and Sandy bridge Xeon processors are limited to addressing 40 bits of real memory • In Haswell and Boradwell this has been lifted to 46-bits • Each Socket has “local” access to a bank of memory, however to access other bank it may need to use a bus. This is much slower • The GC of a JVM can perform very poorly if it doesn’t sit within one NUMA region. Ideally you want a JVM to use just one NUMA region
- 21. NUMA Regions (~40 bits)
- 22. Virtual Address Space (48-bit)
- 23. Virtual Address Space (48-bit)
- 24. Memory Mapped Files (48+ bits) • Memory mappings are not limited to main memory size • 64-bit OS support 128TiB to 256TiB virtual memory at once • For larger data sizes, memory mapping need to be managed and cached manually • Can be shared between processes • A library can hide the 48-bit limitation by caching memory mapping
- 25. Peta Byte JVMs (50+ bits) • If you are receiving 1 GB/s down a 10 Gig-E line in two weeks you will have received over 1 PB • Managing this much data in large servers is more complex than your standard JVM • Replication is critical. Large complex systems, are more likely to fail and take longer to recover • You can have systems which cannot be recovered in the normal way. i.e. Unless you recover faster than new data is added, you will never catch up
- 26. Peta Byte JVMs (50+ bits)
- 27. Peta Byte JVMs (50+ bits)
- 28. 3. Column Key Store • Column Key Stores are used much like database indexes • The Column Key Stores can produce data lazily • Can be mapped onto memory mapped files
- 29. 3. Column Key Store • Features over ConcurrentNavigableMap • Handles null values • Can handle several values for one key • Can serialize data off-heap • Can map onto files • Can compress its keys • Contains a bi-directional skip dictionary • Remains performant for billions and trillions of entries • Supports transactions
- 30. 3. Column Key Store Filesystem Page cache User Space Kernel Space Physical memory Disk blocks SSD Page mapping Microservice JVM mapped buffer filesystem pages memory pages
- 31. 3. Column Key Store Filesystem Page cache User Space Kernel Space Physical memory Disk blocks SSD Page mapping Microservice1 JVM 2 filesystem pages memory pages mapped buffer Microservice1 JVM 1 mapped buffer
- 32. 3. Column Key Store • O(1) operations: • equals, notEquals • isNull, isNotNull • in, notIn • sort • O(log N) operations • <, <=, >, >= • between • startsWith(String) • O(N) • matches(regexp)
- 33. 3. Column Key Store 0 50 100 150 200 250 300 350 1 10 20 30 40 50 60 70 80 90 100 Big-Ohs O(1) O(log N) O(N) O(NlogN)
- 34. 3. Column Key Store Key-Value Store ID Column Key Store User.AGE.greaterThan(42) AGE Column Key Store NAME Column Key Store greaterThan(42) Keys Values User
- 35. 3. Column Key Store - The Stream Pipeline Key-Value Store AGE Column Key Store greaterThan(42) Keys Values
- 36. 3. Column Key Store - Parallelism Key-Value Store AGE Column Key Store greaterThan(42) Parallel Keys Parallel Values Optional Parallel Strategy
- 37. 4. API • Provides application APIs • Java 8 Stream API • REST API
- 38. 4. API • Speedment generates Java code automatically from SQL metadata • Free from hand coding errors • Reduces development time • Improves the application quality • Reduces maintenance for applications • Uses standard streams for querying • Custom code possibilities
- 39. 4. API Java 8 Stream API REST API users.stream() .filter(AGE.greaterThen(42)) .forEach(someOperation()); https://mysite/user/ ?filter=[{"property":”age", ”operator":”gt","value":42}]
- 40. 4. API Java 8 Stream API REST API users.stream() .filter(AGE.greaterThen(42)) .skip(10) .limit(100) .sorted(NAME.comparator()) .forEach(someOperation()); https://mysite/user/ ?filter=[{"property":”age", ”operator":”gt","value":42}] &skip=10 &limit=100 &sort=[{"property":”name", ”direction":”ASC”}]
- 41. 4. API - Parallelism • Java 8 Stream API users.stream() .parallel() .filter(AGE.greaterThen(42)) .flatMap(users::findLogins) .forEach(expensiveOperation());
- 42. Demo 1 • 300,000 Users • Off Heap Data Store and Key-Value map • Test • Find all users with id = 42 • Count them • Add the count to a sum • Iterate 1,000,000 times • JVMWarmup and then Benchmark • Sum will be 1,000,000 • A standard MySQL database will process 2,000TPS on a 4 CPU laptop like mine
- 43. Demo 2 • 1,000,000 Users • Tests • ID.equal(42) • ID.greaterOrEqual(42) • ID.in(42, 43, 44) • ID.between(42, 45)
- 45. Compare Latencies Using the Speed of Light During the time a database makes a 1 s query, how far will the light move? Database CPU L1 cache Conclusion : Do not place your data on the moon, keep it close by using in-JVM-memory technology!
- 46. Epic Threshold for Mankind Passed in 2016 Conclusion: Quit drinking coffee and buy more RAM for your application server instead!
- 47. Microservices Solution MS1 MS1 MS1 MS2 MS2 MS2 MS3 MS3 MS3 Speedment Chronicle Speedment Chronicle Speedment Chronicle
- 48. Microservices JVM • The solution maps Java NavigableMaps to persistent stores (files) • The views of the maps are available for any JVM that has access to these files • Microservice JVMs can be started or restarted very rapidly • Access to these maps will be gained in millisecond time • These maps can be shared instantly between several microservice JVMs • New microservice instances are added, removed, or restarted very quickly
- 49. If You Only Remember One Thing…. Speed Data volume GB TB EB us ms s min hour days
Editor's Notes
- #3: I live in Palo Alto in California..