Infinspan: In-memory data grid meets NoSQL
Download as KEY, PDF8 likes3,058 views
The document discusses Infinispan, an open source in-memory data grid. It describes Infinispan's features like its map-like API, persistence, querying support and transactional capabilities. It also discusses how Infinispan can be used to build elastic, scalable data services and compares data grids to NoSQL databases.
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Infinspan: In-memory data grid meets NoSQL (20)
![[WSO2Con EU 2017] Keynote: Ballerina Plans and Futures](https://cdn.slidesharecdn.com/ss_thumbnails/2017-11-07-wso2coneu-ballerina-171108104750-thumbnail.jpg?width=560&fit=bounds)
Ad
Recently uploaded (20)
Infinspan: In-memory data grid meets NoSQL
- 1. [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 2. Infinispan In-memory data grid meets NoSQL Manik Surtani Founder and project lead, Infinispan Senior Principal Engineer Red Hat, Inc. [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 3. “Could data storage be the one thing that hampers true cloud scalability and elasticity?” [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 4. Who is Manik? • R&D Engineer, Red Hat Inc. • Founder and project lead, Infinispan • Project lead, JBoss Cache • Frequent speaker on cloud computing and cloud data storage http://twitter.com/maniksurtani http://blog.infinispan.org [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 5. Agenda • What is Data-as-a-Service? • Introducing Infinispan • Implementing Data-as-a-Service with data grids • Data grids vs. NoSQL [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 6. Traditional 3-tier App [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 7. Typical IaaS App [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 8. Typical PaaS App [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 9. Where’s your data stored?? [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 10. Clouds are ephemeral! [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 11. State [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 12. Virtualizing Data • Some public services do exist • Amazon RDS and SimpleDB • FathomDB • Cloundant • MongoHQ • etc. [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 13. What about private clouds? • Not all cloud deployments are public! • Private cloud is very important • How can you build DaaS yourself? [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 14. Characteristics of DaaS • Elastic data • Needs to scale with other tiers • Response times should be linear • Needs to be highly available • Nodes will die! The service shouldn’t. [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 15. “Traditional” RDBMSs • Lack of distribution hampers elasticity and HA • These limitations can be worked around • ... but this isn’t trivial • ... or cheap [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 16. “Traditional” RDBMSs • Lack of distribution hampers elasticity and HA • These limitations can be worked around • ... but this isn’t trivial • ... or cheap [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 17. Distributed Data Grid • Far better suited to elastic data • Distributed by nature • Highly available by nature • A good building block for your data service [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 18. API is king • Apps should use their native data storage APIs • E.g., JPA (Java EE), ActiveRecord (Ruby), etc. • Key/value too low level • Akin to direct JDBC calls! [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 19. Introducing [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 20. What is Infinispan? • Open source (LGPL) in-memory Data Grid • Some concepts from Amazon Dynamo 2 usage modes • Embedded • Client-server • memcached • Hot Rod • REST [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 21. Embedded Mode [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 22. P2P Embedded Architecture [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 23. API • Map-like key/value store • Upcoming JPA-like layer • Other high-level APIs being discussed in the community e.g., ActiveRecord [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 24. Consistent hash based distribution • Self healing • No single point of failure Highly concurrent • MVCC locking [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 25. Persistence • Not just in memory! • Write through and write behind • Pluggable “drivers” Eviction and expiry • Efficient, adaptive algorithms • Addresses shortcomings of LRU & FIFO [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 26. XA Transactions • 2-phase commit based • Deadlock detection algorithms • Coming soon: Atomic Broadcast [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 27. Map/Reduce • In a pre-release state right now • Please try out 5.0.0.ALPHA3 with these APIs! Querying • Using Lucene and Hibernate Search to index [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 28. Client/Server Mode [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 29. Client/Server Architecture Supported Protocols • REST • Memcached • Hot Rod [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 30. WTF is Hot Rod? • Wire protocol for client server communications • Open • Language independent • Built-in failover and load balancing • Smart routing [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 31. Server Endpoint Comparison Client Clustered Smart Load Balancing/ Protocol Libraries ? Routing Failover REST Text N/A Yes No Any HTTP load balancer Only with predefined Memcached Text Plenty Yes No server list Currently Hot Rod Binary Yes Yes Dynamic only Java [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 32. So is Infinispan a data grid? • In-memory • P2P, distributed • Low-latency, fast key/ value store [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 33. So is Infinispan a data grid? • In-memory • P2P, distributed ... or is it a NoSQL • Low-latency, fast key/ value store database? • Persistence • Map/Reduce • Client/Server mode [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 34. So is Infinispan a data grid? • In-memory • P2P, distributed ... or is it a NoSQL • Low-latency, fast key/ value store database? • Persistence ... or something • Map/Reduce • Client/Server mode else? • Querying support • Transactional [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 35. DaaS with Infinispan [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 36. Architecture Manage and Monitor [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 37. Data Grids vs. NoSQL [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 38. Data Grids vs. NoSQL • Data grids/distributed caches as proto-NoSQL? • Been around since early 2000’s • Used as database offload • Bottleneck removal • Primary K/V stores for certain types of data • Low latency data access [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 39. Their roots Data Grids in-memory, non-clustered -> add clustering -> add persistence -> add txns -> add M/R NoSQL persistence -> add clustering -> add M/R -> add memory cache [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 40. Evolving: Enterprise NoSQL • Taking NoSQL mainstream • Adding enterprise features • Transactions • Querying • Adding common APIs • Easy access, portability [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 41. NoSQL standards • APIs - basic interactions and Map/Reduce • JCP for Java + others • http://bit.ly/data_grid_jsr • Wire protocol • Remote communications. • Hot Rod? • Query language • Without relational presumptions [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 42. Summing things up • Elastic data is hard • Public data services not always suitable • Data grids make elastic storage easy • Infinispan server endpoints help build elastic data tiers • Discussed data grids versus NoSQL [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 43. Questions? http://www.infinispan.org http://blog.infinispan.org http://github.com/infinispan http://twitter.com/infinispan - #infinispan #infinispan on FreeNode (IRC) [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 44. Starting an Infinispan Server • Hot Rod or memcached server endpoint $ bin/startServer.sh -r hotrod -c infinispan.xml $ bin/startServer.sh -r memcached -c infinispan.xml [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 45. Starting an Infinispan Server • REST endpoint • Deploy infinispan-server-rest.war in your favorite servlet container. [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 46. Roadmap 4.0.0 Starobrno • Map-like API • Async API • Consistent Hash based distribution • Write-through, write-behind • Eviction, expiration • Management tooling • REST API • Hibernate 2nd Level Cache • Released Feb 2010 [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 47. Roadmap 4.1.0 Radegast • Deadlock detection • Client/Server protocols • Memcached • Hot Rod • Smart clients using Hot Rod • Lucene Directory implementation • LIRS: adaptive, recency-based eviction policies • Released August 2010 [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org
- 48. Roadmap 5.0.0 Pagoa • JPA-like API • Fine-grained replication • Distributed code execution • Map/reduce • Virtual nodes for more even distribution • In active development 5.1.0 and beyond • Dynamic provisioning based on SLAs • Complex event processing features [email protected] | http://twitter.com/maniksurtani | http://blog.infinispan.org