Rails on HBase
1 like840 views
This document summarizes a presentation about using the HBase database with Ruby on Rails applications. It discusses what HBase is, some of the tradeoffs it involves compared to relational databases, when it may be suitable versus not suitable for an application, and how to interface with it from Rails. Examples are provided of libraries that can be used to connect Rails and HBase, as well as demos of JRuby scripts and Rails code that access an HBase backend.
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Rails on HBase (20)
![[Hi c2011]building mission critical messaging system(guoqiang jerry)](https://cdn.slidesharecdn.com/ss_thumbnails/hic2011buildingmissioncriticalmessagingsystemguoqiangjerry-111206111202-phpapp02-thumbnail.jpg?width=560&fit=bounds)
Ad
More from EffectiveUI (20)
Recently uploaded (20)
Rails on HBase
- 1. Rails on HBase Zachary Pinter and Tony Hillerson RailsConf 2011
- 2. What we will cover
- 3. What is it?
- 4. What are the tradeoffs that HBase makes?
- 5. Why HBase is probably the wrong choice for your app
- 6. Why HBase might be the right choice for your app
- 7. + How to use HBase with Rails
- 8. What we won’t cover
- 9. This is not a deep dive
- 10. This is not a guide to setting up a production HBase cluster *See http://cloudera.com
- 12. HBase is based on Bigtable
- 13. HBase Grew out of Hadoop
- 14. Core Concepts
- 16. Tables column key core:link row llb0ga http://google.com http://en.oreilly.com/rails2011/public/ llb260 schedule/detail/17886
- 17. Column Families core column family stats column family key core:link stats:hits llb0ga http://google.com 100504 http://en.oreilly.com/rails2011/ llb260 2 public/schedule/detail/17886
- 18. Column Families Must be defined at creation or added to schema later Should group data accessed and sized similarly
- 19. Columns Can be dynamically added given that the family exists Stored as a byte array - do what you need with it
- 20. Column Families: Data Locality
- 21. Regions master llb0ga lld92q ... ... llbee9 lq1rro Regionserver Regionserver
- 24. hbase shell ruby-1.9.2-p180 :002 > status 1 servers, 0 dead, 1.0000 average load ruby-1.9.2-p180 :003 > (1..10).each{ status } 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load 1 servers, 0 dead, 3.0000 average load => 1..10 ruby-1.9.2-p180 :004 >
- 26. What are the tradeoffs that HBase makes? NoSQL = Tradeoffs
- 27. Highly Distributed Pros Cons • Built-in sharding • Makes joins difficult/impossible • No single server holds all the data • Limited real-time queries/sorting • Different servers operate on different slices of the data
- 28. Sorted Row Keys Bytearray keys Any kind of data you want Sorted in byte order All row access through the key
- 29. Sorted Row Keys Pros Cons • Real-time random lookups by key • Conflates unique identifier with default • Real-time range queries sort • Keys are an integral part of schema design, requiring extra consideration
- 30. Map / Reduce Pros Cons • Scales linearly: more servers = faster • Scans every row per query * response times • Not real-time (batch operation) • Operates on huge datasets (billions of rows) • Works well with dynamic/non-uniform schemas
- 31. HBase vs. X http://nosql-database.org http://www.sevenweeks.org/ Seven Databases in Seven Weeks Eric Redmond Don’t Miss His Talk!!
- 32. Why HBase is probably the wrong choice for your app
- 34. HBase Limitations Limited real-time queries (key lookup, range lookup) Limited sorting (one default sort per table) Limited transactions Manual indexing Joins/Normalization Storage of large binary data *
- 35. HBase Requirements • At least 5 Servers (Zookeeper, Region Server, Name Nodes, Data Nodes, etc) • Memory Intensive (2-4gb bar minimum per server, depending on server role) • CPU Intensive • Rough EC2 Cost: 5 c1.xlarge reserved for 1 year = over $850/month (not counting bandwidth!)
- 36. Why HBase might be the right choice for your app
- 37. Large data sets Ability to aggregate and analyze billions of rows Starts to shine when relational databases break down Is your db already sharded? Are you using SQL queries that take hours to run?
- 38. Hadoop Integration HDFS for storing files Native Map/Reduce Supports data locality Doesn’t require data to be transferred
- 39. Flexible schema Supports non-heterogeneous data through column families with mixed key/value pairs Ability to refactor the data store via map/reduce (beats a multi-day ALTER TABLE command)
- 40. Rails on HBase
- 41. hbase-stargate https://github.com/greglu/hbase-stargate Uses Stargate, HBase REST server
- 42. Rhino https://github.com/sqs/rhino/ Thrift API Seems to be abandoned?
- 43. Massive Record https://github.com/CompanyBook/massive_record It works! Uses Thrift API
- 44. Kwisatz: An URL Shortener
- 48. Getting Set Up (Mac) brew install hbase Cloudera VM http://bit.ly/dSfPa9
- 49. Recap
- 50. Where Would I Use HBase?
- 51. Where is HBase’s Sweet Spot
- 52. Where Would I Use Rails With HBase?
- 53. Questions?
- 54. Thank You! Zachary: @zpinter Tony: @thillerson EffectiveUI.com