Successful Software Development with Apache Cassandra
6 likes922 views
The document provides an overview of software development with Cassandra, including data modeling, writing code, testing, reviewing code, and managing environments. It discusses using the Java driver for writing code, preparing statements, batches, and lightweight transactions. Testing recommendations include using naming schemes, grouping tests into logical suites, and integrating unit and integration tests. The document also covers tracing queries, instrumentation with metrics, and reviewing code.
![Testing - Suites:
Interesting test plumbing
• [Before|Afer]Suite
• [Before|After]Group
• Listeners](https://image.slidesharecdn.com/softdevwithcassandraawalkthrough-151109190909-lva1-app6892/85/Successful-Software-Development-with-Apache-Cassandra-78-320.jpg)
![server_count = 3
network = '192.168.2.'
first_ip = 10
servers = []
seeds = []
cassandra_tokens = []
(0..server_count-1).each do |i|
name = 'node' + (i + 1).to_s
ip = network + (first_ip + i).to_s
seeds << ip
servers << {'name' => name,
'ip' => ip,
'initial_token' => (2**64 / server_count * i) - 2**63}
end](https://image.slidesharecdn.com/softdevwithcassandraawalkthrough-151109190909-lva1-app6892/85/Successful-Software-Development-with-Apache-Cassandra-128-320.jpg)
![server_count = 3
network = '192.168.2.'
first_ip = 10
servers = []
seeds = []
cassandra_tokens = []
(0..server_count-1).each do |i|
name = 'node' + (i + 1).to_s
ip = network + (first_ip + i).to_s
seeds << ip
servers << {'name' => name,
'ip' => ip,
'initial_token' => (2**64 / server_count * i) - 2**63}
end](https://image.slidesharecdn.com/softdevwithcassandraawalkthrough-151109190909-lva1-app6892/85/Successful-Software-Development-with-Apache-Cassandra-129-320.jpg)
![server_count = 3
network = '192.168.2.'
first_ip = 10
servers = []
seeds = []
cassandra_tokens = []
(0..server_count-1).each do |i|
name = 'node' + (i + 1).to_s
ip = network + (first_ip + i).to_s
seeds << ip
servers << {'name' => name,
'ip' => ip,
'initial_token' => (2**64 / server_count * i) - 2**63}
end](https://image.slidesharecdn.com/softdevwithcassandraawalkthrough-151109190909-lva1-app6892/85/Successful-Software-Development-with-Apache-Cassandra-130-320.jpg)
![chef.json = {
:cassandra => {'cluster_name' => 'VerifyCluster',
'version' => '2.0.8',
'setup_jna' => false,
'max_heap_size' => '512M',
'heap_new_size' => '100M',
'initial_token' => server['initial_token'],
'seeds' => "192.168.2.10",
'listen_address' => server['ip'],
'broadcast_address' => server['ip'],
'rpc_address' => server['ip'],
'conconcurrent_reads' => "2",
'concurrent_writes' => "2",
'memtable_flush_queue_size' => "2",
'compaction_throughput_mb_per_sec' => "8",
'key_cache_size_in_mb' => "4",
'key_cache_save_period' => "0",
'native_transport_min_threads' => "2",
'native_transport_max_threads' => "4"
},
}](https://image.slidesharecdn.com/softdevwithcassandraawalkthrough-151109190909-lva1-app6892/85/Successful-Software-Development-with-Apache-Cassandra-131-320.jpg)
Successful Software Development with Apache Cassandra
- 1. CASSANDRA DAY DALLAS 2015 SOFTWARE DEVELOPMENT WITH CASSANDRA: A WALKTHROUGH Nate McCall @zznate Co-Founder & Sr.Technical Consultant http://www.slideshare.net/zznate/soft-dev-withcassandraawalkthrough Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License
- 2. AboutThe Last Pickle. Work with clients to deliver and improve Apache Cassandra based solutions. Based in New Zealand,Australia & USA.
- 3. OVERVIEW DATA MODELING WRITING CODE TESTING REVIEWING MANAGING ENVIRONMENTS
- 4. Overview: What makes a software development project successful?
- 5. Overview: Successful Software Development - it ships - maintainable - good test coverage - check out and build
- 7. OVERVIEW DATA MODELING WRITING CODE TESTING REVIEWING MANAGING ENVIRONMENTS
- 8. Data Modeling: … a topic unto itself. But quickly:
- 9. Data Modeling - Quickly • It’s Hard • Do research • #1 performance problem • Don’t “port” your schema!
- 10. Data Modeling - Using CQL: • tools support • easy tracing (and trace discovery) • documentation* *Maintained in-tree: https://github.com/apache/cassandra/blob/cassandra-1.2/doc/cql3/CQL.textile https://github.com/apache/cassandra/blob/cassandra-2.0/doc/cql3/CQL.textile https://github.com/apache/cassandra/blob/cassandra-2.1/doc/cql3/CQL.textile
- 11. Data Modeling - DevCenter: Tools: DataStax DevCenter http://www.datastax.com/what-we-offer/products-services/devcenter
- 13. OVERVIEW DATA MODELING WRITING CODE TESTING REVIEWING MANAGING ENVIRONMENTS
- 15. Writing Code - Java Driver: Use the Java Driver • Reference implementation • Well written, extensive coverage • Open source • Dedicated development resources https://github.com/datastax/java-driver/
- 16. Writing Code - Java Driver: Existing Spring Users: Spring Data Integration http://projects.spring.io/spring-data-cassandra/
- 17. Writing Code - Java Driver: Four rules for Writing Code • one Cluster for physical cluster • one Session per app per keyspace • use PreparedStatements • use Batches to reduce network IO
- 18. Writing Code - Java Driver: Configuration is Similar to Other DB Drivers (with caveats**) http://www.datastax.com/documentation/developer/java-driver/2.1/common/drivers/reference/clusterConfiguration_c.html
- 19. Writing Cluster - Java Driver - Configuration: Major Difference: it’s a Cluster!
- 20. Writing Code - Java Driver - Configuration: Two groups of configurations • policies • connections
- 21. Writing Code - Java Driver - Configuration: Three Policy Types: • load balancing • connection • retry
- 22. Writing Code - Java Driver - Configuration: Connection Options: • protocol* • pooling** • socket *https://github.com/apache/cassandra/blob/cassandra-2.1/doc/native_protocol_v3.spec **https://github.com/datastax/java-driver/tree/2.1/features/pooling
- 23. Writing Code - Java Driver - Configuration: Code sample for building a Cluster
- 28. https://github.com/datastax/java-driver/tree/2.1/features/compression
- 30. Writing Code - Java Driver - Pagination: Simple result iteration CREATE TABLE IF NOT EXISTS transit.vehicle_data ( vehicle_id text, speed double, time timeuuid, PRIMARY KEY ((customer_id), time) );
- 39. Writing Code - Java Driver - Pagination: Simple result iteration: Java 8 style
- 46. Writing Code - Java Driver - Async Async! (not so) Simple result iteration
- 51. Writing Code - Java Driver - Pagination: Not much to it: PreparedStatement prepStmt = session.prepare(CQL_STRING); BoundStatement boundStmt = new BoundStatement(prepStmt); boundStatement.setFetchSize(100) https://github.com/datastax/java-driver/tree/2.1/features/paging
- 52. Writing Code - Java Driver - Inserts and Updates: About Inserts (and updates)
- 53. Writing Code - Java Driver - Inserts and Updates: Batches: three types - logged - unlogged - counter
- 58. Writing Code - Java Driver - Inserts and Updates: unlogged batch
- 60. Writing Code - Java Driver - Inserts and Updates: LWT: INSERT INTO vehicle (vehicle_id, make, model, vin) VALUES ('VHE-101', 'Toyota','Tercel','1234f') IF NOT EXISTS;
- 61. Writing Code - Java Driver - Inserts and Updates: LWT: UPDATE vehicle SET vin = '123fa' WHERE vehichle_id = 'VHE-101' IF vin = '1234f';
- 62. Writing Code: ORM? Great for basic CRUD operations http://www.datastax.com/documentation/developer/java-driver/2.1/java-driver/reference/crudOperations.html
- 67. Writing Code - Java Driver: A note about User Defined Types (UTDs)
- 68. Writing Code - Java Driver - Using UDTs: Wait. - serialized as blobs !!?! - new version already being discussed* - will be a painful migration path * https://issues.apache.org/jira/browse/CASSANDRA-7423
- 69. OVERVIEW DATA MODELING WRITING CODE TESTING REVIEWING MANAGING ENVIRONMENTS
- 70. Testing: Use a Naming Scheme • *UnitTest.java: no external resources • *ITest.java: uses external resources • *PITest.java: safely parallel “ITest”
- 71. Testing: Tip: wildcards on the CLI are not a naming schema.
- 73. Testing - Suites: Benefits of Suites: • share test data • share Cassandra instance(s) • build profiles
- 76. Testing - Suites: Using annotations for suites in code
- 78. Testing - Suites: Interesting test plumbing • [Before|Afer]Suite • [Before|After]Group • Listeners
- 80. Testing: scassandra: not quite integration http://www.scassandra.org/
- 86. Testing: Verify Assumptions: test failure scenarios explicitly
- 87. Testing - Integration: Runtime Integrations: • local • in-process • forked-process
- 88. Testing - Integration - Runtime: EmbeddedCassandra https://github.com/jsevellec/cassandra-unit/
- 89. Testing - Integration - Runtime: ProcessBuilder to fork Cassandra(s)
- 90. Testing - Integration - Runtime: CCMBridge: delegate to CCM https://github.com/datastax/java-driver/blob/2.1/driver-core/src/test/java/com/datastax/driver/core/CCMBridge.java
- 91. Testing - Integration: Best Practice: Jenkins should be able to manage your cluster
- 92. Testing: Load Testing Goals • reproducible metrics • catch regressions • test to breakage point
- 93. Testing - LoadTesting: Stress.java (lot’s of changes recently) https://www.datastax.com/documentation/cassandra/2.1/cassandra/tools/toolsCStress_t.html http://www.datastax.com/dev/blog/improved-cassandra-2-1-stress-tool-benchmark-any-schema
- 94. Testing - LoadTesting: Workload recording and playback coming soon one day https://issues.apache.org/jira/browse/CASSANDRA-8929
- 95. Testing: Primary testing goal: Don’t let cluster behavior surprise you.
- 96. OVERVIEW DATA MODELING WRITING CODE TESTING REVIEWING MANAGING ENVIRONMENTS
- 97. Writing Code: Metrics API for your own code https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/metrics/ColumnFamilyMetrics.java https://dropwizard.github.io/metrics/3.1.0/
- 98. Writing Code - Instrumentation via Metrics API: Run Riemann locally http://riemann.io/
- 100. Reviewing Said Code: Using Trace (and doing so frequently)
- 101. Writing Code -Tracing: Trace per query via DevCenter http://www.datastax.com/documentation/cql/3.0/cql/cql_reference/tracing_r.html
- 102. Writing Code -Tracing: Trace per query via cqlsh http://www.datastax.com/documentation/cql/3.0/cql/cql_reference/tracing_r.html
- 103. Writing Code -Tracing: Trace per query via Java Driver http://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/Statement.html#enableTracing()
- 104. cqlsh> tracing on; Now tracing requests. cqlsh> SELECT doc_version FROM data.documents_by_version ... WHERE application_id = myapp ... AND document_id = foo ... AND chunk_index = 0 ... ORDER BY doc_version ASC ... LIMIT 1; doc_version ------------- 65856 Tracing session: 46211ab0-2702-11e4-9bcf-8d157d448e6b
- 105. Preparing statement | 18:05:44,845 | 192.168.1.197 | 22337 Enqueuing data request to /192.168.1.204 | 18:05:44,845 | 192.168.1.197 | 22504 Sending message to /192.168.1.204 | 18:05:44,847 | 192.168.1.197 | 24498 Message received from /192.168.1.197 | 18:05:44,854 | 192.168.1.204 | 872 Executing single-partition query on documents_by_version | 18:05:44,888 | 192.168.1.204 | 35183 Acquiring sstable references | 18:05:44,888 | 192.168.1.204 | 35459 Merging memtable tombstones | 18:05:44,889 | 192.168.1.204 | 35675 Key cache hit for sstable 2867 | 18:05:44,889 | 192.168.1.204 | 35792 Seeking to partition beginning in data file | 18:05:44,889 | 192.168.1.204 | 35817 …
- 106. Preparing statement | 18:05:44,845 | 192.168.1.197 | 22337 Enqueuing data request to /192.168.1.204 | 18:05:44,845 | 192.168.1.197 | 22504 Sending message to /192.168.1.204 | 18:05:44,847 | 192.168.1.197 | 24498 Message received from /192.168.1.197 | 18:05:44,854 | 192.168.1.204 | 872 Executing single-partition query on documents_by_version | 18:05:44,888 | 192.168.1.204 | 35183 Acquiring sstable references | 18:05:44,888 | 192.168.1.204 | 35459 Merging memtable tombstones | 18:05:44,889 | 192.168.1.204 | 35675 Key cache hit for sstable 2867 | 18:05:44,889 | 192.168.1.204 | 35792 Seeking to partition beginning in data file | 18:05:44,889 | 192.168.1.204 | 35817 …
- 107. … Merging data from memtables and 8 sstables | 18:05:44,892 | 192.168.1.204 | 38605 Read 1 live and 2667 tombstoned cells | 18:05:54,135 | 192.168.1.204 | 9282428 Enqueuing response to /192.168.1.197 | 18:05:54,136 | 192.168.1.204 | 9283423 Sending message to /192.168.1.197 | 18:05:54,138 | 192.168.1.204 | 9284753 Message received from /192.168.1.204 | 18:05:54,155 | 192.168.1.197 | 9332505 Processing response from /192.168.1.204 | 18:05:54,158 | 192.168.1.197 | 9335372 Request complete | 18:05:54,158 | 192.168.1.197 | 9335592
- 108. … Merging data from memtables and 8 sstables | 18:05:44,892 | 192.168.1.204 | 38605 Read 1 live and 2667 tombstoned cells | 18:05:54,135 | 192.168.1.204 | 9282428 Enqueuing response to /192.168.1.197 | 18:05:54,136 | 192.168.1.204 | 9283423 Sending message to /192.168.1.197 | 18:05:54,138 | 192.168.1.204 | 9284753 Message received from /192.168.1.204 | 18:05:54,155 | 192.168.1.197 | 9332505 Processing response from /192.168.1.204 | 18:05:54,158 | 192.168.1.197 | 9335372 Request complete | 18:05:54,158 | 192.168.1.197 | 9335592 !!?!
- 109. … Merging data from memtables and 8 sstables | 18:05:44,892 | 192.168.1.204 | 38605 Read 1 live and 2667 tombstoned cells | 18:05:54,135 | 192.168.1.204 | 9282428 Enqueuing response to /192.168.1.197 | 18:05:54,136 | 192.168.1.204 | 9283423 Sending message to /192.168.1.197 | 18:05:54,138 | 192.168.1.204 | 9284753 Message received from /192.168.1.204 | 18:05:54,155 | 192.168.1.197 | 9332505 Processing response from /192.168.1.204 | 18:05:54,158 | 192.168.1.197 | 9335372 Request complete | 18:05:54,158 | 192.168.1.197 | 9335592
- 110. Writing Code -Tracing: Enable traces in the driver http://www.datastax.com/documentation/developer/java-driver/2.0/java-driver/tracing_t.html
- 111. Writing Code -Tracing: `nodetool settraceprobability`
- 112. Writing Code -Tracing: …then make sure you try it again with a node down!
- 113. Writing Code -Tracing: Final note on tracing: do it sparingly
- 114. Writing Code -Tracing: Enable query latency logging https://github.com/datastax/java-driver/tree/2.1/features/logging
- 115. Writing Code: LoggingVerbosity can be changed dynamically** ** since 0.4rc1 http://www.datastax.com/documentation/cassandra/2.0/cassandra/configuration/configLoggingLevels_r.html
- 116. Writing Code: nodetool for developers • cfstats • cfshistograms • proxyhistograms
- 117. Writing Code - nodetool - cfstats: cfstats: per-table statistics about size and performance (single most useful command)
- 118. Writing Code - nodetool - cfhistograms: cfhistograms: column count and partition size vs. latency distribution
- 119. Writing Code - nodetool - proxyhistograms: proxyhistograms: performance of inter-cluster requests
- 120. OVERVIEW DATA MODELING WRITING CODE TESTING REVIEWING MANAGING ENVIRONMENTS
- 121. Managing Environments: Configuration Management is Essential
- 122. Managing Environments: Laptop to Production with NO Manual Modifications!
- 123. Managing Environments: Running Cassandra during development
- 124. Managing Environments - Running Cassandra: Local Cassandra • easy to setup • you control it • but then you control it!
- 125. Managing Environments - Running Cassandra: CCM • supports multiple versions • clusters and datacenters • up/down individual nodes https://github.com/pcmanus/ccm
- 126. Managing Environments - Running Cassandra: Docker: • Official image available with excellent docs* • Docker Compose for more granular control** *https://hub.docker.com/_/cassandra/ **https://docs.docker.com/compose/
- 127. Managing Environments - Running Cassandra: Vagrant • isolated, controlled environment • configuration mgmt integration • same CM for production! http://www.vagrantup.com/
- 128. server_count = 3 network = '192.168.2.' first_ip = 10 servers = [] seeds = [] cassandra_tokens = [] (0..server_count-1).each do |i| name = 'node' + (i + 1).to_s ip = network + (first_ip + i).to_s seeds << ip servers << {'name' => name, 'ip' => ip, 'initial_token' => (2**64 / server_count * i) - 2**63} end
- 129. server_count = 3 network = '192.168.2.' first_ip = 10 servers = [] seeds = [] cassandra_tokens = [] (0..server_count-1).each do |i| name = 'node' + (i + 1).to_s ip = network + (first_ip + i).to_s seeds << ip servers << {'name' => name, 'ip' => ip, 'initial_token' => (2**64 / server_count * i) - 2**63} end
- 130. server_count = 3 network = '192.168.2.' first_ip = 10 servers = [] seeds = [] cassandra_tokens = [] (0..server_count-1).each do |i| name = 'node' + (i + 1).to_s ip = network + (first_ip + i).to_s seeds << ip servers << {'name' => name, 'ip' => ip, 'initial_token' => (2**64 / server_count * i) - 2**63} end
- 131. chef.json = { :cassandra => {'cluster_name' => 'VerifyCluster', 'version' => '2.0.8', 'setup_jna' => false, 'max_heap_size' => '512M', 'heap_new_size' => '100M', 'initial_token' => server['initial_token'], 'seeds' => "192.168.2.10", 'listen_address' => server['ip'], 'broadcast_address' => server['ip'], 'rpc_address' => server['ip'], 'conconcurrent_reads' => "2", 'concurrent_writes' => "2", 'memtable_flush_queue_size' => "2", 'compaction_throughput_mb_per_sec' => "8", 'key_cache_size_in_mb' => "4", 'key_cache_save_period' => "0", 'native_transport_min_threads' => "2", 'native_transport_max_threads' => "4" }, }
- 132. Managing Environments - Running Cassandra: Mesos? Compelling features, but not quite there (though it won't be long) http://mesosphere.github.io/cassandra-mesos/docs/ http://www.datastax.com/2015/08/a-match-made-in-heaven-cassandra-and-mesos
- 133. Summary: • Cluster-level defaults, override in queries • Follow existing patterns (it's not that different) • Segment your tests and use build profiles • Monitor and Instrument • Use reference implementation drivers • Control your environments • Verify any assumptions about failures
- 134. Thanks.
- 135. Nate McCall @zznate Co-Founder & Sr.Technical Consultant www.thelastpickle.com #CassandraDays