C* Summit 2013: Time for a New Relationship - Intuit's Journey from RDBMS to Cassandra by Mohit Anchlia

Jun 18, 20131 like29,291 views

This session talks about Intuit’s journey of our Consumer Financial Platform that is built to scale to petabytes of data. The original system used a major RDBMS and from there, we redesigned to use the distributed nature of Cassandra. This talk will go through our transition including the data model used for the final product. As with any large system transition, many hard lessons are learned and we will discuss those and share our experiences.

Intuit Proprietary & Confidential
people
The Consumer Financial Platform (CFP)
Mohit Anchlia
Architect, Intuit

Intuit Proprietary & Confidential
Agenda
2
•  Background
•  Problem statement
•  Idea of a Platform
•  Why Cassandra?
•  CFP Stack
•  CFP Cassandra Data Model
•  Learning in Production
•  Q&A

Intuit Proprietary & Confidential
Background
3
•  Intuit is maker of TurboTax, Quicken, Quickbooks and many other products
for SBUs.
•  Many services work together to deliver awesome product experience

Intuit Proprietary & Confidential
Problem Statement (Service explosion)
4
•  Service explosion over the years
–  Code duplication
–  Cross cutting concern
–  Data silos (information silos)
–  Operational challenges - schema design, installs
–  Added overhead to test and repeat test in production –
slow prototyping

Intuit Proprietary & Confidential
5
Idea of a Platform
•  Brings information together
to avoid data silos
•  Quick turnaround time
•  Plug and play service
framework
•  Don’t need IT and
operations
•  Highly personalized
experience
•  Security
•  Share data between
products, between
users
to plug ‘n’
play

Intuit Proprietary & Confidential
Data Platform/Tier
6
•  Principles – Highly Available, Highly Scalable, Fast, Easy to operate
software only solution for structured and unstructured data (blobs)
•  Projection – Petabyte in 2-3 yrs
•  Support – Critical application with 99.99%(5 nines) SLA
•  But Wait …No Stress

Intuit Proprietary & Confidential
Traditional RDBMS?
7
•  Challenges with availability and
scalability
•  Sharding works well, but introduces new challenges as well

Intuit Proprietary & Confidential
NoSQL?
8
•  Easy?
•  Core use cases – Most of the use cases don’t need transactions and with
good design, consistency can be managed properly.
•  Evaluated Hbase, MongoDB and Cassandra.

Intuit Proprietary & Confidential
Why Cassandra?
9
•  Scalable
–  Easy to scale horizontally
•  Availability
–  Highly Available, can be designed for no SPOF
–  Easy to setup clusters and replication between DC
–  Fast snapshots
–  Rolling upgrades
•  Operations
–  Easy to install and operate
–  Easy to make schema changes
•  Fast
–  Given the right hardware, Cassandra provides low latency response times.

Intuit Proprietary & Confidential
High Level CFP Stack
10
Data Platform
Services Platform
Mule ESB
Queue Service Cache service
Cassandra
RedHat Storage
(DFS)
Analytics Platform
Mule ESB
(services)
Mule ESB
HBase Hadoop Search Engine MPP
Flume
•  MuleSoft ESB for
business logic
orchestration, with
frameworks for
additional
authoring
Cassandra-powered
schemaless database
wrapped in entity and
relationship logic.
RHS – a distributed
file system for blob
storage
Hadoop/Hbase/Solr/
CEP-to meet batch
processing and near
real time analytics

Intuit Proprietary & Confidential
CFP Active/Active Multi-Data Center
11
Data Platform
Services Platform
Cassandra
RedHat Storage
(DFS)
Analytics Platform
Hadoop
Mule
Data Platform
Services Platform
Cassandra
RedHat Storage
(DFS)
Analytics Platform
Hadoop
Mule
Replication
Replication
Replication
Load
Balancer
Load
Balancer
Global Load
Balancer
•  30mt Session
stickiness
•  Provides HA
•  Low Latency
DC-A DC-B

Intuit Proprietary & Confidential
CFP Schema
12
•  Represented as a graph
–  Entity
–  Relationships
•  Additional CF for indexes
–  Inverted Indexes driven by schema
Entity User
Entity
Document
Index CF

Intuit Proprietary & Confidential
Learning in Production
13
•  Monitor Heap Usage
–  High and uneven CPU usage
–  Add nodes if you can
–  Reduce Bloom Filters
–  Increase heap if you have to, don’t be scared
Before After
•  Monitor Data per Node – Most importantly keys per node
•  Monitor disk IO

Intuit Proprietary & Confidential
The End
14
We are hiring.
Contact @ mohit_anchlia@intuit.com

1. Apache Ambari is an open-source platform for provisioning, managing and monitoring Hadoop clusters. 2. New features in Ambari 2.4 include additional services, role-based access control, management packs and a Grafana UI for visualizing metrics. 3. Ambari simplifies cluster operations through deploying clusters via blueprints, automated Kerberos integration, host discovery and stack advisors. It also supports upgrading clusters with either rolling or express upgrades.

Near-realtime analytics with Kafka and HBasedave_revell

Managing multi tenant resource toward Hive 2.0Kai Sasaki

This document discusses Treasure Data's migration architecture for managing resources across multiple clusters when upgrading from Hive 1.x to Hive 2.0. It introduces components like PerfectQueue and Plazma that enable blue-green deployment without downtime. It also describes how automatic testing and validation is done to prevent performance degradation. Resource management is discussed to define resources per account across different job queues and Hadoop clusters. Brief performance comparisons show improvements from Hive 2.x features like Tez and vectorization.

Apache Kafka at LinkedInGuozhang Wang

The document discusses Apache Kafka, a distributed publish-subscribe messaging system developed at LinkedIn. It describes how LinkedIn uses Kafka to integrate large amounts of user activity and other data across its products. Key aspects of Kafka's design allow it to scale to LinkedIn's high throughput requirements, including using a log structure and data partitioning for parallelism. LinkedIn relies on Kafka to transport over 500 billion messages per day between systems and for real-time analytics.

Architecture of a Kafka camus infrastructuremattlieber

This document summarizes the results of a performance evaluation of Kafka and Camus to ingest streaming data into Hadoop. It finds that Kafka can ingest data at rates from 15,000-50,000 messages per second depending on data format (Avro is fastest). Camus can move the data to HDFS at rates from 54,000-662,000 records per second. Once in HDFS, queries on Avro-formatted data are fastest, with count and max aggregation queries completing in under 100 seconds for 20 million records. The customer's goal of 5000 events per second can be easily achieved with this architecture.

October 2016 HUG: Architecture of an Open Source RDBMS powered by HBase and ...Yahoo Developer Network

Splice Machine is an open-source database that combines the benefits of modern lambda architectures with the full expressiveness of ANSI-SQL. Like lambda architectures, it employs separate compute engines for different workloads - some call this an HTAP database (Hybrid Transactional and Analytical Platform). This talk describes the architecture and implementation of Splice Machine V2.0. The system is powered by a sharded key-value store for fast short reads and writes, and short range scans (Apache HBase) and an in-memory, cluster data flow engine for analytics (Apache Spark). It differs from most other clustered SQL systems such as Impala, SparkSQL, and Hive because it combines analytical processing with a distributed Multi-Value Concurrency Method that provides fine-grained concurrency which is required to power real-time applications. This talk will highlight the Splice Machine storage representation, transaction engine, cost-based optimizer, and present the detailed execution of operational queries on HBase, and the detailed execution of analytical queries on Spark. We will compare and contrast how Splice Machine executes queries with other HTAP systems such as Apache Phoenix and Apache Trafodian. We will end with some roadmap items under development involving new row-based and column-based storage encodings. Speakers: Monte Zweben, is a technology industry veteran. Monte’s early career was spent with the NASA Ames Research Center as the Deputy Chief of the Artificial Intelligence Branch, where he won the prestigious Space Act Award for his work on the Space Shuttle program. He then founded and was the Chairman and CEO of Red Pepper Software, a leading supply chain optimization company, which merged in 1996 with PeopleSoft, where he was VP and General Manager, Manufacturing Business Unit. In 1998, he was the founder and CEO of Blue Martini Software – the leader in e-commerce and multi-channel systems for retailers. Blue Martini went public on NASDAQ in one of the most successful IPOs of 2000, and is now part of JDA. Following Blue Martini, he was the chairman of SeeSaw Networks, a digital, place-based media company. Monte is also the co-author of Intelligent Scheduling and has published articles in the Harvard Business Review and various computer science journals and conference proceedings. He currently serves on the Board of Directors of Rocket Fuel Inc. as well as the Dean’s Advisory Board for Carnegie-Mellon’s School of Computer Science.

January 2015 HUG: Using HBase Co-Processors to Build a Distributed, Transacti...Yahoo Developer Network

Monte Zweben Co-Founder and CEO of Splice Machine, will discuss how to use HBase co-processors to build an ANSI-99 SQL database with 1) parallelization of SQL execution plans, 2) ACID transactions with snapshot isolation and 3) consistent secondary indexing. Transactions are critical in traditional RDBMSs because they ensure reliable updates across multiple rows and tables. Most operational applications require transactions, but even analytics systems use transactions to reliably update secondary indexes after a record insert or update. In the Hadoop ecosystem, HBase is a key-value store with real-time updates, but it does not have multi-row, multi-table transactions, secondary indexes or a robust query language like SQL. Combining SQL with a full transactional model over HBase opens a whole new set of OLTP and OLAP use cases for Hadoop that was traditionally reserved for RDBMSs like MySQL or Oracle. However, a transactional HBase system has the advantage of scaling out with commodity servers, leading to a 5x-10x cost savings over traditional databases like MySQL or Oracle. HBase co-processors, introduced in release 0.92, provide a flexible and high-performance framework to extend HBase. In this talk, we show how we used HBase co-processors to support a full ANSI SQL RDBMS without modifying the core HBase source. We will discuss how endpoint transactions are used to serialize SQL execution plans over to regions so that computation is local to where the data is stored. Additionally, we will show how observer co-processors simultaneously support both transactions and secondary indexing. The talk will also discuss how Splice Machine extended the work of Google Percolator, Yahoo Labs’ OMID, and the University of Waterloo on distributed snapshot isolation for transactions. Lastly, performance benchmarks will be provided, including full TPC-C and TPC-H results that show how Hadoop/HBase can be a replacement of traditional RDBMS solutions.

Low latency high throughput streaming using Apache Apex and Apache KuduDataWorks Summit

True streaming is fast becoming a necessity for many business use cases. On the other hand the data set sizes and volumes are also growing exponentially compounding the complexity of data processing pipelines.There exists a need for true low latency streaming coupled with very high throughput data processing. Apache Apex as a low latency and high throughput data processing framework and Apache Kudu as a high throughput store form a nice combination which solves this pattern very efficiently. This session will walk through a use case which involves writing a high throughput stream using Apache Kafka,Apache Apex and Apache Kudu. The session will start with a general overview of Apache Apex and capabilities of Apex that form the foundation for a low latency and high throughput engine with Apache kafka being an example input source of streams. Subsequently we walk through Kudu integration with Apex by walking through various patterns like end to end exactly once, selective column writes and timestamp propagations for out of band data. The session will also cover additional patterns that this integration will cover for enterprise level data processing pipelines. The session will conclude with some metrics for latency and throughput numbers for the use case that is presented. Speaker Ananth Gundabattula, Senior Architect, Commonwealth Bank of Australia

Stream your Operational Data with Apache Spark & Kafka into Hadoop using Couc...Data Con LA

Abstract:- Tracking user events as they happen can challenge anyone providing real time user interaction. It can demand both huge scale and a lot of processing to support dynamic adjustment to targeting products and services. As the operational data store Couchbase data services are capable of processing tens of millions of updates a day. Streaming through systems such as Apache Spark and Kafka into Hadoop, information about these key events can be turned into deeper knowledge. We will review Lambda architectures deployed at sites like PayPal, Live Person and LinkedIn that leverage a Couchbase Data Pipeline. Bio:- Justin Michaels. With over 20 years experience in deploying mission critical systems, Justin Michaels industry experience covers capacity planning, architecture and industry vertical experience. Justin brings his passion for architecting, implementing and improving Couchbase to the community as a Solution Architect. His expertise involves both conventional application platforms as well as distributed data management systems. He regularly engages with existing and new Couchbase customers in performance reviews, architecture planning and best practice guidance.

Hive vs. ImpalaOmid Vahdaty

Hive and Impala are tools for querying data stored in Hadoop, but they have key differences. Hive uses MapReduce to transform SQL queries into jobs and is better for long-running ETL processes due to fault tolerance. Impala is a massively parallel processing database that pushes processing directly to nodes, making it faster and more suitable for interactive queries from data analysts. The main differences are that Hive uses disk-based operations while Impala keeps data and calculations in memory, and Hive provides fault tolerance by restarting failed queries while Impala would need to restart from the beginning.

Kafka to the Maxka - (Kafka Performance Tuning)DataWorks Summit

Kafka is becoming an ever more popular choice for users to help enable fast data and Streaming. Kafka provides a wide landscape of configuration to allow you to tweak its performance profile. Understanding the internals of Kafka is critical for picking your ideal configuration. Depending on your use case and data needs, different settings will perform very differently. Lets walk through performance essentials of Kafka. Let's talk about how your Consumer configuration, can speed up or slow down the flow of messages to Brokers. Lets talk about message keys, their implications and their impact on partition performance. Lets talk about how to figure out how many partitions and how many Brokers you should have. Let's discuss consumers and what effects their performance. How do you combine all of these choices and develop the best strategy moving forward? How do you test performance of Kafka? I will attempt a live demo with the help of Zeppelin to show in real time how to tune for performance.

Apache kafkaShravan (Sean) Pabba

Data Architectures for Robust Decision MakingGwen (Chen) Shapira

The document discusses designing robust data architectures for decision making. It advocates for building architectures that can easily add new data sources, improve and expand analytics, standardize metadata and storage for easy data access, discover and recover from mistakes. The key aspects discussed are using Kafka as a data bus to decouple pipelines, retaining all data for recovery and experimentation, treating the filesystem as a database by storing intermediate data, leveraging Spark and Spark Streaming for batch and stream processing, and maintaining schemas for integration and evolution of the system.

Kudu - Fast Analytics on Fast DataRyan Bosshart

This document discusses Apache Kudu, an open source column-oriented storage system that provides fast analytics on fast data. It describes Kudu's design goals of high throughput for large scans, low latency for short accesses, and database-like semantics. The document outlines Kudu's architecture, including its use of columnar storage, replication for fault tolerance, and integrations with Spark, Impala and other frameworks. It provides examples of using Kudu for IoT and real-time analytics use cases. Performance comparisons show Kudu outperforming other NoSQL systems on analytics and operational workloads.

HBaseCon 2013:High-Throughput, Transactional Stream Processing on Apache HBase Cloudera, Inc.

This document discusses high-throughput transactional stream processing on HBase using Continuuity Reactor. Continuuity Reactor is a platform built on Hadoop and HBase that allows collecting, processing, storing and querying data with ACID guarantees using a flow-based programming model. Flows in Reactor are composed of flowlets connected by queues, with all processing occurring within transactions for consistency. The document outlines how Reactor implements transactions using an optimistic concurrency control approach based on multi-version concurrency control and HBase timestamps. It also discusses queue design and optimizations for performance.

Emerging technologies /frameworks in Big DataRahul Jain

Hive on kafkaSzehon Ho

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...Data Con LA

1) Apache Kudu is a new updatable columnar storage engine for Apache Hadoop that facilitates fast analytics on fast data. 2) Kudu is designed to address gaps in the current Hadoop storage landscape by providing both high throughput for big scans and low latency for short accesses simultaneously. 3) Kudu integrates with various Hadoop components like Spark, Impala, MapReduce to enable SQL queries and other analytics workloads on fast updating data.

Building Effective Near-Real-Time Analytics with Spark Streaming and KuduJeremy Beard

This document discusses building near-real-time analytics pipelines using Apache Spark Streaming and Apache Kudu on the Cloudera platform. It defines near-real-time analytics, describes the relevant components of the Cloudera stack (Kafka, Spark, Kudu, Impala), and how they can work together. The document then outlines the typical stages involved in implementing a Spark Streaming to Kudu pipeline, including sourcing from a queue, translating data, deriving storage records, planning mutations, and storing the data. It provides performance considerations and introduces Envelope, a Spark Streaming application on Cloudera Labs that implements these stages through configurable pipelines.

HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster Cloudera, Inc.

This document discusses implementing an HBase coprocessor to index columns from HBase into an Elasticsearch cluster. It describes storing book records from publishers and libraries in HBase and indexing them into Elasticsearch using MapReduce jobs. To handle updates, a coprocessor uses HBase's checkAndPut method to verify the record version before updating and indexing the new version into Elasticsearch, ensuring consistency between the two systems.

Apache Flink & Kudu: a connector to develop Kappa architecturesNacho García Fernández

Kappa Architecture is a software architecture pattern that makes use of an immutable, append only log. All the processing of the event will be performed in the input streams and persisted as real-time views. Apache Flink is very well suited to be the processing engine because it provides support for event-time semantics, stateful exactly-once processing, and achieves high throughput and low latency at the same time. Apache Kudu Kudu is a storage system good at both ingesting streaming data and analysing it using ad-hoc queries (e.g. interactive SQL based) and full-scan processes (e.g Spark/Flink). So Kudu is a good fit to store the real-time views in a Kappa Architecture. We have developed and open-sourced a connector to integrate Apache Kudu and Apache Flink. It allows reading/writing data from/to Kudu using the DataSet and DataStream Flink’s APIs. The connector has been submitted to the Apache Bahir project and is already available from maven central repository.

HBaseCon 2013: Apache HBase, Meet Ops. Ops, Meet Apache HBase.Cloudera, Inc.

This document provides an overview of operating HBase clusters in production environments. It discusses leveraging existing knowledge of distributed systems, getting metrics set up using tools like Ganglia and OpenTSDB, automating tasks with Puppet, Chef and Fabric, setting up alerting with Nagios and Zabbix, and different backup strategies for HBase including offline distcp backups, replication to another cluster, and using HBase snapshots. The goals are to help operations teams understand how to manage HBase and empower them to work with their own operations organizations.

Harmonizing Multi-tenant HBase Clusters for Managing Workload DiversityHBaseCon

Speakers: Dheeraj Kapur, Rajiv Chittajallu & Anish Mathew (Yahoo!) In early 2013, Yahoo! introduced multi-tenancy to HBase to offer it as a platform service for all Hadoop users. A certain degree of customization per tenant (a user or a project) was achieved through RegionServer groups, namespaces, and customized configs for each tenant. This talk covers how to accommodate diverse needs to individual tenants on the cluster, as well as operational tips and techniques that allow Yahoo! to automate the management of multi-tenant clusters at petabyte scale without errors.

Spark on MesosJen Aman

This document summarizes a talk on Spark on Mesos given by Dean Wampler from Lightbend and Timothy Chen from Mirantis. It discusses why Spark on Mesos is useful by allowing one cluster system to run multiple tools. It then covers recent updates like a new integration test suite, coarse-grained scheduler improvements, and Mesos framework authentication. Finally, it outlines future plans such as GPU support on Mesos and making "production" use of Spark on Mesos easier.

HBaseCon 2012 | Building a Large Search Platform on a Shoestring BudgetCloudera, Inc.

This document discusses YapMap, a visual search platform built on Hadoop and HBase. It summarizes how YapMap interfaces with HBase data, uses HBase as a data processing pipeline with checkpoints, and had to adjust schemas and migrate data as the system evolved. It also covers how YapMap constructs search indexes in shards based on HBase regions and stored indexes on HDFS. The document concludes with some lessons learned around optimizing HBase operations.

Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsightHBaseCon

Nitin Verma, Pravin Mittal, and Maxim Lukiyanov (Microsoft) This session presents our success story of enabling a big internal customer on Microsoft Azure’s HBase service along with the methodology and tools used to meet high-throughput goals. We will also present how new features in HBase (like BucketCache and MultiWAL) are helping our customers in the medium-latency/high-bandwidth cloud-storage scenario.

Cassandra & puppet, scaling data at $15 per monthdaveconnors

Migrating Netflix from Datacenter Oracle to Global CassandraAdrian Cockcroft

Netflix is migrating its datacenter infrastructure from Oracle databases to a globally distributed Apache Cassandra database on AWS. This will allow Netflix to scale more easily and deploy new features faster without being limited by the capacity of its own datacenters. The migration involves transitionally replicating data between Oracle and AWS services like SimpleDB while new services are deployed directly on Cassandra. This will cut Netflix's dependence on its existing datacenters and allow it to fully leverage the elasticity of the public cloud.

More Related Content

What's hot (20)

January 2015 HUG: Using HBase Co-Processors to Build a Distributed, Transacti...Yahoo Developer Network

Low latency high throughput streaming using Apache Apex and Apache KuduDataWorks Summit

Stream your Operational Data with Apache Spark & Kafka into Hadoop using Couc...Data Con LA

Hive vs. ImpalaOmid Vahdaty

Kafka to the Maxka - (Kafka Performance Tuning)DataWorks Summit

Apache kafkaShravan (Sean) Pabba

Data Architectures for Robust Decision MakingGwen (Chen) Shapira

Kudu - Fast Analytics on Fast DataRyan Bosshart

HBaseCon 2013:High-Throughput, Transactional Stream Processing on Apache HBase Cloudera, Inc.

Emerging technologies /frameworks in Big DataRahul Jain

Hive on kafkaSzehon Ho

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...Data Con LA

Building Effective Near-Real-Time Analytics with Spark Streaming and KuduJeremy Beard

HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster Cloudera, Inc.

Apache Flink & Kudu: a connector to develop Kappa architecturesNacho García Fernández

HBaseCon 2013: Apache HBase, Meet Ops. Ops, Meet Apache HBase.Cloudera, Inc.

Harmonizing Multi-tenant HBase Clusters for Managing Workload DiversityHBaseCon

Spark on MesosJen Aman

HBaseCon 2012 | Building a Large Search Platform on a Shoestring BudgetCloudera, Inc.

Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsightHBaseCon

January 2015 HUG: Using HBase Co-Processors to Build a Distributed, Transacti...Yahoo Developer Network

Low latency high throughput streaming using Apache Apex and Apache KuduDataWorks Summit

Stream your Operational Data with Apache Spark & Kafka into Hadoop using Couc...Data Con LA

Hive vs. ImpalaOmid Vahdaty

Kafka to the Maxka - (Kafka Performance Tuning)DataWorks Summit

Apache kafkaShravan (Sean) Pabba

Data Architectures for Robust Decision MakingGwen (Chen) Shapira

Kudu - Fast Analytics on Fast DataRyan Bosshart

HBaseCon 2013:High-Throughput, Transactional Stream Processing on Apache HBase Cloudera, Inc.

Emerging technologies /frameworks in Big DataRahul Jain

Hive on kafkaSzehon Ho

Big Data Day LA 2016/ NoSQL track - Apache Kudu: Fast Analytics on Fast Data,...Data Con LA

Building Effective Near-Real-Time Analytics with Spark Streaming and KuduJeremy Beard

HBaseCon 2013: Using Coprocessors to Index Columns in an Elasticsearch Cluster Cloudera, Inc.

Apache Flink & Kudu: a connector to develop Kappa architecturesNacho García Fernández

HBaseCon 2013: Apache HBase, Meet Ops. Ops, Meet Apache HBase.Cloudera, Inc.

Harmonizing Multi-tenant HBase Clusters for Managing Workload DiversityHBaseCon

Spark on MesosJen Aman

HBaseCon 2012 | Building a Large Search Platform on a Shoestring BudgetCloudera, Inc.

Optimizing Apache HBase for Cloud Storage in Microsoft Azure HDInsightHBaseCon

Viewers also liked (20)

Cassandra & puppet, scaling data at $15 per monthdaveconnors

Migrating Netflix from Datacenter Oracle to Global CassandraAdrian Cockcroft

NYC* 2013 — "Using Cassandra for DVR Scheduling at Comcast"DataStax Academy

Comcast is developing a highly scalable cloud DVR scheduling system on top of Cassandra. The system is responsible for managing all DVR data and scheduling logic for devices on the X1 platform. This talk will cover the overall architecture of the scheduling system, data model, message queue and notification software that have been developed as part of this ambitious project. We'll take a deep dive into the details of our data model and review the implementation of Comcast's open-source, Cassandra-based clones of Amazon SQS and SNS.

Blueflood: Open Source Metrics Processing at CassandraEU 2013gdusbabek

The document discusses Blueflood, a system for processing simple metrics. It handles 120 million metrics per hour and 40 million Cassandra write operations per hour. It ingests metrics, performs rollups to aggregate data, and allows querying data. The system is scalable, processes metrics in real-time, and stores data in Cassandra for querying and rollups. It uses libraries for ingestion, rollups and querying and a scheduler to determine when to perform rollups.

Best Practices for couchDB developers on Microsoft AzureBrian Benz

Flash Economics and Lessons learned from operating low latency platforms at h...Aerospike, Inc.

The document discusses requirements for internet enterprises, including responding to interactions in real-time, determining user intent based on context, responding immediately using big data, and ensuring systems never go down. It then discusses Aerospike's in-memory database capabilities for handling high transaction volumes with low latency and unlimited scalability. Finally, it outlines lessons learned from operating high performance systems, including keeping architectures simple, automating operations, and separating online and offline workloads.

Cassandra Summit - What's New In Apache TinkerPop?Stephen Mallette

Kai – An Open Source Implementation of Amazon’s DynamoTakeru INOUE

This document provides an overview of Amazon's Dynamo database and Kai, an open source implementation of Dynamo. It describes Dynamo's motivation as providing scalable, fault-tolerant storage needed by Amazon's large-scale systems. Key features of Dynamo discussed include its use as a distributed key-value store, high availability even during failures, and eventual consistency. The document also outlines Dynamo's algorithms for partitioning, replication, and spreading membership changes using a gossip protocol. Kai is presented as a way to make these Dynamo capabilities openly available.

Descargando blobs desde el blob storage Víctor Moreno

Este documento explica cómo crear una cuenta de almacenamiento en Azure y descargar blobs desde el almacenamiento de blobs. Primero, se debe crear una cuenta de almacenamiento en Azure a través del portal. Luego, se generan las claves de acceso de la cuenta. Finalmente, se agrega código a un proyecto de Visual Studio para descargar blobs específicos desde la cuenta de almacenamiento utilizando las claves de acceso.

Copiando blobs desde el blob storage Víctor Moreno

Configuración de escritorio remoto en windows azureVíctor Moreno

Azure tip crea y elimina bases de datos sql azure desde códigoVíctor Moreno

Como configurar visual svn server desde azureVíctor Moreno

Este documento explica cómo instalar y configurar Visual SVN Server en una máquina virtual de Azure para usarlo como repositorio de archivos, y cómo instalar Visual SVN Client y Tortoise SVN en un equipo local para conectarse al repositorio remoto. Primero se crea una máquina virtual con Windows Server 2012, luego se descargan e instalan Visual SVN Server, Visual SVN Client y Tortoise SVN para configurar el control de versiones entre el equipo local y la máquina virtual de Azure.

24 ejecutando ie con azure remoteappVíctor Moreno

Este documento explica cómo ejecutar Internet Explorer desde un dispositivo Android utilizando Azure RemoteApp. Primero, el usuario debe iniciar sesión en remote.modern.ie y seleccionar su región. Luego, debe descargar e instalar la aplicación Microsoft Remote Desktop en su dispositivo Android. Una vez iniciada la sesión, el usuario puede acceder a IE como si estuviera instalado localmente en su dispositivo.

Como escalar una máquina virtualVíctor Moreno

Agregando y actualizando entidades desde el table storage Víctor Moreno

Crear un sitio web joomlaVíctor Moreno

Almacenando con blobs en el blob storage Víctor Moreno

Como ejecutar aplicaciones desde cualquier plataformaVíctor Moreno

Eliminando entidades de una tabla desde el table storage Víctor Moreno