AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore

Oct 2, 20150 likes1,768 views

At AdStage we have a large volume of data about ads and their relationships: campaigns, ad groups, keywords, bids, budgets, targeting info - the list goes on. We started out storing all this data in Postgres, but even before we reached public beta we were already putting too much strain on the largest Postgres instance we could run. We considered sharding, but instead we decided to embark on a project to store our data in Cassandra. Now, after more than a year of development, we present Monacella, a relational object database that uses Cassandra as the datastore. In this talk we'll examine the architecture of Monacella, its features and use cases, and plans for future development.

Monacella: An Object-Relational Database

Origin Story
• need to import ad network data
• highly parallelized ETL pipeline
• hit Postgres connection limits during beta
• considered sharding, but didn't have ops experience for it
• instead decided to try Cassandra because
• we already had ops experience with it
• low maintenance
• complexity would be in app layer where we were experts
© 2015. All Rights Reserved. 2

Naive Architecture
© 2015. All Rights Reserved. 3
campaign_id:
string
account_id:
string
created_at:
timestamp
data:
string:string
/network/adwords/ad/1234
ads campaigns accounts
secondary
indexes
ad_id:
string

Sensible Architecture
© 2015. All Rights Reserved. 4
filter_name:
string
shard_id:
integer
entity_id:
string
none | active | inactive | ...
ads campaigns accounts
secondary
indexes
stored_filters
...
facet_id:
string
...

Oops! I Made an Object-Relational Database
• at first didn't know what I had, but then slowly realized:
• rows are objects with types/classes
• objects have type-based relationships to other objects
• support for object relationships but not arbitrary joins
• who would want this?
• need horizontal scalability
• don't need all the features of a relational database...
• ...but do need object relationships
© 2015. All Rights Reserved. 5

Monacella and the Future
• build out as a database with Cassandra as the datastore
• simplify data model
• stored filter optimizations
• change datastore to Dynomite over Redis or Postgres
• Doradus
• Spider does the same thing as Monacella but maybe better
• and it has an OLAP layer - something we also built!
• sharded Postgres?
© 2015. All Rights Reserved. 6

The document discusses building a system for processing machine and event-oriented data in real-time. It describes the high-level architecture which involves data acquisition, processing, storage and querying. Events are modeled and transformed through stream processing jobs. Metrics and time series data are aggregated. Challenges include dealing with distributed systems issues, data quality, and immaturity of stream processing technologies.

End of the Myth: Ultra-Scalable Transactional Management by Ricardo Jiménez-P...Big Data Spain

Druid + Kafka: transform your data-in-motion to analytics-in-motion | Gian Me...HostedbyConfluent

Apache Druid is a high-performance distributed analytics store for modern analytics applications. It supports ingesting millions of events per second and sub-second query processing. Druid supports various types of data sources for ingestion, including Apache Kafka. You can immediately query on stream events once they get ingested into Druid. Since Kafka provides scalable and robust data delivery while Druid supports advanced complex analysis on streams, Kafka and Druid are widely used together for BI and operational analytics use cases, which require interactivity, scalability, real-time, and performance. This talk is based on our real-world experiences building out streaming analytics stacks powering production use cases across many industries.

Lessons From HPE: From Batch To Streaming For 20 Billion Sensors With Lightbe...Lightbend

Scaling a backend for a big data and blockchain environment by Rafael Ríos at...Big Data Spain

This document discusses scaling the backend of a financial platform for big data and blockchain. It describes challenges integrating big data using Apache Spark and Cassandra for tasks like predictive modeling, recommendations, and credit scoring. It also covers using a microservices architecture with Spring Cloud, Docker, and Kubernetes for deployment. Blockchain integration involves a private Ethereum network on Kubernetes for tokenization and a connection to the public Ethereum mainnet using Infura for payments and transfers.

Javantura v3 - Microservice – no fluff the REAL stuff – Nakul MishraHUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association

This document discusses microservices and provides an overview of common microservice concepts. It begins with discussing problems with monolithic architectures and then covers topics like service registration with Eureka, load balancing with Ribbon, edge services with Zuul, and failure management with Hystrix. Both pros and cons of the microservices approach are presented. The document concludes with an example demo of a microservices architecture using Spring Cloud and a request for any questions.

Machine Learning on Streaming Data using Kafka, Beam, and TensorFlow (Mikhail...confluent

Leveraging Spark to Democratize Data for Omni-Commerce with Shafaq AbdullahDatabricks

Insnap, a hyper-personalized ML-based platform acquired by The Honest Company, has been used to build a real-time data platform based on Apache Spark, Cassandra and Redshift. Users’ behavioral and transactional data have been used to build data models and ML models, and to drive use cases for marketing, growth, finance and operations. Learn how Honest Company has used Spark as a workhorse for 1) collecting, ETL and storing data from various sources including mysql, mongo, jde, Google analytics, Facebook, Localytics and REST API; 2) building data models and aggregating and generating reports of revenue, order fulfillment tracking, data pipeline monitoring and subscriptions; 3) Using ML to build model for user acquisitions, LTV and recommendations use cases. Spark replaced the monolithic codebase with flexible, scalable and robust pipelines. Databricks helped The Honest Company to focus on data instead of maintaining infrastructure. While Honest users got delightful recommendations to improve experience, data users at Honest understood users much better in terms of segmenting with behavioral information and advanced ML models, leading to increased revenue and retention.

Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks

Data Mesh is an innovative concept addressing many data challenges from an architectural, cultural, and organizational perspective. But is the world ready to implement Data Mesh? In this session, we will review the importance of core Data Mesh principles, what they can offer, and when it is a good idea to try a Data Mesh architecture. We will discuss common challenges with implementation of Data Mesh systems and focus on the role of open-source projects for it. Projects like Apache Spark can play a key part in standardized infrastructure platform implementation of Data Mesh. We will examine the landscape of useful data engineering open-source projects to utilize in several areas of a Data Mesh system in practice, along with an architectural example. We will touch on what work (culture, tools, mindset) needs to be done to ensure Data Mesh is more accessible for engineers in the industry. The audience will leave with a good understanding of the benefits of Data Mesh architecture, common challenges, and the role of Apache Spark and other open-source projects for its implementation in real systems. This session is targeted for architects, decision-makers, data-engineers, and system designers.

User Focused Security at Netflix: StethoscopeJesse Kriss

Presented by Andrew White and Jesse Kriss at ShmooCon 2017. User Focused Security is an approach we are using to address employee information security at Netflix. If we provide employees with the right information and low-friction tools, we believe they can get their devices into a more secure state without heavy-handed policy enforcement. Letting people retain control over their devices means that they can maintain flexibility and productivity and address security recommendations as appropriate to their levels of access. This approach will only be successful, though, if we can provide clear and specific action, and make it easy to do the right thing. Stethoscope is a web-based tool that gives Netflix employees a view into the security state of their devices, with specific recommendations regarding disk encryption, firewalls, and other device settings. The website, in conjunction with email alerts, gives Netflix employees a straightforward way to see what actions they should take to remain safe. Andrew White and Jesse Kriss are both members of the Information Security team at Netflix, where they work on designing and building software tools that help people make good decisions around corporate security. Andrew holds a PhD in Computer Science from the University of North Carolina at Chapel Hill and a B.S. in Computer Science and B.A. in Mathematics from the University of Richmond. Jesse (@jkriss) holds a Master’s in Human-Computer Interaction from Carnegie Mellon University and B.A. in Music from Carleton College. Prior to Netflix, he worked at NASA/JPL, Obama 2012, Figure 53, and IBM Research.

Maximize the Business Value of Machine Learning and Data Science with Kafka (...confluent

Today, many companies that have lots of data are still struggling to derive value from machine learning (ML) and data science investments. Why? Accessing the data may be difficult. Or maybe it’s poorly labeled. Or vital context is missing. Or there are questions around data integrity. Or standing up an ML service can be cumbersome and complex. At Nuuly, we offer an innovative clothing rental subscription model and are continually evolving our ML solutions to gain insight into the behaviors of our unique customer base as well as provide personalized services. In this session, I’ll share how we used event streaming with Apache Kafka® and Confluent Cloud to address many of the challenges that may be keeping your organization from maximizing the business value of machine learning and data science. First, you’ll see how we ensure that every customer interaction and its business context is collected. Next, I’ll explain how we can replay entire interaction histories using Kafka as a transport layer as well as a persistence layer and a business application processing layer. Order management, inventory management, logistics, subscription management – all of it integrates with Kafka as the common backbone. These data streams enable Nuuly to rapidly prototype and deploy dynamic ML models to support various domains, including pricing, recommendations, product similarity, and warehouse optimization. Join us and learn how Kafka can help improve machine learning and data science initiatives that may not be delivered to their full potential.

Cloudstate - Towards Stateful ServerlessLightbend

The document discusses stateful serverless computing and the CloudState project. It begins by outlining some technical requirements for building general-purpose applications in a serverless environment, including support for state management, distributed coordination, and predictable performance. It then introduces CloudState, an open-source project that aims to make stateful serverless applications easier to build by abstracting over complex distributed systems concerns like state management, databases, and infrastructure. CloudState provides client libraries in multiple languages and supports powerful state models and databases. It also handles operations when deployed as a managed service. The document concludes by describing CloudState's architecture, which uses Akka, gRPC, Kubernetes, and databases.

Big Data Day LA 2016/ Use Case Driven track - Shaping the Role of Data Scienc...Data Con LA

How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...Lightbend

Full Stack Reactive In PracticeLightbend

dotScale 2017 Keynote: The Rise of Real Time by Neha Narkhedeconfluent

Slides from Neha Narkhede's keynote at the dotScale conference in Paris on April 24th, 2017. There is a tectonic shift happening in how data powers the core of a company's business. This shift is about the rise of real-time. Apache Kafka was built with the vision to help companies navigate this change and become the central nervous system that makes data available in real-time to all the applications that need to use it. This talk is about how you can put Apache Kafka to practice to help your company make this shift to real-time. And how the Connect and Streams API in Apache Kafka capture the entire scope of what it means to put streams into practice.

INTERFACE, by apidays - Apache Cassandra now speaks developer with Stargate ...apidays

The document discusses Stargate, an open source data gateway that sits between applications and Cassandra databases. It allows Cassandra to speak to developers through REST, GraphQL, and CQL APIs accessed through SDKs and endpoints. Stargate abstracts Cassandra concepts from applications and supports multiple programming languages. Speakers from Netflix and Yelp praise how Stargate removes barriers for developers and helps unify teams by breaking data silos. The project aims to further enhance GraphQL and microservices while improving usability.

Introducing the Hub for Data OrchestrationAlluxio, Inc.

Redash: Open Source SQL Analytics on Data LakesDatabricks

How To Use Kafka and Druid to Tame Your Router Data (Rachel Pedreschi, Imply ...confluent

Do you know who is knocking on your network’s door? Have new regulations left you scratching your head on how to handle what is happening in your network? Network flow data helps answer many questions across a multitude of use cases including network security, performance, capacity planning, routing, operational troubleshooting and more. Today’s modern day streaming data pipelines need to include tools that can scale to meet the demands of these service providers while continuing to provide responsive answers to difficult questions. In addition to stream processing, data needs to be stored in a redundant, operationally focused database to provide fast, reliable answers to critical questions. Together, Kafka and Druid work together to create such a pipeline. In this talk Eric Graham and Rachel Pedreschi will discuss these pipelines and cover the following topics: -Network flow use cases and why this data is important. -Reference architectures from production systems at a major international Bank. -Why Kafka and Druid and other OSS tools for Network Flows. -A demo of one such system.

Hard Truths About Streaming and Eventing (Dan Rosanova, Microsoft) Kafka Summ...confluent

Eventing and streaming open a world of compelling new possibilities to our software and platform designs. They can reduce time to decision and action while lowering total platform cost. But they are not a panacea. Understanding the edges and limits of these architectures can help you avoid painful missteps. This talk will focus on event-driven and streaming architectures and how Apache Kafka can help you implement these. It will also discuss key tradeoffs you will face along the way from partitioning schemes to the impact of availability vs. consistency (CAP Theorem). Finally, we’ll discuss some challenges of scale for patterns like Event Sourcing and how you can use other tools and even features of Kafka to work around them. This talk assumes a basic understanding of Kafka and distributed computing but will include brief refresher sections.

Hybrid Kafka, Taking Real-time Analytics to the Business (Cody Irwin, Google ...HostedbyConfluent

Apache Kafka users who want to leverage Google Cloud Platform's (GCPs) data analytics platform and open source hosting capabilities can bridge their existing Kafka infrastructure on-premise or in other clouds to GCP using Confluent's replicator tool and managed Kafka service on GCP. Using actual customer examples and a reference architecture, we'll showcase how existing Kafka users can stream data to GCP and use it in popular tools like Apache Beam on Dataflow, BigQuery, Google Cloud Storage (GCS), Spark on Dataproc, and Tensorflow for data warehousing, data processing, data storage, and advanced analytics using AI and ML.

Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...HostedbyConfluent

This document discusses streaming data between Confluent Cloud and MongoDB Atlas. It provides an overview of MongoDB Atlas and its fully managed database capabilities in the cloud. It then demonstrates how to stream data from a Python generator application to MongoDB Atlas using Confluent Cloud and its connectors. The presentation concludes by providing a reference architecture for connecting Confluent Platform to MongoDB.

Auto-Train a Time-Series Forecast Model With AML + ADBDatabricks

Supply Chain, Healthcare, Insurance, and Finance often require highly accurate forecasting models in an enterprise large-scale fashion. With Azure Machine Learning on Azure Databricks, the scale and speed to large-scale many-models can be achieved and time-to-product decreases drastically. The better-together story poses an enterprise approach to AI/ML. Azure AutoML offers an elegant solution efficiently to build forecasting models on Azure Databricks compute solving sophisticated business problems. The presentation covers the Azure Machine Learning + Azure Databricks approach (see slides attached) while the demo covers a hands-on business problem building a forecasting model in Azure Databricks using Azure Machine Learning. The AI/ML better-together story is elevated as MLFlow for Data Science Lifecycle Management and Hyperopt for distributed model execution completes AI/ML enterprise readiness for industry problems.

Accelerate Data Science Initiatives: Databricks & PrivaceraDatabricks

Accelerating Data Science Initiatives with Databricks’ Rapid SQL Analytics and Privacera’s Centralized Data Access Governance. Databricks’ SQL Analytics helps data teams consolidate and simplify their data architectures. With SQL Analytics, data teams can perform BI and SQL workloads on the same multi-cloud lakehouse architecture enabling data scientists to perform advanced analytics on unstructured and large-scale data. This session will explore how Privacera’s advanced security, privacy, and governance capabilities seamlessly integrate with Databricks’ unified SQL Analytics approach to provide single pane visibility of data analytics from a centralized location. Attendees will learn how to: Rapidly access data to run high-fidelity analytics Implement a fully secure solution that ensures productivity, while controlling data access at fine-grained levels (row, column, and file) Easily enable consistent access policies across all systems and applications Support true data transparency across enterprises Comply with stringent industry and privacy regulations like GDPR, LGPD, HIPAA, CCPA, PCI DSS, RTBF, and more with rich auditing and reporting

Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...HostedbyConfluent

Time series data is everywhere -- connected IoT devices, application monitoring & observability platforms, and more. What makes time series datastreams challenging is that they often have orders of magnitude more data than other workloads, with millions of time series datapoints being quite common. Given its ability to ingest high volumes of data, Kafka is a natural part of any data architecture handling large volumes of time series telemetry, specifically as an intermediate buffer before that data is persisted in InfluxDB for processing, analysis, and use in other applications. In this session, we will show you how you can stream time series data to your IoT application using Kafka queues and InfluxDB, drawing upon deployments done at Hulu and Wayfair that allow both to ingest 1 million metrics per second. Once this session is complete, you’ll be able to connect a Kafka queue to an InfluxDB instance as the beginning of your own time series data pipeline.

Real time big data stream processing Luay AL-Assadi

a simple presentation about different big data stream processing systems such as SPARK, SAMZA and STORM and the difference between their architectures and purpose, in addition we talk about streaming layers tools such as Kafka and rabbitMQ, this presentation refer to this paper https://vsis-www.informatik.uni-hamburg.de/getDoc.php/publications/561/Real-time%20stream%20processing%20for%20Big%20Data.pdf and other useful links.

Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...HostedbyConfluent

A Beginner's Guide to EmberRichard Martin

Ember.js is a JavaScript web framework that allows building single-page applications. It uses conventions like MVC and handles routing, templates, controllers and models. The document provides an overview of Ember.js concepts like routers, routes, templates, controllers and models. It also lists several free and paid resources for learning Ember.js, including guides, tutorials and blogs. Core concepts are explained, including how routers handle requests, routes provide data to controllers and templates, and models define data attributes.

AALL 2015: Hands on Linked Data Tools for Catalogers: MarcEdit and MARCNextTerry Reese

MarcEdit and MARCNext provide linked data tools for catalogers to experiment with BIBFRAME and linked data concepts. The MARCNext tools allow visualization of metadata in various BIBFRAME formats and embedding URIs. The Link Identifiers tool embeds URIs from sources like VIAF, ID.LOC.GOV, and FAST in authority fields. The SPARQL Browser allows querying SPARQL endpoints. The Zepheira training plugin transforms records for the LibHub initiative. However, many linking services are still evolving and local implementations vary, posing challenges for widespread adoption.

More Related Content

What's hot (20)

Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks

User Focused Security at Netflix: StethoscopeJesse Kriss

Maximize the Business Value of Machine Learning and Data Science with Kafka (...confluent

Cloudstate - Towards Stateful ServerlessLightbend

Big Data Day LA 2016/ Use Case Driven track - Shaping the Role of Data Scienc...Data Con LA

How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...Lightbend

Full Stack Reactive In PracticeLightbend

dotScale 2017 Keynote: The Rise of Real Time by Neha Narkhedeconfluent

INTERFACE, by apidays - Apache Cassandra now speaks developer with Stargate ...apidays

Introducing the Hub for Data OrchestrationAlluxio, Inc.

Redash: Open Source SQL Analytics on Data LakesDatabricks

How To Use Kafka and Druid to Tame Your Router Data (Rachel Pedreschi, Imply ...confluent

Hard Truths About Streaming and Eventing (Dan Rosanova, Microsoft) Kafka Summ...confluent

Hybrid Kafka, Taking Real-time Analytics to the Business (Cody Irwin, Google ...HostedbyConfluent

Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...HostedbyConfluent

Auto-Train a Time-Series Forecast Model With AML + ADBDatabricks

Accelerate Data Science Initiatives: Databricks & PrivaceraDatabricks

Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...HostedbyConfluent

Real time big data stream processing Luay AL-Assadi

Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...HostedbyConfluent

Architect’s Open-Source Guide for a Data Mesh ArchitectureDatabricks

User Focused Security at Netflix: StethoscopeJesse Kriss

Maximize the Business Value of Machine Learning and Data Science with Kafka (...confluent

Cloudstate - Towards Stateful ServerlessLightbend

Big Data Day LA 2016/ Use Case Driven track - Shaping the Role of Data Scienc...Data Con LA

How to build streaming data pipelines with Akka Streams, Flink, and Spark usi...Lightbend

Full Stack Reactive In PracticeLightbend

dotScale 2017 Keynote: The Rise of Real Time by Neha Narkhedeconfluent

INTERFACE, by apidays - Apache Cassandra now speaks developer with Stargate ...apidays

Introducing the Hub for Data OrchestrationAlluxio, Inc.

Redash: Open Source SQL Analytics on Data LakesDatabricks

How To Use Kafka and Druid to Tame Your Router Data (Rachel Pedreschi, Imply ...confluent

Hard Truths About Streaming and Eventing (Dan Rosanova, Microsoft) Kafka Summ...confluent

Hybrid Kafka, Taking Real-time Analytics to the Business (Cody Irwin, Google ...HostedbyConfluent

Streaming Data in the Cloud with Confluent and MongoDB Atlas | Robert Walters...HostedbyConfluent

Auto-Train a Time-Series Forecast Model With AML + ADBDatabricks

Accelerate Data Science Initiatives: Databricks & PrivaceraDatabricks

Stream processing IoT time series data with Kafka & InfluxDB | Al Sargent, In...HostedbyConfluent

Real time big data stream processing Luay AL-Assadi

Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...HostedbyConfluent

Similar to AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore (20)

A Beginner's Guide to EmberRichard Martin

AALL 2015: Hands on Linked Data Tools for Catalogers: MarcEdit and MARCNextTerry Reese

Practical approaches to entification in library bibliographic dataTerry Reese

This document discusses practical approaches to linking library bibliographic data to external identifiers. It describes BIBFRAME as the new model for linked data and challenges around reconciling legacy data and different model flavors. Tools for experimenting are discussed, including MarcEdit's linked data tools that allow embedding URIs directly in MARC records. The document focuses on MarcEdit's MARCNext toolset and its features for testing BIBFRAME transformations and viewing linked data relationships. It addresses challenges around the readiness of linking services and local implementation of identifiers. Source code for the BIBFRAME testing plugin is also provided.

Ladies Be Architects - Integration - Multi-Org, Security, JSON, Backup & Restoregemziebeth

Headless cms architectureKevin Wenger

This document discusses headless CMS architecture, including its pros and cons. A headless CMS separates the content management system from the presentation layer, allowing content to be delivered via API to any device. Benefits include frontend flexibility, improved user experience through animation and components, and ability to take advantage of new technologies. Challenges include increased dependencies, need for synchronous releases, building interfaces from scratch, and requiring more skills. The document also provides examples of implementing headless architectures with WordPress, Drupal, and Symfony. It stresses the importance of common documentation and avoiding issues in the development process.

How to Build Deep Learning ModelsJosh Patterson

Managing Data in MicroservicesRandy Shoup

From a talk at the SF CTO Summit 2017 (https://www.ctoconnection.com/summits/sf2017), these slides cover the speaker's experience at Stitch Fix with managing data in a microservices environment. Areas include: * Breaking up a monolithic database into services * Using events as a first-class part of your architecture * Sharing data among microservices * Handling "joins" among microservices * Simulating "transactions" among microservices using the Saga pattern

LF_APIStrat17_Diving Deep into the API Ocean with Open Source Deep Learning T...LF_APIStrat

"The world is full of seas of data. Some of these seas are created and exchanged by APIs; some of them are even about APIs. Since 2014, APImetrics has accumulated over 100 GB of data on API test calls made to over 5000 API endpoints by agents deployed in cloud locations on 5 continents. There's a huge amount of insight trapped at the bottom of that sea of (in our case unlabelled) data. Getting at it would've been nearly impossible before the emergence of powerful open source deep learning libraries in the mid-2010s. APImetrics will share how we chose a deep learning library and the data munging we did to get our data to work with the library. We will explain how we were able carry out unsupervised and semi-supervised learning and discuss the insights on global API performance and quality we were able to dredge from the bottom of our sea of data. We'll provide pointers on how organizations, from startups like APImetrics to megacorporations, can use deep learning to create oceans of knowledge from their own seas of data."

Cloud-native Datacornelia davis

<November 2017 Updated from earlier presentations on Cloud-native Data> Cloud-native applications form the foundation for modern, cloud-scale digital solutions, and the patterns and practices for cloud-native at the app tier are becoming widely understood – statelessness, service discovery, circuit breakers and more. But little has changed in the data tier. Our modern apps are often connected to monolithic shared databases that have monolithic practices wrapped around them. As a result, the autonomy promised by moving to a microservices application architecture is compromised. What we need are patterns and practices for cloud-native data. The anti-patterns of shared databases and simple proxy-style web services to front them give way to approaches that include use of caches (Netflix calls caching their hidden microservice), database per service and polyglot persistence, modern versions of ETL and data integration and more. In this session, aimed at the application developer/architect, Cornelia will look at those patterns and see how they serve the needs of the cloud-native application.

Cloud-Native-Data with Cornelia DavisVMware Tanzu

This document discusses cloud-native data and patterns for managing data in microservices architectures. It describes using data services and APIs to interface with existing data sources. Patterns like caching data at the edge with various caching strategies are discussed. The document also covers using multiple small databases with each microservice rather than a shared database. Event sourcing and CQRS patterns are presented as ways to integrate data across services. Finally, the impact on roles like database administrators is considered in cloud-native data environments.

4 Mapping the ApplicationSam Bowne

Ben Kehoe - Serverless Architecture for the Internet of ThingsServerlessConf

Week 2: Getting Your Hands Dirty – Part 2Jamshid Hashimi

Social Security Company Nexgate's Success Relies on Apache CassandraDataStax Academy

PHISHING URL - Review 1.pptxArulvincent3

This document summarizes a project that aims to detect phishing websites and encrypt user search data for privacy. The objectives are to detect malicious URLs and encrypt user search data before storing it on the server. The proposed system uses a support vector machine to classify URLs as safe or unsafe. It also uses RSA encryption to encrypt user search URLs before transmitting them to the server. The system requirements, module descriptions, algorithms and screenshots are also outlined in the document.

Software Development: Beyond Training wheelsNaveenkumar Muguda

Building a Lightning App with Angular Material DesignSalesforce Developers

Join us as we go over how to make a custom Lightning component utilizing Angular's Material Design framework. We'll cover how to build Salesforce Lightning components that will display your Salesforce data in high quality Material Design-centric views. We will go over the step-by-step process of configuring a Lightning component to work using AngularJS and Angular Material Design. We'll discuss the parts of a Lightning component, how to architect your Angular code, how to load dependencies, and finally, integrating your Angular code with the Lightning component code. We will then review the components of the Material Design library and learn how to utilize the features to create a sleek interface. We'll end with an overview of tools and common issues or considerations.

Introduction to IoT.pptx for btech studentssaritanayak23

Software design with Domain-driven design Allan Mangune

The document discusses domain-driven software design and related concepts. It begins with an overview of monolithic architecture and modular monoliths. It then covers IdentityServer4 for centralized authentication. Next, it defines domain-driven design and the utility of domain models. It discusses ingredients for effective modeling like prototyping and collaboration. It also covers bounded contexts, entities, value objects, and repositories. Finally, it provides tips for optimizing entities and database operations.

Machine Learning With ML.NETDev Raj Gautam

A Beginner's Guide to EmberRichard Martin

AALL 2015: Hands on Linked Data Tools for Catalogers: MarcEdit and MARCNextTerry Reese

Practical approaches to entification in library bibliographic dataTerry Reese

Ladies Be Architects - Integration - Multi-Org, Security, JSON, Backup & Restoregemziebeth

Headless cms architectureKevin Wenger

How to Build Deep Learning ModelsJosh Patterson

Managing Data in MicroservicesRandy Shoup

LF_APIStrat17_Diving Deep into the API Ocean with Open Source Deep Learning T...LF_APIStrat

Cloud-native Datacornelia davis

Cloud-Native-Data with Cornelia DavisVMware Tanzu

4 Mapping the ApplicationSam Bowne

Ben Kehoe - Serverless Architecture for the Internet of ThingsServerlessConf

Week 2: Getting Your Hands Dirty – Part 2Jamshid Hashimi

Social Security Company Nexgate's Success Relies on Apache CassandraDataStax Academy

PHISHING URL - Review 1.pptxArulvincent3

Software Development: Beyond Training wheelsNaveenkumar Muguda

Building a Lightning App with Angular Material DesignSalesforce Developers

Introduction to IoT.pptx for btech studentssaritanayak23

Software design with Domain-driven design Allan Mangune

Machine Learning With ML.NETDev Raj Gautam

More from DataStax Academy (20)

Forrester CXNYC 2017 - Delivering great real-time cx is a true craftDataStax Academy

Introduction to DataStax Enterprise Graph DatabaseDataStax Academy

DataStax Enterprise (DSE) Graph is a built to manage, analyze, and search highly connected data. DSE Graph, built on NoSQL Apache Cassandra delivers continuous uptime along with predictable performance and scales for modern systems dealing with complex and constantly changing data. Download DataStax Enterprise: Academy.DataStax.com/Download Start free training for DataStax Enterprise Graph: Academy.DataStax.com/courses/ds332-datastax-enterprise-graph

Introduction to DataStax Enterprise Advanced Replication with Apache CassandraDataStax Academy

DataStax Enterprise Advanced Replication supports one-way distributed data replication from remote database clusters that might experience periods of network or internet downtime. Benefiting use cases that require a 'hub and spoke' architecture. Learn more at http://www.datastax.com/2016/07/stay-100-connected-with-dse-advanced-replication Advanced Replication docs – https://docs.datastax.com/en/latest-dse/datastax_enterprise/advRep/advRepTOC.html

Cassandra on Docker @ Walmart LabsDataStax Academy

This document discusses using Docker containers to run Cassandra clusters at Walmart. It proposes transforming existing Cassandra hardware into containers to better utilize unused compute. It also suggests building new Cassandra clusters in containers and migrating old clusters to double capacity on existing hardware and save costs. Benchmark results show Docker containers outperforming virtual machines on OpenStack and Azure in terms of reads, writes, throughput and latency for an in-house application.

Cassandra 3.0 Data ModelingDataStax Academy

The document discusses the evolution of Cassandra's data modeling capabilities over different versions of CQL. It covers features introduced in each version such as user defined types, functions, aggregates, materialized views, and storage attached secondary indexes (SASI). It provides examples of how to create user defined types, functions, materialized views, and SASI indexes in CQL. It also discusses when each feature should and should not be used.

Cassandra Adoption on Cisco UCS & Open stackDataStax Academy

Cisco has a large global IT infrastructure supporting many applications, databases, and employees. The document discusses Cisco's existing customer service and commerce systems (CSCC/SMS3) and some of the performance, scalability, and user experience issues. It then presents a proposed new architecture using modern technologies like Elasticsearch, Cassandra, and microservices to address these issues and improve agility, performance, scalability, uptime, and the user interface.

Data Modeling for Apache CassandraDataStax Academy

Data Modeling is the one of the first things to sink your teeth into when trying out a new database. That's why we are going to cover this foundational topic in enough detail for you to get dangerous. Data Modeling for relational databases is more than a touch different than the way it's approached with Cassandra. We will address the quintessential query-driven methodology through a couple of different use cases, including working with time series data for IoT. We will also demo a new tool to get you bootstrapped quickly with MovieLens sample data. This talk should give you the basics you need to get serious with Apache Cassandra.

Coursera Cassandra DriverDataStax Academy

Hear about how Coursera uses Cassandra as the core of its scalable online education platform. I'll discuss the strengths of Cassandra that we leverage, as well as some limitations that you might run into as well in practice. In the second part of this talk, we'll dive into how best to effectively use the Datastax Java drivers. We'll dig into how the driver is architected, and use this understanding to develop best practices to follow. I'll also share a couple of interesting bug we've run into at Coursera.

Production Ready CassandraDataStax Academy

Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & PythonDataStax Academy

This document summarizes three presentations from a Cassandra Meetup: 1. Jason Cacciatore discussed monitoring Cassandra health at scale across hundreds of clusters and thousands of nodes using the reactive stream processing system Mantis. 2. Minh Do explained how Cassandra uses the gossip protocol for tasks like discovering cluster topology and sharing load information. Gossip also has limitations and race conditions that can cause problems. 3. Chris Kalantzis presented Cassandra Tickler, an open source tool he created to help repair operations that get stuck by running lightweight consistency checks on an old Cassandra version or a node with space issues.

Cassandra @ Sony: The good, the bad, and the ugly part 1DataStax Academy

Cassandra @ Sony: The good, the bad, and the ugly part 2DataStax Academy

The document discusses Cassandra's use by Sony Network Entertainment to handle the large amount of user and transaction data from the growing PlayStation Network. It describes how the relational database they previously used did not scale sufficiently, so they transitioned to using Cassandra in a denormalized and customized way. Some of the techniques discussed include caching user data locally on application servers, secondary indexing, and using a real-time indexer to enable personalized search by friends.

Standing Up Your First ClusterDataStax Academy

This document provides guidance on setting up server monitoring, application metrics, log aggregation, time synchronization, replication strategies, and garbage collection for a Cassandra cluster. Key recommendations include: 1. Use monitoring tools like Monit, Munin, Nagios, or OpsCenter to monitor processes, disk usage, and system performance. Aggregate all logs centrally with tools like Splunk, Logstash, or Greylog. 2. Install NTP to synchronize server times which are critical for consistency. 3. Use the NetworkTopologyStrategy replication strategy and avoid SimpleStrategy for production. 4. Avoid shared storage and focus on low latency and high throughput using multiple local disks. 5. Understand

Real Time Analytics with DseDataStax Academy

This document discusses real time analytics using Spark and Spark Streaming. It provides an introduction to Spark and highlights limitations of Hadoop for real-time analytics. It then describes Spark's advantages like in-memory processing and rich APIs. The document discusses Spark Streaming and the Spark Cassandra Connector. It also introduces DataStax Enterprise which integrates Spark, Cassandra and Solr to allow real-time analytics without separate clusters. Examples of streaming use cases and demos are provided.

Introduction to Data Modeling with Apache CassandraDataStax Academy

This document provides an introduction to data modeling with Apache Cassandra. It discusses how Cassandra data models are designed based on the queries an application will perform, unlike relational databases which are designed based on normalization rules. Key aspects covered include avoiding joins by denormalizing data, using a partition key to group related data on nodes, and controlling the clustering order of columns. The document provides examples of modeling time series and tag data in Cassandra.

Cassandra Core ConceptsDataStax Academy

The document discusses different data storage options for small, medium, and large datasets. It argues that relational databases do not scale well for large datasets due to limitations with replication, normalization, sharding, and high availability. The document then introduces Apache Cassandra as a fast, distributed, highly available, and linearly scalable database that addresses these limitations through its use of a hash ring architecture and tunable consistency levels. It describes Cassandra's key features including replication, compaction, and multi-datacenter support.

Enabling Search in your Cassandra Application with DataStax EnterpriseDataStax Academy

This document provides an overview of using Datastax Enterprise (DSE) Search to enable full-text search capabilities in Cassandra applications. It discusses how DSE Search integrates Solr/Lucene indexing with the Cassandra database to allow searching of application data without requiring a separate search cluster, external ETL processes, or custom application code for data management. The document also includes examples of different types of searches that can be performed, such as filtering, faceting, geospatial searches, and joins. It concludes with basic steps for getting started with DSE Search such as creating a Solr core and executing search queries using CQL.

Bad Habits Die Hard DataStax Academy

The document discusses common bad habits that can occur when working with Apache Cassandra and provides recommendations to avoid them. Specifically, it addresses issues like sliding back into a relational mindset when the data model is different, improperly benchmarking Cassandra systems, having slow client performance, and neglecting important operations tasks. The presentation provides guidance on how to approach data modeling, querying, benchmarking, driver usage, and operations management in a Cassandra-oriented way.

Advanced Data Modeling with Apache CassandraDataStax Academy

This document provides an overview and examples of modeling data in Apache Cassandra. It begins with an introduction to thinking about data models and queries before modeling, and emphasizes that Cassandra requires modeling around queries due to its limitations on joins and indexes. The document then provides examples of modeling user, video, and other entity data for a video sharing application to support common queries. It also discusses techniques for handling queries that could become hotspots, such as bucketing or adding random values. The examples illustrate best practices for data duplication, materialized views, and time series data storage in Cassandra.

Advanced CassandraDataStax Academy

The document discusses best practices for using Apache Cassandra, including: - Topology considerations like replication strategies and snitches - Booting new datacenters and replacing nodes - Security techniques like authentication, authorization, and SSL encryption - Using prepared statements for efficiency - Asynchronous execution for request pipelining - Batch statements and their appropriate uses - Improving performance through techniques like the new row cache

Forrester CXNYC 2017 - Delivering great real-time cx is a true craftDataStax Academy

Introduction to DataStax Enterprise Graph DatabaseDataStax Academy

Introduction to DataStax Enterprise Advanced Replication with Apache CassandraDataStax Academy

Cassandra on Docker @ Walmart LabsDataStax Academy

Cassandra 3.0 Data ModelingDataStax Academy

Cassandra Adoption on Cisco UCS & Open stackDataStax Academy

Data Modeling for Apache CassandraDataStax Academy

Coursera Cassandra DriverDataStax Academy

Production Ready CassandraDataStax Academy

Cassandra @ Netflix: Monitoring C* at Scale, Gossip and Tickler & PythonDataStax Academy

Cassandra @ Sony: The good, the bad, and the ugly part 1DataStax Academy

Cassandra @ Sony: The good, the bad, and the ugly part 2DataStax Academy

Standing Up Your First ClusterDataStax Academy

Real Time Analytics with DseDataStax Academy

Introduction to Data Modeling with Apache CassandraDataStax Academy

Cassandra Core ConceptsDataStax Academy

Enabling Search in your Cassandra Application with DataStax EnterpriseDataStax Academy

Bad Habits Die Hard DataStax Academy

Advanced Data Modeling with Apache CassandraDataStax Academy

Advanced CassandraDataStax Academy

Recently uploaded (20)

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Build Your Own Copilot & Agents For DevsBrian McKeiver

Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan

This is a Quick Research Guide (QRG). QRGs include the following: - A brief, high-level overview of the QRG topic. - A milestone timeline for the QRG topic. - Links to various free online resource materials to provide a deeper dive into the QRG topic. - Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic. QRGs planned for the series: - Artificial Intelligence QRG - Quantum Computing QRG - Big Data Analytics QRG - Spacecraft Guidance, Navigation & Control QRG (coming 2026) - UK Home Computing & The Birth of ARM QRG (coming 2027) Any questions or comments? - Please contact Arthur Morgan at [email protected]. 100% human made.

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

HCL Nomad Web – Best Practices and Managing Multiuser Environmentspanagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-nomad-web-best-practices-and-managing-multiuser-environments/ HCL Nomad Web is heralded as the next generation of the HCL Notes client, offering numerous advantages such as eliminating the need for packaging, distribution, and installation. Nomad Web client upgrades will be installed “automatically” in the background. This significantly reduces the administrative footprint compared to traditional HCL Notes clients. However, troubleshooting issues in Nomad Web present unique challenges compared to the Notes client. Join Christoph and Marc as they demonstrate how to simplify the troubleshooting process in HCL Nomad Web, ensuring a smoother and more efficient user experience. In this webinar, we will explore effective strategies for diagnosing and resolving common problems in HCL Nomad Web, including - Accessing the console - Locating and interpreting log files - Accessing the data folder within the browser’s cache (using OPFS) - Understand the difference between single- and multi-user scenarios - Utilizing Client Clocking

Rusty Waters: Elevating Lakehouses Beyond Sparkcarlyakerly1

Spark is a powerhouse for large datasets, but when it comes to smaller data workloads, its overhead can sometimes slow things down. What if you could achieve high performance and efficiency without the need for Spark? At S&P Global Commodity Insights, having a complete view of global energy and commodities markets enables customers to make data-driven decisions with confidence and create long-term, sustainable value. 🌍 Explore delta-rs + CDC and how these open-source innovations power lightweight, high-performance data applications beyond Spark! 🚀

Cyber Awareness overview for 2025 month of securityriccardosl1

Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Join this UiPath Community Berlin meetup to explore the Orchestrator API, Swagger interface, and the Test Manager API. Learn how to leverage these tools to streamline automation, enhance testing, and integrate more efficiently with UiPath. Perfect for developers, testers, and automation enthusiasts! 📕 Agenda Welcome & Introductions Orchestrator API Overview Exploring the Swagger Interface Test Manager API Highlights Streamlining Automation & Testing with APIs (Demo) Q&A and Open Discussion Perfect for developers, testers, and automation enthusiasts! 👉 Join our UiPath Community Berlin chapter: https://community.uipath.com/berlin/ This session streamed live on April 29, 2025, 18:00 CET. Check out all our upcoming UiPath Community sessions at https://community.uipath.com/events/.

Cybersecurity Identity and Access Solutions using Azure ADVICTOR MAESTRE RAMIREZ

Greenhouse_Monitoring_Presentation.pptx.hpbmnnxrvb

The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfAbi john

Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxAnoop Ashok

Splunk Security Update | Public Sector Summit Germany 2025Splunk

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Talk at the final event of Data Fusion Dynamics: A Collaborative UK-Saudi Initiative in Cybersecurity and Artificial Intelligence funded by the British Council UK-Saudi Challenge Fund 2024, Cardiff Metropolitan University, 29th April 2025 https://alandix.com/academic/talks/CMet2025-AI-Changes-Everything/ Is AI just another technology, or does it fundamentally change the way we live and think? Every technology has a direct impact with micro-ethical consequences, some good, some bad. However more profound are the ways in which some technologies reshape the very fabric of society with macro-ethical impacts. The invention of the stirrup revolutionised mounted combat, but as a side effect gave rise to the feudal system, which still shapes politics today. The internal combustion engine offers personal freedom and creates pollution, but has also transformed the nature of urban planning and international trade. When we look at AI the micro-ethical issues, such as bias, are most obvious, but the macro-ethical challenges may be greater. At a micro-ethical level AI has the potential to deepen social, ethnic and gender bias, issues I have warned about since the early 1990s! It is also being used increasingly on the battlefield. However, it also offers amazing opportunities in health and educations, as the recent Nobel prizes for the developers of AlphaFold illustrate. More radically, the need to encode ethics acts as a mirror to surface essential ethical problems and conflicts. At the macro-ethical level, by the early 2000s digital technology had already begun to undermine sovereignty (e.g. gambling), market economics (through network effects and emergent monopolies), and the very meaning of money. Modern AI is the child of big data, big computation and ultimately big business, intensifying the inherent tendency of digital technology to concentrate power. AI is already unravelling the fundamentals of the social, political and economic world around us, but this is a world that needs radical reimagining to overcome the global environmental and human challenges that confront us. Our challenge is whether to let the threads fall as they may, or to use them to weave a better future.

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Impelsys Inc.

How Can I use the AI Hype in my Business Context?Daniel Lehner

𝙄𝙨 𝘼𝙄 𝙟𝙪𝙨𝙩 𝙝𝙮𝙥𝙚? 𝙊𝙧 𝙞𝙨 𝙞𝙩 𝙩𝙝𝙚 𝙜𝙖𝙢𝙚 𝙘𝙝𝙖𝙣𝙜𝙚𝙧 𝙮𝙤𝙪𝙧 𝙗𝙪𝙨𝙞𝙣𝙚𝙨𝙨 𝙣𝙚𝙚𝙙𝙨? Everyone’s talking about AI but is anyone really using it to create real value? Most companies want to leverage AI. Few know 𝗵𝗼𝘄. ✅ What exactly should you ask to find real AI opportunities? ✅ Which AI techniques actually fit your business? ✅ Is your data even ready for AI? If you’re not sure, you’re not alone. This is a condensed version of the slides I presented at a Linkedin webinar for Tecnovy on 28.04.2025.

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

Manifest Pre-Seed Update | A Humanoid OEM Deeptech In Francechb3

Build Your Own Copilot & Agents For DevsBrian McKeiver

Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

HCL Nomad Web – Best Practices and Managing Multiuser Environmentspanagenda

Rusty Waters: Elevating Lakehouses Beyond Sparkcarlyakerly1

Cyber Awareness overview for 2025 month of securityriccardosl1

Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo

UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPathCommunity

Cybersecurity Identity and Access Solutions using Azure ADVICTOR MAESTRE RAMIREZ

Greenhouse_Monitoring_Presentation.pptx.hpbmnnxrvb

The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfAbi john

Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxAnoop Ashok

Splunk Security Update | Public Sector Summit Germany 2025Splunk

AI Changes Everything – Talk at Cardiff Metropolitan University, 29th April 2...Alan Dix

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Impelsys Inc.

How Can I use the AI Hype in my Business Context?Daniel Lehner

2025-05-Q4-2024-Investor-Presentation.pptxSamuele Fogagnolo

AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore

1. Monacella: An Object-Relational Database

2. Origin Story • need to import ad network data • highly parallelized ETL pipeline • hit Postgres connection limits during beta • considered sharding, but didn't have ops experience for it • instead decided to try Cassandra because • we already had ops experience with it • low maintenance • complexity would be in app layer where we were experts © 2015. All Rights Reserved. 2

3. Naive Architecture © 2015. All Rights Reserved. 3 campaign_id: string account_id: string created_at: timestamp data: string:string /network/adwords/ad/1234 ads campaigns accounts secondary indexes ad_id: string

4. Sensible Architecture © 2015. All Rights Reserved. 4 filter_name: string shard_id: integer entity_id: string none | active | inactive | ... ads campaigns accounts secondary indexes stored_filters ... facet_id: string ...

5. Oops! I Made an Object-Relational Database • at first didn't know what I had, but then slowly realized: • rows are objects with types/classes • objects have type-based relationships to other objects • support for object relationships but not arbitrary joins • who would want this? • need horizontal scalability • don't need all the features of a relational database... • ...but do need object relationships © 2015. All Rights Reserved. 5

6. Monacella and the Future • build out as a database with Cassandra as the datastore • simplify data model • stored filter optimizations • change datastore to Dynomite over Redis or Postgres • Doradus • Spider does the same thing as Monacella but maybe better • and it has an OLAP layer - something we also built! • sharded Postgres? © 2015. All Rights Reserved. 6

7. Thank you

AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore

Recommended

More Related Content

What's hot (20)

Similar to AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore (20)

More from DataStax Academy (20)

Recently uploaded (20)

AdStage: Monacella: An Relational Object Database using Cassandra as the Datastore