[PASS Summit 2016] Azure DocumentDB: A Deep Dive into Advanced Features

Editor's Notes

• #4: Today our lives are informed and influenced by lots of data. Data is the new currency. From how we find a ride to what we watch and how we shop, data drives all these experiences. From the world of structured data for operational needs and analytical needs, today each LOB applications and services we consume use several variety of data. Variety of data dictates that operational databases become schema agnostic and support schema free data storage. Not only variety, but volume of data being generated and the applications deriving intelligence from data is increasing. Velocity of data is also increasing which means applications also expect high throughput for data ingestion and low latency for data retrieval. Databases that are able to provide intelligence to modern applications from high velocity data of different varieties of data at very high volume become what we call as intellibases. These design patterns create endless possibilities in all forms of business applications that improve products and services for consumers. Online retailing experiences, gaming experiences, mobile experiences, vehicle telematics and IoT are some of the key applications that take advantage of the 3V’s of data.
• #5: Let’s talk about database systems… because that’s what we’re here for. More specifically, let’s talk about common tradeoffs in database systems. E.g. Do you want to be rich? Or do you want to be happy? Well… ideally, I’d like to be both  Elastically Scalable: You want it to be not only highly scalable… but you also want elastically scalable. You don’t want to just say I want 100 machines… and be stuck with 100 machines. You want to be able to grow and shrink. Distribution: You don’t want to just want to span a single cluster. You want this elasticity to span multiple regions and datacenters. This is hard…. You can then look at your favorite database… E.g. does MongoDB support geo distribution? Does it support elastic scale? SQL? Oracle? Low latency vs durability…. You want low latency, but you also want high durability. You don’t want low latency, where your database says got it… but it doesn’t actually commit or persist the data. Unfortunately, there are already too many databases that do that… and they lose data left and right. You want transactions, because they give you a friendly programming model… but you also want high scale. There are databases that give you very rich transactions. But what happens when you try to scale this across many many machines globally? There also database that give you no transactions, or transactions over single items… but that’s too extreme. You want have low-cost schema/index management, but you also want rich queries. There are many database out there today that give you flexible schemas, but they can’t auto index everything and give you rich queryability over that data. Programmability vs Availability Many of you may recognize this as the CAP theorem. You want to be able to trade off availability with consistency to provide a friendly programmability model. TCO vs Performance Isolation - Most databases are designed to run on-premise (outside from Google Big Table, AWS DynamoDB, and Azure DocumentDB). But they weren’t designed to run on the cloud, and require dedicated VMs on AWS or Azure to delivered performance… which isn’t very cost effective. If you try to host these at high density to achieve a lower TCO – you end up with noisy neighbor effects. In order to gaurentee predictable performance in a cost-effective multi-tenant model – you need to build a strong resource governance model directly in to the database to isolate tenants.
• #6: PACELC: if there is a partition (P), how does the system trade off availability and consistency (A and C); else (E), when the system is running normally in the absence of partitions, how does the system trade off latency (L) and consistency (C)? Reference: Consistency Tradeoffs in Modern Distributed Database System Design
• #23: What are partitions? Each partition has ​ fixed amount of SSD storage ​ highly available​ hosts one or many partition key value documents​ Collection = Logical container of physical partitions Practically unlimited scale for storage and/or throughput​
• #24: Understanding partition keys Partition key – decides the placement of documents Any document property can be promoted to partition key Document => Hash(Partition key) => Physical partition Example partition key – on collection body Partitioning characteristics Each partition key is given 10GB space Supports online auto-partitioning Transactions scoped to partition key CRUD on partitioned collections Read/Write via examples x-ms-documentdb-partitionKey Query via examples x-ms-documentdb-query-enablecrosspartition Parallel Query execution Stored Procedures