SlideShare a Scribd company logo
Apache
                           CouchDB
Monday, November 2, 2009

how couchdb treats the disks
append only btree
Replication
                           JSON
 Map Reduce
                           HTTP
Monday, November 2, 2009

intro:
offline apps
simple development
easy to deploy
Stuart Langridge - Canonical




      !                                       !


Monday, November 2, 2009

15 million desktops by the end of the year.
solve the data island problem
Monday, November 2, 2009

raindrop - next gen messaging platform
name: your own personal piece of the cloud
Monday, November 2, 2009
Monday, November 2, 2009

Couchapps
-- easiest way to deploy an Ajax application that needs to store data.
-- capable of scaling up with CouchDB. (etags etc)
“Ground Computing”
                                                                          @jhuggins




                                                     http://www.flickr.com/photos/mcpig/872293700/
Monday, November 2, 2009

app runs near the user. lower latency, lower cost.
Monday, November 2, 2009
Offline by Default




                                               http://www.flickr.com/photos/shane-h/280084650
Monday, November 2, 2009

reliable
telephone poles / like akamai but for dynamic database backed applications.
 - same application on the client and server or the cloud
http://apod.nasa.gov/apod/ap050930.html




                                  “Of the Web”



                            http://jacobian.org/writing/of-the-web/



Monday, November 2, 2009
Let me tell you something: Django may be built for the Web, but CouchDB is built of the Web. I've never seen software that so
completely embraces the philosophies behind HTTP. ... this is what the software of the future looks like. Jacob Kaplan-Moss --
October, 2007

perfect spot - discovered not invented

When you can move data via replication, it

 changes the physics of the web

so data is useful independent of location.
Lockless
                           Storage
Monday, November 2, 2009

html5 story

   js

   ruby

   
   transactions?

-- getting CouchDB right involves going all the way to core lockless storage engine.

performance, map reduce, and deployment
B-Tree

Monday, November 2, 2009

data structure
sorted by keys
minimize disk seeks
Monday, November 2, 2009

logical btree
Monday, November 2, 2009

indexes - efficient
key lookups
range scans
Append

                                Only
Monday, November 2, 2009

robust / reliable
never touch bytes that are on disk
can’t be inconsistent
Monday, November 2, 2009

append only - in action
update J to a new value (rewrite j’s leaf node)
Monday, November 2, 2009

rewrite the path along the tree to J
up the parent path until we write the root.
Monday, November 2, 2009

all pointers point backwards into the file
Monday, November 2, 2009

update cost?

only has O(log n) cost where n is the # of keys
Monday, November 2, 2009

another update - add 5.

rewrite path to Z node to add 5
Monday, November 2, 2009

time follows the blue arrow.
top half is actually an accumulation of updates like the bottom.
note: batch efficiency -- save block updates
Monday, November 2, 2009

2 day test run
couchdb over long haul
4 writers, 16 readers, random access, small keyspace - lots of 409s etc.
Incremental

                           Reduce
Monday, November 2, 2009

takes advantage of the write pattern we’ve seen
setup main storage doc count example
Monday, November 2, 2009

rereduce = false
count # of keys in the block
Monday, November 2, 2009

rereduce = true
sum # of docs in child nodes
flexible range querying
Monday, November 2, 2009

intermediate reductions stored during a view calculation
updating J doesn’t change any counts, adding the 5 key does
Monday, November 2, 2009

we are rewriting the parent nodes anyway, so it makes sense to run the reduce calculation
Concurrent
               Access
Monday, November 2, 2009

readers see a consistent view
we can support many readers
reads don’t block writes
writes don’t block reads
Monday, November 2, 2009

concurrent readers
list all docs, then a write comes in, you don’t want to be surprised by an extra doc
readers can be as long lived as they want without blocking writers or seeing inconsistencies
Local Web
                    Platform
Monday, November 2, 2009

user interviews
http://www.longnow.org/projects/clock/orrery/
Monday, November 2, 2009

append-only
no fixup phase
reliable and concurrent
sequential write pattern (few writer seeks)
hot backups
compaction
Ad

More Related Content

Similar to Btree Nosql Oak (20)

Performance Strategies
Performance StrategiesPerformance Strategies
Performance Strategies
Alexander Meindl
 
Go Course Day1
Go Course Day1Go Course Day1
Go Course Day1
guest83950e2
 
Apache Hadoop Talk at QCon
Apache Hadoop Talk at QConApache Hadoop Talk at QCon
Apache Hadoop Talk at QCon
Cloudera, Inc.
 
Docker
DockerDocker
Docker
Thiago Rondon
 
Game Changing Dependency Management
Game Changing Dependency ManagementGame Changing Dependency Management
Game Changing Dependency Management
Jeremy Kendall
 
The State of the Social Desktop 2009
The State of the Social Desktop 2009The State of the Social Desktop 2009
The State of the Social Desktop 2009
Frank Karlitschek
 
Caching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practicesCaching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practices
Lars Jankowfsky
 
Fast & Furious: Speed in the Opera browser
Fast & Furious: Speed in the Opera browserFast & Furious: Speed in the Opera browser
Fast & Furious: Speed in the Opera browser
Andreas Bovens
 
Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010
Matt Aimonetti
 
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 20092009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
Caue Guerra
 
Aloha on-rails-2009
Aloha on-rails-2009Aloha on-rails-2009
Aloha on-rails-2009
John Woodell
 
Building A Scalable Open Source Storage Solution
Building A Scalable Open Source Storage SolutionBuilding A Scalable Open Source Storage Solution
Building A Scalable Open Source Storage Solution
Phil Cryer
 
Everyday - mongodb
Everyday - mongodbEveryday - mongodb
Everyday - mongodb
elliando dias
 
The Open-PC - OpenSourceExpo 2009
The Open-PC - OpenSourceExpo 2009The Open-PC - OpenSourceExpo 2009
The Open-PC - OpenSourceExpo 2009
Frank Karlitschek
 
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of FacebookTech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
The Hive
 
Functional Java Script - Webtechcon 2009
Functional Java Script - Webtechcon 2009Functional Java Script - Webtechcon 2009
Functional Java Script - Webtechcon 2009
wolframkriesing
 
JRubyConf 2009
JRubyConf 2009JRubyConf 2009
JRubyConf 2009
John Woodell
 
Cloudera Desktop
Cloudera DesktopCloudera Desktop
Cloudera Desktop
Hadoop User Group
 
Google Web Toolkit for the Enterprise Developer - JBoss World 2009
Google Web Toolkit for the Enterprise Developer - JBoss World 2009Google Web Toolkit for the Enterprise Developer - JBoss World 2009
Google Web Toolkit for the Enterprise Developer - JBoss World 2009
Fred Sauer
 
Nuxeo Iks 2009 11 13
Nuxeo Iks 2009 11 13Nuxeo Iks 2009 11 13
Nuxeo Iks 2009 11 13
Olivier Grisel
 
Apache Hadoop Talk at QCon
Apache Hadoop Talk at QConApache Hadoop Talk at QCon
Apache Hadoop Talk at QCon
Cloudera, Inc.
 
Game Changing Dependency Management
Game Changing Dependency ManagementGame Changing Dependency Management
Game Changing Dependency Management
Jeremy Kendall
 
The State of the Social Desktop 2009
The State of the Social Desktop 2009The State of the Social Desktop 2009
The State of the Social Desktop 2009
Frank Karlitschek
 
Caching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practicesCaching, sharding, distributing - Scaling best practices
Caching, sharding, distributing - Scaling best practices
Lars Jankowfsky
 
Fast & Furious: Speed in the Opera browser
Fast & Furious: Speed in the Opera browserFast & Furious: Speed in the Opera browser
Fast & Furious: Speed in the Opera browser
Andreas Bovens
 
Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010Macruby - RubyConf Presentation 2010
Macruby - RubyConf Presentation 2010
Matt Aimonetti
 
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 20092009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
2009, o ano do Ruby on Rails no Brasil - CaelumDay 2009
Caue Guerra
 
Aloha on-rails-2009
Aloha on-rails-2009Aloha on-rails-2009
Aloha on-rails-2009
John Woodell
 
Building A Scalable Open Source Storage Solution
Building A Scalable Open Source Storage SolutionBuilding A Scalable Open Source Storage Solution
Building A Scalable Open Source Storage Solution
Phil Cryer
 
The Open-PC - OpenSourceExpo 2009
The Open-PC - OpenSourceExpo 2009The Open-PC - OpenSourceExpo 2009
The Open-PC - OpenSourceExpo 2009
Frank Karlitschek
 
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of FacebookTech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
The Hive
 
Functional Java Script - Webtechcon 2009
Functional Java Script - Webtechcon 2009Functional Java Script - Webtechcon 2009
Functional Java Script - Webtechcon 2009
wolframkriesing
 
Google Web Toolkit for the Enterprise Developer - JBoss World 2009
Google Web Toolkit for the Enterprise Developer - JBoss World 2009Google Web Toolkit for the Enterprise Developer - JBoss World 2009
Google Web Toolkit for the Enterprise Developer - JBoss World 2009
Fred Sauer
 

More from Chris Anderson (6)

CouchDB - Local Web Platform
CouchDB - Local Web PlatformCouchDB - Local Web Platform
CouchDB - Local Web Platform
Chris Anderson
 
CouchDB Open Source Bridge
CouchDB Open Source BridgeCouchDB Open Source Bridge
CouchDB Open Source Bridge
Chris Anderson
 
CouchDB To The Edge
CouchDB To The EdgeCouchDB To The Edge
CouchDB To The Edge
Chris Anderson
 
CouchDB to the Edge ApacheCon EU
CouchDB to the  Edge ApacheCon EUCouchDB to the  Edge ApacheCon EU
CouchDB to the Edge ApacheCon EU
Chris Anderson
 
Vidoop CouchDB Talk
Vidoop CouchDB TalkVidoop CouchDB Talk
Vidoop CouchDB Talk
Chris Anderson
 
CouchDB Talk JChris NYC
CouchDB Talk JChris NYCCouchDB Talk JChris NYC
CouchDB Talk JChris NYC
Chris Anderson
 
CouchDB - Local Web Platform
CouchDB - Local Web PlatformCouchDB - Local Web Platform
CouchDB - Local Web Platform
Chris Anderson
 
CouchDB Open Source Bridge
CouchDB Open Source BridgeCouchDB Open Source Bridge
CouchDB Open Source Bridge
Chris Anderson
 
CouchDB to the Edge ApacheCon EU
CouchDB to the  Edge ApacheCon EUCouchDB to the  Edge ApacheCon EU
CouchDB to the Edge ApacheCon EU
Chris Anderson
 
CouchDB Talk JChris NYC
CouchDB Talk JChris NYCCouchDB Talk JChris NYC
CouchDB Talk JChris NYC
Chris Anderson
 
Ad

Recently uploaded (20)

Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPathCommunity
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded DevelopersLinux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Toradex
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptxSpecial Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
shyamraj55
 
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveDesigning Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
ScyllaDB
 
Cyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of securityCyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of security
riccardosl1
 
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
BookNet Canada
 
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsAI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global Trends
InData Labs
 
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.
hpbmnnxrvb
 
Drupalcamp Finland – Measuring Front-end Energy Consumption
Drupalcamp Finland – Measuring Front-end Energy ConsumptionDrupalcamp Finland – Measuring Front-end Energy Consumption
Drupalcamp Finland – Measuring Front-end Energy Consumption
Exove
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager APIUiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPath Community Berlin: Orchestrator API, Swagger, and Test Manager API
UiPathCommunity
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded DevelopersLinux Support for SMARC: How Toradex Empowers Embedded Developers
Linux Support for SMARC: How Toradex Empowers Embedded Developers
Toradex
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptxSpecial Meetup Edition - TDX Bengaluru Meetup #52.pptx
Special Meetup Edition - TDX Bengaluru Meetup #52.pptx
shyamraj55
 
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveDesigning Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep Dive
ScyllaDB
 
Cyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of securityCyber Awareness overview for 2025 month of security
Cyber Awareness overview for 2025 month of security
riccardosl1
 
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...
BookNet Canada
 
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsAI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global Trends
InData Labs
 
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.
hpbmnnxrvb
 
Drupalcamp Finland – Measuring Front-end Energy Consumption
Drupalcamp Finland – Measuring Front-end Energy ConsumptionDrupalcamp Finland – Measuring Front-end Energy Consumption
Drupalcamp Finland – Measuring Front-end Energy Consumption
Exove
 
Ad

Btree Nosql Oak

  • 1. Apache CouchDB Monday, November 2, 2009 how couchdb treats the disks append only btree
  • 2. Replication JSON Map Reduce HTTP Monday, November 2, 2009 intro: offline apps simple development easy to deploy
  • 3. Stuart Langridge - Canonical ! ! Monday, November 2, 2009 15 million desktops by the end of the year. solve the data island problem
  • 4. Monday, November 2, 2009 raindrop - next gen messaging platform name: your own personal piece of the cloud
  • 6. Monday, November 2, 2009 Couchapps -- easiest way to deploy an Ajax application that needs to store data. -- capable of scaling up with CouchDB. (etags etc)
  • 7. “Ground Computing” @jhuggins http://www.flickr.com/photos/mcpig/872293700/ Monday, November 2, 2009 app runs near the user. lower latency, lower cost.
  • 9. Offline by Default http://www.flickr.com/photos/shane-h/280084650 Monday, November 2, 2009 reliable telephone poles / like akamai but for dynamic database backed applications. - same application on the client and server or the cloud
  • 10. http://apod.nasa.gov/apod/ap050930.html “Of the Web” http://jacobian.org/writing/of-the-web/ Monday, November 2, 2009 Let me tell you something: Django may be built for the Web, but CouchDB is built of the Web. I've never seen software that so completely embraces the philosophies behind HTTP. ... this is what the software of the future looks like. Jacob Kaplan-Moss -- October, 2007 perfect spot - discovered not invented When you can move data via replication, it changes the physics of the web so data is useful independent of location.
  • 11. Lockless Storage Monday, November 2, 2009 html5 story js ruby transactions? -- getting CouchDB right involves going all the way to core lockless storage engine. performance, map reduce, and deployment
  • 12. B-Tree Monday, November 2, 2009 data structure sorted by keys minimize disk seeks
  • 13. Monday, November 2, 2009 logical btree
  • 14. Monday, November 2, 2009 indexes - efficient key lookups range scans
  • 15. Append Only Monday, November 2, 2009 robust / reliable never touch bytes that are on disk can’t be inconsistent
  • 16. Monday, November 2, 2009 append only - in action update J to a new value (rewrite j’s leaf node)
  • 17. Monday, November 2, 2009 rewrite the path along the tree to J up the parent path until we write the root.
  • 18. Monday, November 2, 2009 all pointers point backwards into the file
  • 19. Monday, November 2, 2009 update cost? only has O(log n) cost where n is the # of keys
  • 20. Monday, November 2, 2009 another update - add 5. rewrite path to Z node to add 5
  • 21. Monday, November 2, 2009 time follows the blue arrow. top half is actually an accumulation of updates like the bottom. note: batch efficiency -- save block updates
  • 22. Monday, November 2, 2009 2 day test run couchdb over long haul 4 writers, 16 readers, random access, small keyspace - lots of 409s etc.
  • 23. Incremental Reduce Monday, November 2, 2009 takes advantage of the write pattern we’ve seen setup main storage doc count example
  • 24. Monday, November 2, 2009 rereduce = false count # of keys in the block
  • 25. Monday, November 2, 2009 rereduce = true sum # of docs in child nodes flexible range querying
  • 26. Monday, November 2, 2009 intermediate reductions stored during a view calculation updating J doesn’t change any counts, adding the 5 key does
  • 27. Monday, November 2, 2009 we are rewriting the parent nodes anyway, so it makes sense to run the reduce calculation
  • 28. Concurrent Access Monday, November 2, 2009 readers see a consistent view we can support many readers reads don’t block writes writes don’t block reads
  • 29. Monday, November 2, 2009 concurrent readers list all docs, then a write comes in, you don’t want to be surprised by an extra doc readers can be as long lived as they want without blocking writers or seeing inconsistencies
  • 30. Local Web Platform Monday, November 2, 2009 user interviews
  • 31. http://www.longnow.org/projects/clock/orrery/ Monday, November 2, 2009 append-only no fixup phase reliable and concurrent sequential write pattern (few writer seeks) hot backups compaction