WebWorkersCamp 2010

Jul 3, 2010Download as KEY, PDF1 like541 views

Olivier Gutknecht

Presentation from my talk at WebWorkersCamp 2010, probably a bit hard to follow with the slides only.

Me

Apple alumnus

Fotopedia co-founder & alumnus

Author, french ebook - introduction to REST
http://izibook.eyrolles.com/

Getting in tune

We have a lot of sliders, faders and
buttons to tweak.

And we often ignore most of them.

Especially at architecture level.

What changed ?

Concepts | Context | Implementations

Concepts | Context | Implementations

“The Google File System” / Google 2003

“MapReduce: Simpliﬁed Data Processing on Large
Clusters” / Google 2004

“Life beyond Distributed Transactions: an
Apostate’s Opinion” / Pat Helland, Amazon 2007

[ZFS, Dynamo, PNUTS, CAP, REST,...] papers

Concepts | Context | Implementations

Client and apps evolution:
html5, native apps, COMET, web of [docs,apps], ...

Modeling problems:
tags, social graphs, walls, feeds, ...

“Like the big boys” - brought legitimacy

Concepts | Context | Implementations

Nutch - 2003
Memcached - 2003
Rails - 2004
CouchDB - 2005
S3 & EC2 - 2006
Redis, RabbitMQ,
MongoDB, Node.JS...

Sliders ?

# of backend servers
cache size
fsync / checkpointing
invalidation delay

Sliders

Consistency level
Sync/Async mix
R/W actions balance
Model heterogeneity

It’s the app, really.

And use cases.
And business cases.

Latency

“Disk is the new tape”
Cache ? invalidation & cold starts
...

Read

Reads are a memory & locality problem.

Caches. Partitions. Dupes. Smart shards.

Write

Write now. Work later. (maybe not on reads)

Divide and Conquer. And cheat.

Our frameworks are not ready yet.

Acceptable Latency
Mandatory consistency (and transactions)
Synchronous actions
Maintainability
(Cost)

NoSQL is not about No SQL.

It’s about revisiting assumptions.

CAP ? ACID ? Choose your poisons.
(and enjoy your sliders, yay for Cassandra)

The state of art is not
good enough.
(we’ve been stuck in the
same local minima for
several years)

Ilya Grigorik

Yay for node.js - and Grand Central - and
EM - and Goliath - and JRuby - and ...

HTTP everywhere ?
& AMQP ?

& Thrift ? ProtoBufs ? Avro ?

Because our stacks are
state of the art
(or bleeding edge)

Beyond Packages
Global conﬁg
Change

Do you really want to ssh to your servers ?

You’re 10% unique

Jesse Robbins, Opscode CEO

You’re 10% unique
(and it’s probably the
things you did wrong)

Jesse Robbins, Opscode CEO

The Cloud
What’s great about it ?

Scalability ?
On-demand capacity ?
Reduced CapEx ?

“Everything fails all the time”
Werner Vogels (Amazon CTO)

The Cloud
The best way to discover
new problems.

(and it’s great)

The Cloud
Think stateless. Think
automation. Think latency. Think
crappy machines. Think lost
servers. Thinks lost disks. Think
ofﬂine datacenters. Think ACL.
Think security. Think autoconf.

Episode IV: a New Hope
C h e f , P u p p e t , D r i z z l e , C a s s a n d ra ,
MongoDB, Redis, HTTP, Rails, Node.js,
EventMachine, Twisted, Heroku, SimpleDB,
EC2, GAE, REST, Rack, CouchDB, Redis,
Bitcask, Voldemort, Varnish, Pig, Hadoop,
HBase, Thrift, Protobufs, Avro, ESI, Mahout,
SolR, AMQP, Grand Central Dispatch,
WebSockets, LINQ, Reconnoiter, Nginx, ...

Thanks.

Creative Contents Images by
Big Red Button by thirtyfootscrew - http://www.ﬂickr.com/photos/thirtyfootscrew/3324101122/
The Steam Pipeline by stuck in customs - http://www.ﬂickr.com/photos/stuckincustoms/3780481716
Digidesign Icon Faders by CJ Sorg - http://www.ﬂickr.com/photos/cjsorg/3692124539/
Pro Control 24 by aud1073ch - qhttp://www.ﬂickr.com/photos/aud1073ch/34438156/
Pipe Dream by Rishi Menon - http://www.ﬂickr.com/photos/rxmﬂickr/4102530508/

"As SimilarWeb R&D grew, it has accumulated more and more physical servers, virtual machines and cloud instances. Being production engineers, we regularly needed to locate servers when debugging issues, or when doing capacity planning. Trying to avoid adding another system to the mix, we managed to use our existing services, only adding a nice frontend for the humans. In this talk I’ll present GRU, our lightweight, extensible inventory system. As part of this talk we will release GRU as a BSD 2 licensed open source project."

The cassandra odysseyPhillip Gentry

Php johannesburg meetup - talk 2014 - scaling php in the enterpriseSarel van der Walt

This document discusses scaling PHP applications for enterprise environments. It provides tips on optimizing various aspects of PHP applications and infrastructure to improve scalability. These include optimizing databases, caching, background tasks, frameworks, monitoring, and more. Specific technologies and strategies mentioned include Redis, memcached, haproxy, MySQL optimization techniques like archiving, and moving work to the client side where possible using techniques like AngularJS.

Apache Mesos Distributed Computing Talk brandongulla

This document provides an agenda for a talk on Apache Mesos, a distributed systems kernel that provides resource management and helps to abstract cluster resources. The talk discusses why distributed computing is hard, how Mesos helps by turning servers into a single large computer and improving multi-tenancy. It also covers Mesos history, how it works, examples of Mesos in production use, and alternatives to writing custom frameworks like Marathon.

Ceph & OpenStack - Boston MeetupPatrick McGarry

This document summarizes a presentation about Ceph and OpenStack. The presentation introduces Ceph as software that provides object, block, and file storage. It discusses how Ceph can run on commodity hardware and scale to exabyte sizes. The presentation also covers how Ceph integrates well with OpenStack for tasks like live instance migration between compute nodes with shared Ceph storage. It outlines upcoming improvements to Ceph in areas like geo-replication, erasure coding, and tiering. The document encourages community involvement through online summits, Ceph Days events, IRC channels, and mailing lists.

DrupalCon Barcelona 2015 - Drupal Extreme Scalingzekivazquez

The document discusses a project to build a Drupal installation capable of hosting over 30,000 sites with high performance, availability, and scalability. Technologies used to achieve this include Docker, Mesos, Marathon, DynamoDB, Memcache, MySQL, and S3. Key challenges involved distributing databases and tables across multiple servers, managing site deployment and configuration from a Node.js application, and designing the system to automatically scale on demand using AWS services. The project was ultimately successful in meeting its goals of hosting over 30,000 sites with 99.999% availability.

Removing Environmental Differences - Simon PearsonOutlyer

Is Docker the answer to the Stack x Platform x Cloud explosion thats engulfed the Enterprise? Is IaaS really the right model? or just the one that worked? Has PaaS’s time finally arrived. Can Docker make OS, library, stack choices irrelevant to hosting and Ops? This talk looks at how Pearson is investigating and moving towards Docker, what we’ve learned so far, and what you can learn from our experiences.

Cuisiner ses infrastructures avec ChefOlivier Gutknecht

Introduction à MacRubyOlivier Gutknecht

Chef - Paris BlockCamp - Nov 09Olivier Gutknecht

Paris RailsCamp 2009Olivier Gutknecht

中國的文化遺產x2324589

Lib qual+ à l'ubsNicolas Alarcon

MacRuby, an introductionOlivier Gutknecht

MacRuby is a Ruby implementation for Mac OS X that allows Ruby code to directly interact with Cocoa and other OS X frameworks. It uses a single runtime and garbage collector to bind Ruby and Objective-C. MacRuby aims to provide the best platform for Ruby developers and a great platform for Cocoa developers. Benchmarks show that Fibonacci code runs faster in MacRuby compared to pure Ruby and Objective-C implementations, due to optimizations from the LLVM compiler.

UnConference for Georgia Southern Computer Science March 31, 2015Christopher Curtin

SQL or NoSQL, that is the question!Andraz Tori

The document discusses SQL versus NoSQL databases. It provides background on SQL databases and their advantages, then explains why some large tech companies have adopted NoSQL databases instead. Specifically, it describes how companies like Amazon, Facebook, and Google have such massive amounts of data that traditional SQL databases cannot adequately handle the scale, performance, and flexibility needs. It then summarizes some popular NoSQL databases like Cassandra, Hadoop, MongoDB that were developed to solve the challenges of scaling to big data workloads.

Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCCal Henderson

Java PaaS Comparisons - Khanderao Kandjaxconf

There are many Java Cloud Platforms in the market today. CloudFoundry, CloudBees, Google App Engine, OpenShift, Heroku and, recently announced, Oracle Java Cloud Service. These variety of options often create a dilemma to choose a platform. This presentation would help in understanding and using these platform with some examples. This comparison would help in comparing them for selecting for your application. This presentation would also help in understanding various key factors to look for.

Sparklife - Life In The Trenches With SparkIan Pointer

This document provides tips and tricks for using Apache Spark. It discusses both the benefits of Spark, such as its developer-friendly API and performance advantages over MapReduce, as well as challenges, such as unstable APIs and the difficulty of distributed systems. It provides recommendations for optimizing Spark applications, including choosing the right data structures, partitioning strategies, and debugging and monitoring techniques. It also briefly compares Spark to other streaming frameworks like Storm, Heron, Flink, and Kafka.

Webdevcon Keynote hh-2012-09-18Pierre Joye

The document discusses various technologies and languages including PHP, Python, Ruby, Go, Scala, and Red. It mentions tools for mobile development like PhoneGap and frameworks like jQuery, Dojo, and Sencha. Other topics covered include Node.js, HTML5, cloud computing platforms, virtualization, and decoupled content management systems. Resources and links are provided for many of the languages and technologies discussed.

Building a Database for the End of the Worldjhugg

John Hugg presented on building an operational database for high-performance applications. Some key points: - He set out to reinvent OLTP databases to be 10x faster by leveraging multicore CPUs and partitioning data across cores. - The database, called VoltDB, uses Java for transaction management and networking while storing data in C++ for better performance. - It partitions data and transactions across server cores for parallelism. Global transactions can access all partitions transactionally. - VoltDB is well-suited for fast data applications like IoT, gaming, ad tech which require high write throughput, low latency, and global understanding of live data.

Cloud Computing Bootcamp On The Google App Engine [v1.1]Matthew McCullough

Virtualization Techniques & Cloud ComptingAhmed Mekkawy

This document summarizes different virtualization techniques and cloud computing. It discusses full virtualization, OS-level virtualization, paravirtualization, and hardware-assisted virtualization. It then defines cloud computing and discusses concerns about security, performance, and maturity. Specific cloud services from Amazon Web Services are outlined, including Elastic Compute Cloud (EC2) for computing instances, Elastic Block Storage (EBS), and Simple Storage Service (S3) for storage.

Front Range PHP NoSQL DatabasesJon Meredith

NoSQL databases are non-relational databases designed for large volumes of data across many servers. They emerged to address scaling and reliability issues with relational databases. While different technologies, NoSQL databases are designed for distribution without a single point of failure and to sacrifice consistency for availability if needed. Examples include Dynamo, BigTable, Cassandra and CouchDB.

OpenStack Preso: DevOps on Hybrid Infrastructurerhirschfeld

2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructuredevopsdaysaustin

The document discusses the need for hybrid infrastructure and hybrid DevOps to manage different cloud platforms and physical infrastructure in a consistent way. It notes that while no single API or platform can meet all needs, AWS dominance means its operational patterns have become the benchmark. The key is developing composable infrastructure modules that can be orchestrated together to provide portability across environments using a common operational process.

Keynote: The Future of Apache HBaseHBaseCon

In-Ceph-tion: Deploying a Ceph cluster on DreamComputePatrick McGarry

This document discusses deploying a Ceph cluster on DreamCompute, an OpenStack-powered cloud computing service from DreamHost. It begins with an overview of Ceph's scalability and uses for object, block, and file storage. The document then discusses DreamCompute's open source infrastructure and deploying Ceph using tools like Juju. It provides details on configuring the Ceph cluster by deploying MONs, OSDs, the RGW gateway, and MDS. It concludes by discussing next steps like geo-replication and erasure coding, and opportunities to get involved with the Ceph community.

Building Asynchronous ApplicationsJohan Edstrom

Johan Edstrom discussed scaling applications by making them more asynchronous and distributed. He covered several Apache projects like Camel, Karaf, ActiveMQ, Cassandra and CXF that can help achieve this. Specifically, he showed how to: 1. Use Camel and OSGi to build asynchronous microservices that communicate via an enterprise integration pattern like a message queue. 2. Store data in Cassandra for asynchronous and high-performance access across data centers. 3. Expose APIs asynchronously using CXF and handle requests with non-blocking techniques like futures. 4. Offload business logic to an asynchronous process running on a distributed cache like HazelCast for shared data and parallel processing across nodes

Reflections On ServerlessDiego Pacheco

This document discusses reflections on serverless computing from Diego Pacheco. It describes the evolution from SOA to microservices to serverless. Serverless wave 1 focused on functions and pay-per-use billing but had tooling and scaling limits. Serverless wave 2 enables multi-cloud deployment without vendor lock-in but still has challenges around data management and potential for new integration needs. Kubernetes is the new Linux but other abstractions are adding complexity, and fully serverless applications may require solutions for data integration challenges not yet addressed.

More Related Content

Viewers also liked (6)

Introduction à MacRubyOlivier Gutknecht

Chef - Paris BlockCamp - Nov 09Olivier Gutknecht

Paris RailsCamp 2009Olivier Gutknecht

中國的文化遺產x2324589

Lib qual+ à l'ubsNicolas Alarcon

MacRuby, an introductionOlivier Gutknecht

Introduction à MacRubyOlivier Gutknecht

Chef - Paris BlockCamp - Nov 09Olivier Gutknecht

Paris RailsCamp 2009Olivier Gutknecht

中國的文化遺產x2324589

Lib qual+ à l'ubsNicolas Alarcon

MacRuby, an introductionOlivier Gutknecht

Similar to WebWorkersCamp 2010 (20)

UnConference for Georgia Southern Computer Science March 31, 2015Christopher Curtin

SQL or NoSQL, that is the question!Andraz Tori

Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCCal Henderson

Java PaaS Comparisons - Khanderao Kandjaxconf

Sparklife - Life In The Trenches With SparkIan Pointer

Webdevcon Keynote hh-2012-09-18Pierre Joye

Building a Database for the End of the Worldjhugg

Cloud Computing Bootcamp On The Google App Engine [v1.1]Matthew McCullough

Virtualization Techniques & Cloud ComptingAhmed Mekkawy

Front Range PHP NoSQL DatabasesJon Meredith

OpenStack Preso: DevOps on Hybrid Infrastructurerhirschfeld

2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructuredevopsdaysaustin

Keynote: The Future of Apache HBaseHBaseCon

In-Ceph-tion: Deploying a Ceph cluster on DreamComputePatrick McGarry

Building Asynchronous ApplicationsJohan Edstrom

Reflections On ServerlessDiego Pacheco

Datacenter Computing with Apache Mesos - BigData DCPaco Nathan

The document discusses datacenter computing using Apache Mesos. It begins by discussing concepts like "data democratization" and "cluster democratization", which refer to making data and computing resources available throughout an organization. It then discusses lessons from Google's approach to datacenter computing, and frameworks that can be integrated with Mesos like Hadoop, Spark, and Docker. Examples of companies using Mesos in production are provided, including Twitter, Airbnb, and eBay. Mesos provides a common substrate that makes heterogeneous computing resources available as a homogeneous set, improving scalability, elasticity, fault tolerance and resource utilization.

Web20expo Scalable Web Archguest18a0f1

The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication, and data federation. The overall goal of application architecture is to scale traffic and data while maintaining high availability and performance. Horizontal scaling by adding more servers is preferable to vertical scaling of buying larger servers.

Web20expo Scalable Web Archroyans

The document discusses scalable web architectures and common patterns for scaling web applications. It covers key topics like load balancing, caching, database replication and sharding, and asynchronous queuing to distribute workloads across multiple servers. The goal of these patterns is to scale traffic, data size, and maintainability through horizontal expansion rather than just vertical upgrades.

Web20expo Scalable Web Archmclee

UnConference for Georgia Southern Computer Science March 31, 2015Christopher Curtin

SQL or NoSQL, that is the question!Andraz Tori

Scalable Web Architectures: Common Patterns and Approaches - Web 2.0 Expo NYCCal Henderson

Java PaaS Comparisons - Khanderao Kandjaxconf

Sparklife - Life In The Trenches With SparkIan Pointer

Webdevcon Keynote hh-2012-09-18Pierre Joye

Building a Database for the End of the Worldjhugg

Cloud Computing Bootcamp On The Google App Engine [v1.1]Matthew McCullough

Virtualization Techniques & Cloud ComptingAhmed Mekkawy

Front Range PHP NoSQL DatabasesJon Meredith

OpenStack Preso: DevOps on Hybrid Infrastructurerhirschfeld

2016 - Open Mic - IGNITE - Open Infrastructure = ANY Infrastructuredevopsdaysaustin

Keynote: The Future of Apache HBaseHBaseCon

In-Ceph-tion: Deploying a Ceph cluster on DreamComputePatrick McGarry

Building Asynchronous ApplicationsJohan Edstrom

Reflections On ServerlessDiego Pacheco

Datacenter Computing with Apache Mesos - BigData DCPaco Nathan

Web20expo Scalable Web Archguest18a0f1

Web20expo Scalable Web Archroyans

Web20expo Scalable Web Archmclee

Recently uploaded (20)

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

This is a Quick Research Guide (QRG). QRGs include the following: - A brief, high-level overview of the QRG topic. - A milestone timeline for the QRG topic. - Links to various free online resource materials to provide a deeper dive into the QRG topic. - Conclusion and a recommendation for at least two books available in the SJPL system on the QRG topic. QRGs planned for the series: - Artificial Intelligence QRG - Quantum Computing QRG - Big Data Analytics QRG - Spacecraft Guidance, Navigation & Control QRG (coming 2026) - UK Home Computing & The Birth of ARM QRG (coming 2027) Any questions or comments? - Please contact Arthur Morgan at [email protected]. 100% human made.

Mobile App Development Company in Saudi ArabiaSteve Jonas

EmizenTech is a globally recognized software development company, proudly serving businesses since 2013. With over 11+ years of industry experience and a team of 200+ skilled professionals, we have successfully delivered 1200+ projects across various sectors. As a leading Mobile App Development Company In Saudi Arabia we offer end-to-end solutions for iOS, Android, and cross-platform applications. Our apps are known for their user-friendly interfaces, scalability, high performance, and strong security features. We tailor each mobile application to meet the unique needs of different industries, ensuring a seamless user experience. EmizenTech is committed to turning your vision into a powerful digital product that drives growth, innovation, and long-term success in the competitive mobile landscape of Saudi Arabia.

DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxJustin Reock

Building 10x Organizations with Modern Productivity Metrics 10x developers may be a myth, but 10x organizations are very real, as proven by the influential study performed in the 1980s, ‘The Coding War Games.’ Right now, here in early 2025, we seem to be experiencing YAPP (Yet Another Productivity Philosophy), and that philosophy is converging on developer experience. It seems that with every new method we invent for the delivery of products, whether physical or virtual, we reinvent productivity philosophies to go alongside them. But which of these approaches actually work? DORA? SPACE? DevEx? What should we invest in and create urgency behind today, so that we don’t find ourselves having the same discussion again in a decade?

Splunk Security Update | Public Sector Summit Germany 2025Splunk

Cyber Awareness overview for 2025 month of securityriccardosl1

Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan

IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...organizerofv

Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Impelsys Inc.

How analogue intelligence complements AIPaul Rowe

Artificial Intelligence is providing benefits in many areas of work within the heritage sector, from image analysis, to ideas generation, and new research tools. However, it is more critical than ever for people, with analogue intelligence, to ensure the integrity and ethical use of AI. Including real people can improve the use of AI by identifying potential biases, cross-checking results, refining workflows, and providing contextual relevance to AI-driven results. News about the impact of AI often paints a rosy picture. In practice, there are many potential pitfalls. This presentation discusses these issues and looks at the role of analogue intelligence and analogue interfaces in providing the best results to our audiences. How do we deal with factually incorrect results? How do we get content generated that better reflects the diversity of our communities? What roles are there for physical, in-person experiences in the digital world?

Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma

Electronic_Mail_Attacks-1-35.pdf by xploitniftliyevhuseyn

SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfPrecisely

TrsLabs - Fintech Product & Business ConsultingTrs Labs

Hybrid Growth Mandate Model with TrsLabs Strategic Investments, Inorganic Growth, Business Model Pivoting are critical activities that business don't do/change everyday. In cases like this, it may benefit your business to choose a temporary external consultant. An unbiased plan driven by clearcut deliverables, market dynamics and without the influence of your internal office equations empower business leaders to make right choices. Getting things done within a budget within a timeframe is key to Growing Business - No matter whether you are a start-up or a big company Talk to us & Unlock the competitive advantage

Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul

Artificial intelligence is changing how businesses operate. Companies are using AI agents to automate tasks, reduce time spent on repetitive work, and focus more on high-value activities. Noah Loul, an AI strategist and entrepreneur, has helped dozens of companies streamline their operations using smart automation. He believes AI agents aren't just tools—they're workers that take on repeatable tasks so your human team can focus on what matters. If you want to reduce time waste and increase output, AI agents are the next move.

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Want to learn practical tips for designing systems that can scale efficiently without compromising speed? Join us for a workshop where we’ll address these challenges head-on and explore how to architect low-latency systems using Rust. During this free interactive workshop oriented for developers, engineers, and architects, we’ll cover how Rust’s unique language features and the Tokio async runtime enable high-performance application development. As you explore key principles of designing low-latency systems with Rust, you will learn how to: - Create and compile a real-world app with Rust - Connect the application to ScyllaDB (NoSQL data store) - Negotiate tradeoffs related to data modeling and querying - Manage and monitor the database for consistently low latencies

Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada

Book industry standards are evolving rapidly. In the first part of this session, we’ll share an overview of key developments from 2024 and the early months of 2025. Then, BookNet’s resident standards expert, Tom Richardson, and CEO, Lauren Stewart, have a forward-looking conversation about what’s next. Link to recording, presentation slides, and accompanying resource: https://bnctechforum.ca/sessions/standardsgoals-for-2025-standards-certification-roundup/ Presented by BookNet Canada on May 6, 2025 with support from the Department of Canadian Heritage.

Massive Power Outage Hits Spain, Portugal, and France: Causes, Impact, and On...Aqusag Technologies

Quantum Computing Quick Research Guide by Arthur MorganArthur Morgan

Mobile App Development Company in Saudi ArabiaSteve Jonas

DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxJustin Reock

Splunk Security Update | Public Sector Summit Germany 2025Splunk

Cyber Awareness overview for 2025 month of securityriccardosl1

Big Data Analytics Quick Research Guide by Arthur MorganArthur Morgan

IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...organizerofv

Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Impelsys Inc.

How analogue intelligence complements AIPaul Rowe

Role of Data Annotation Services in AI-Powered ManufacturingAndrew Leo

Into The Box Conference Keynote Day 1 (ITB2025)Ortus Solutions, Corp

Linux Professional Institute LPIC-1 Exam.pdfRHCSA Guru

Generative Artificial Intelligence (GenAI) in BusinessDr. Tathagat Varma

Electronic_Mail_Attacks-1-35.pdf by xploitniftliyevhuseyn

SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfPrecisely

TrsLabs - Fintech Product & Business ConsultingTrs Labs

Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul

Designing Low-Latency Systems with Rust and ScyllaDB: An Architectural Deep DiveScyllaDB

Transcript: #StandardsGoals for 2025: Standards & certification roundup - Tec...BookNet Canada

WebWorkersCamp 2010

1. Click & Tune What’s new in our architectures ? Olivier Gutknecht [email protected] / twitter.com/olg

2. Me Apple alumnus Fotopedia co-founder & alumnus Author, french ebook - introduction to REST http://izibook.eyrolles.com/

3. Me Apple alumnus Fotopedia co-founder & alumnus Author, french ebook - introduction to REST http://izibook.eyrolles.com/

4. Click & Tune ?

6. START

8. Getting in tune

9. Getting in tune We have a lot of sliders, faders and buttons to tweak. And we often ignore most of them. Especially at architecture level.

10. What changed ?

11. What changed ? Concepts | Context | Implementations

12. Concepts | Context | Implementations “The Google File System” / Google 2003 “MapReduce: Simpliﬁed Data Processing on Large Clusters” / Google 2004 “Life beyond Distributed Transactions: an Apostate’s Opinion” / Pat Helland, Amazon 2007 [ZFS, Dynamo, PNUTS, CAP, REST,...] papers

13. Concepts | Context | Implementations Client and apps evolution: html5, native apps, COMET, web of [docs,apps], ... Modeling problems: tags, social graphs, walls, feeds, ... “Like the big boys” - brought legitimacy

14. Concepts | Context | Implementations Nutch - 2003 Memcached - 2003 Rails - 2004 CouchDB - 2005 S3 & EC2 - 2006 Redis, RabbitMQ, MongoDB, Node.JS...

16. Sliders ? # of backend servers cache size fsync / checkpointing invalidation delay

17. interesting, but...

18. Sliders Consistency level Sync/Async mix R/W actions balance Model heterogeneity

19. Trade-offs

21. It’s the app, really. And use cases. And business cases.

22. Latency “Disk is the new tape” Cache ? invalidation & cold starts ...

23. Read Reads are a memory & locality problem. Caches. Partitions. Dupes. Smart shards.

24. Write Write now. Work later. (maybe not on reads) Divide and Conquer. And cheat. Our frameworks are not ready yet.

25. Trade-offs

26. Acceptable Latency Mandatory consistency (and transactions) Synchronous actions Maintainability (Cost)

27. NoSQL is not about No SQL. It’s about revisiting assumptions. CAP ? ACID ? Choose your poisons. (and enjoy your sliders, yay for Cassandra)

28. Consistency Poison #1

29. Partition Tolerance Poison #2

30. Availability

31. Ditto for async.

32. The state of art is not good enough. (we’ve been stuck in the same local minima for several years) Ilya Grigorik

33. Yay for node.js - and Grand Central - and EM - and Goliath - and JRuby - and ...

34. Ditto for batches.

35. Hadoop. Pig. Hive. Sqoop. Flume. HBase.

36. So ?

37. Full Stack

40. HTTP everywhere ? & AMQP ? & Thrift ? ProtoBufs ? Avro ?

41. AUT OMA

42. Because our stacks are state of the art (or bleeding edge) Beyond Packages Global conﬁg Change Do you really want to ssh to your servers ?

43. “But we have very speciﬁc needs”

44. You’re 10% unique Jesse Robbins, Opscode CEO

45. You’re 10% unique (and it’s probably the things you did wrong) Jesse Robbins, Opscode CEO

46. The Cloud What’s great about it ? Scalability ? On-demand capacity ? Reduced CapEx ?

47. “Everything fails all the time” Werner Vogels (Amazon CTO)

48. The Cloud The best way to discover new problems. (and it’s great)

49. The Cloud Think stateless. Think automation. Think latency. Think crappy machines. Think lost servers. Thinks lost disks. Think ofﬂine datacenters. Think ACL. Think security. Think autoconf.

50. “MyPAAS”

51. Think Holistic

52. Speed. Availability. Maintainability.

53. Know the limits of your tools

54. Episode IV: a New Hope C h e f , P u p p e t , D r i z z l e , C a s s a n d ra , MongoDB, Redis, HTTP, Rails, Node.js, EventMachine, Twisted, Heroku, SimpleDB, EC2, GAE, REST, Rack, CouchDB, Redis, Bitcask, Voldemort, Varnish, Pig, Hadoop, HBase, Thrift, Protobufs, Avro, ESI, Mahout, SolR, AMQP, Grand Central Dispatch, WebSockets, LINQ, Reconnoiter, Nginx, ...

55. Thanks. Creative Contents Images by Big Red Button by thirtyfootscrew - http://www.flickr.com/photos/thirtyfootscrew/3324101122/ The Steam Pipeline by stuck in customs - http://www.flickr.com/photos/stuckincustoms/3780481716 Digidesign Icon Faders by CJ Sorg - http://www.flickr.com/photos/cjsorg/3692124539/ Pro Control 24 by aud1073ch - qhttp://www.flickr.com/photos/aud1073ch/34438156/ Pipe Dream by Rishi Menon - http://www.flickr.com/photos/rxmflickr/4102530508/

WebWorkersCamp 2010

Recommended

More Related Content

Viewers also liked (6)

Similar to WebWorkersCamp 2010 (20)

Recently uploaded (20)

WebWorkersCamp 2010