SlideShare a Scribd company logo
Realtime at Lore
                      Using Node.js and Socket.io




                      Dan Getelman — May 9, 2012




Tuesday, May 15, 12
Why am I here?

                      • CTO/Co-founder at Lore (formerly
                        Coursekit)

                      • We built a system where real-time
                        updates are global. It’s been happily
                        running in production for about a year.




Tuesday, May 15, 12
Outline

                      • What we have
                      • How we ended up there
                      • How it works
                      • Lessons learned
                      • Miscellaneous Thoughts

Tuesday, May 15, 12
What’s our stack?

                      • Rich front-end framework
                      • node.js + socket.io for real-time
                        communications

                      • Python API

Tuesday, May 15, 12
How did we end up
                             here?
                      • Build best experience possible
                      • Strong vision for what we wanted —
                        solving general case way less work
                        than tons of specifics

                      • Able to leverage infrastructure both for
                        speed gains and real-time updating



Tuesday, May 15, 12
How did we end up
                             here?

                      • Sat down last summer, built it
                      • Pleasantly surprised by ease of
                        socket.io + Redis




Tuesday, May 15, 12
Goals

                      • Build awesome experience for
                        teachers/students

                      • Performance
                      • Secure

Tuesday, May 15, 12
Data Model
                              Restrictions
                      • Permission Groups let us simplify
                        updates significantly

                      • Individual models being pushed to front-
                        end

                      • Originally, circular dependencies not
                        allowed



Tuesday, May 15, 12
Two Browser
                       Connections
                 API                 Push


                        HTTPS
                                 WebSocket
                                 / Fallback



Tuesday, May 15, 12
Beauty of Push
                              Server
                      • Fills two roles:
                       • Lets us bypass browser concurrent
                          requests limits

                       • Lets us push any changes relevant to
                          a user down the pipe

                      • Socket.io generally lets us do this
                        without worrying about browsers


Tuesday, May 15, 12
Redis — Wonderchild

                      • Used in three very different ways,
                        generally happy with all

                      • Message queue
                      • Cache
                      • Pubsub

Tuesday, May 15, 12
How It Fits Together

                      • Subscribe to models (message via
                        socket.io)

                      • On write, determine which models care
                        about the change / addition

                      • Publish to their channel in Redis

Tuesday, May 15, 12
Front-end
                              Touchpoints
                      • Receiving and dispatching messages
                        (chat, notifications, updates)

                      • Fetching models
                      • Batch calls to improve performance

Tuesday, May 15, 12
Running in
                              production
                      • Handling HTTPS
                      • Load balancing
                      • Monit
                      • Metrics

Tuesday, May 15, 12
HTTPS

                 https = require 'https'
                 privateKey = fs.readFileSync("key.key").toString()
                 certificate = fs.readFileSync("ssl.pem").toString()

                 ca1 = fs.readFileSync("blah.pem").toString()

                 https.createServer {key:privateKey,cert:certificate, ca:
                 [ca1]}




Tuesday, May 15, 12
Lessons Learned

                      • Log everything
                      • Measure everything
                      • Unstable standards are unstable
                      • socket.io can be a bit opaque if things
                        aren’t going well



Tuesday, May 15, 12
Thanks!


                      • Follow me on Twitter: @dget
                      • We’re hiring!


Tuesday, May 15, 12

More Related Content

Similar to Dan node meetup_socket_talk (20)

PDF
Introducing Immutant
Jim Crossley
 
PDF
How MongoDB is Being Used in China - Case Studies
MongoDB
 
PDF
Qcon talk
bcoverston
 
PDF
static ABAP code analyzers
Markus Theilen
 
PDF
Cloud4all Architecture Overview
icchp2012
 
PDF
MuraCon 2012 - Creating a Mura CMS plugin with FW/1
jpanesar
 
PDF
Phingified ci and deployment strategies ipc 2012
TEQneers GmbH & Co. KG
 
PPTX
Scaling a High Traffic Web Application: Our Journey from Java to PHP
120bi
 
PPTX
Scaling High Traffic Web Applications
Achievers Tech
 
PDF
node.js in action
Karan Misra
 
PDF
Kubernetes Scaling SIG (K8Scale)
KubeAcademy
 
PDF
K8scale update-kubecon2015
Bob Wise
 
PDF
Measure Everything
Arik Fraimovich
 
PPT
MetaZeta Clusters Overview
Paul Baclace
 
PDF
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
JBossArchitectForum
 
PDF
Architecting for failure
Michael Brunton-Spall
 
PPTX
“Scaling with LeSS” by Kārlis Cinis from Tieto Latvia at Large Scale Agile fo...
DevClub_lv
 
PPTX
Getting out of the monolith hell
mimmozzo_
 
PDF
Front end performance improvements
Matthew Farina
 
PPT
IBM and Node.js - Old Doge, New Tricks
Dejan Glozic
 
Introducing Immutant
Jim Crossley
 
How MongoDB is Being Used in China - Case Studies
MongoDB
 
Qcon talk
bcoverston
 
static ABAP code analyzers
Markus Theilen
 
Cloud4all Architecture Overview
icchp2012
 
MuraCon 2012 - Creating a Mura CMS plugin with FW/1
jpanesar
 
Phingified ci and deployment strategies ipc 2012
TEQneers GmbH & Co. KG
 
Scaling a High Traffic Web Application: Our Journey from Java to PHP
120bi
 
Scaling High Traffic Web Applications
Achievers Tech
 
node.js in action
Karan Misra
 
Kubernetes Scaling SIG (K8Scale)
KubeAcademy
 
K8scale update-kubecon2015
Bob Wise
 
Measure Everything
Arik Fraimovich
 
MetaZeta Clusters Overview
Paul Baclace
 
JBoss Architect Meetup - December 2013 - JBoss Fuse in Vodafone’s Global Inte...
JBossArchitectForum
 
Architecting for failure
Michael Brunton-Spall
 
“Scaling with LeSS” by Kārlis Cinis from Tieto Latvia at Large Scale Agile fo...
DevClub_lv
 
Getting out of the monolith hell
mimmozzo_
 
Front end performance improvements
Matthew Farina
 
IBM and Node.js - Old Doge, New Tricks
Dejan Glozic
 

Dan node meetup_socket_talk

  • 1. Realtime at Lore Using Node.js and Socket.io Dan Getelman — May 9, 2012 Tuesday, May 15, 12
  • 2. Why am I here? • CTO/Co-founder at Lore (formerly Coursekit) • We built a system where real-time updates are global. It’s been happily running in production for about a year. Tuesday, May 15, 12
  • 3. Outline • What we have • How we ended up there • How it works • Lessons learned • Miscellaneous Thoughts Tuesday, May 15, 12
  • 4. What’s our stack? • Rich front-end framework • node.js + socket.io for real-time communications • Python API Tuesday, May 15, 12
  • 5. How did we end up here? • Build best experience possible • Strong vision for what we wanted — solving general case way less work than tons of specifics • Able to leverage infrastructure both for speed gains and real-time updating Tuesday, May 15, 12
  • 6. How did we end up here? • Sat down last summer, built it • Pleasantly surprised by ease of socket.io + Redis Tuesday, May 15, 12
  • 7. Goals • Build awesome experience for teachers/students • Performance • Secure Tuesday, May 15, 12
  • 8. Data Model Restrictions • Permission Groups let us simplify updates significantly • Individual models being pushed to front- end • Originally, circular dependencies not allowed Tuesday, May 15, 12
  • 9. Two Browser Connections API Push HTTPS WebSocket / Fallback Tuesday, May 15, 12
  • 10. Beauty of Push Server • Fills two roles: • Lets us bypass browser concurrent requests limits • Lets us push any changes relevant to a user down the pipe • Socket.io generally lets us do this without worrying about browsers Tuesday, May 15, 12
  • 11. Redis — Wonderchild • Used in three very different ways, generally happy with all • Message queue • Cache • Pubsub Tuesday, May 15, 12
  • 12. How It Fits Together • Subscribe to models (message via socket.io) • On write, determine which models care about the change / addition • Publish to their channel in Redis Tuesday, May 15, 12
  • 13. Front-end Touchpoints • Receiving and dispatching messages (chat, notifications, updates) • Fetching models • Batch calls to improve performance Tuesday, May 15, 12
  • 14. Running in production • Handling HTTPS • Load balancing • Monit • Metrics Tuesday, May 15, 12
  • 15. HTTPS https = require 'https' privateKey = fs.readFileSync("key.key").toString() certificate = fs.readFileSync("ssl.pem").toString() ca1 = fs.readFileSync("blah.pem").toString() https.createServer {key:privateKey,cert:certificate, ca: [ca1]} Tuesday, May 15, 12
  • 16. Lessons Learned • Log everything • Measure everything • Unstable standards are unstable • socket.io can be a bit opaque if things aren’t going well Tuesday, May 15, 12
  • 17. Thanks! • Follow me on Twitter: @dget • We’re hiring! Tuesday, May 15, 12