Building a Node.JS accelerator for your headless Drupal backend - DrupalCamp London 2015
7 likes2,976 views
This document discusses using Node.JS to accelerate a headless Drupal backend. It explains that headless Drupal involves using Drupal only for content management and delivering content via JSON. While caching can help performance for anonymous users, authenticated users pose challenges at scale. The proposed solution is to index Drupal content into MongoDB using a module and deliver it from Node.JS, gaining significant performance increases over delivering directly from Drupal. An example real-world video platform project is discussed that migrated from Drupal to this architecture for improved performance when integrating with mobile and TV apps.
Building a Node.JS accelerator for your headless Drupal backend - DrupalCamp London 2015
- 1. Building a Node.JS accelerator for your headless Drupal backend Kalle Varisvirta @kvirta Technology Director Exove @exove
- 2. Pre-requisites Moderate understanding of PHP and Javascript Moderate understanding of Drupal’s internals Some idea of how NoSQL, SOLR and Varnish work
- 3. What you’ll learn What’s headless Drupal Why use headless Drupal How to speed up headless Drupal content delivery with Node.JS in real life Why to speed up Drupal content delivery with Node.JS
- 4. About me Kalle Varisvirta Technology Director at Exove Exove is a technology company based in Finland, Estonia and UK with over 70 professionals
- 6. Drupal Typically Drupal gets a HTTP request and responds with an HTML page Theming is done in Drupal No front-end framework is used Drupal User
- 7. Headless Drupal When you add a front-end framework and let it do the theming, you get headless Drupal Drupal will then deliver content in JSON Drupal User Front-end framework
- 8. Why headless Drupal? If you want full control over the display of your content If you want to create a app-like experience If you want to create an app :-)
- 9. Why not headless Drupal? More manual labor than regular Drupal Half of Drupal’s functionality is ignored, as it won’t affect how the content is displayed
- 10. Headless Drupal App, you say? When just delivering content, Drupal doesn’t care what happens downstream Drupal iOS app Android app Smart TV app Drupal site
- 11. Headless Drupal Drupal is delivering content in JSON via HTTP, so caching is a valid option for anonymous users Using a Varnish layer to cache the JSON will make your site fast enough for most setups Drupal Varnish iOS app Android app Smart TV app Drupal site
- 12. Authenticated users Authenticated users in Drupal, especially with long content listings from Views, makes things complicated If you have a lot of authenticated users concurrently using your headless Drupal, you’ll be in trouble fast
- 13. Solution
- 14. The architecture
- 15. Drupal content is indexed on every change using the MongoDB indexer module (drupal.org/project/ mongodb_indexer)
- 16. We’re using MongoDB here as a storage We could be using any NoSQL, including search platforms like Elasticsearch or Apache SOLR We prefer a real database for fast indexing
- 17. For delivery, we opt for Node.JS A framework like Express or Koa is suggested for better structure
- 18. A deeper look
- 19. Drupal Focusing solely in content management, not delivering any web pages outside the admin
- 20. MongoDB indexer MongoDB indexer is just a small module we created to index Drupal’s content outside, without interfering the storage engine of Drupal
- 21. MongoDB indexer MongoDB indexer is just a small module we created to index Drupal’s content outside, without interfering the storage engine of Drupal
- 22. MongoDB We chose MongoDB over other NoSQL systems due to the popularity and fast development It’s had its issues, but they’ve fixed them all Now expecting Wired Tiger storage engine that’ll improve the locking mechanism
- 23. Delivery We have optimized data in the MongoDB We want to deliver it out, fast We want to be able to do some logic while delivering it
- 24. Node.JS Node.JS is Javascript running on the server The Javascript is ran by Chrome’s Javascript engine, the V8 It’s non-blocking, event-based and when used correctly, blazing fast
- 25. RPS Avg response time Drupal Node.JS
- 26. Node.JS Cluster (npm module) Forever (npm module)
- 28. Node.JS For a PHP programmer, it’s quite a change Due to the you need to understand parallel programming There are npm modules there to help with that, too During the lifetime our big project (24 months) both Node.JS and MongoDB have evolved quite a bit
- 29. Node.JS The code is very simple as it’s mostly just passing information out from the MongoDB If you’re building a REST api, as you should, you can add filters to the calls and pass them on to the MongoDB But… you can also check for user session and fetch user-related content
- 30. REST API REST API allows writing, too, so you’ll have to handle the writes Most of writing in these cases are left to the Node app -level Obviously, writes reaching the Drupal are also possible
- 31. Real world case?
- 32. Video platform project Started three years ago Started as a headless Drupal project Originally was supposed to feed content straight from Drupal
- 33. Drupal optimizations It’s not like we didn’t think of the performance early on The Drupal 7 was built on MongoDB field storage and thus standing on a fast database The Views feeds (JSON) were coming from SOLR backend
- 34. Caching feeds Downstream clients want integration feeds limited by time Time attribute by seconds of the last fetch To deal with existing content To deal with changed content We tried to cache to Varnish, it didn’t work at all
- 35. We need a new approach Indexing outside of Drupal felt like the only way out First we decided we’ll go with SOLR via ApacheSOLR integration Distribution by a simple REST API Later on, we wrote the MongoDB indexer and switched to MongoDB fully
- 36. Video content management system (Drupal 7) Linear television data (ERP) Drupal 7 site Drupal 7 site Wordpress site iOS app Android app Samsung SmartTV app MongoDB and Node.JS RESTAPI Indexing is done using a Drupal module, MongoDB indexer
- 37. Recap
- 38. Recap Headless Drupal is when you use Drupal only for the data and theme/display somewhere else Headless Drupal needs performance that Drupal might not be able to provide MongoDB indexer will index your Drupal entities to MongoDB From MongoDB, it’s easy to distribute your data with Node.JS and enjoy the high performance