Shift Remote: JS - Node.js Scalability Tips - Luciano Mammino (FabFitFun)

1. Node.js Scalability Tips Luciano Mammino ( )@loige 2020-07-09 loige.link/node-scale 1

2. 👋 Hello, I am Luciano! Principal Software Engineer at FabFitFun Blog: Twitter: GitHub: loige.co @loige @lmammino nodejsdp.link 2

3. @loige Get the slides! loige.link/node-scale 3

4. Node.js + Scalability? 🤔 @loige 4

5. @loige 5

6. @loige 6

7. @loige 6

8. @loige 6

9. @loige 6

10. @loige 7

11. @loige 7

12. @loige 7

13. @loige 7

14. 🤣 @loige 7

15. "Scalability is the property of a system to handle a growing amount of work by adding resources to the system" — Wikipedia @loige 8

16. "A service is said to be scalable if when we increase the resources in a system, it results in increased performance in a manner proportional to resources added" — Werner Vogels @loige 9

17. 🛫 Tip 1. Establish a baseline @loige 10

18. /?data=Hello%20Shift https://repl.it/@lmammino/QRGen @loige 11

19. const { createServer } = require('http') const { URL } = require('url') const QRCode = require('qrcode') createServer(function handler (req, res) { const url = new URL(req.url, 'http://localhost:8080') const data = url.searchParams.get('data') if (!data) { res.writeHead(400) // bad request return res.end() } res.writeHead(200, { 'Content-Type': 'image/png' }) QRCode.toFileStream(res, data, { width: 300 }) }) .listen(8080) @loige 12

20. autocannon -c 200 --on-port / -- node server.js wrk node server.js& wrk -t8 -c200 -d10s http://localhost:8080/ @loige 13

21. autocannon -c 200 --on-port /?data=Hello%20Shift -- node server.js Running 10s test @ http://localhost:8080/?data=Hello%20Shift 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 14

22. autocannon -c 200 --on-port /?data=Hello%20Shift -- node server.js Running 10s test @ http://localhost:8080/?data=Hello%20Shift 200 connections ┌─────────┬─────────┬─────────┬─────────┬─────────┬────────────┬─────────┬────────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼─────────┼─────────┼─────────┼─────────┼────────────┼─────────┼────────────┤ │ Latency │ 1899 ms │ 1951 ms │ 2053 ms │ 2054 ms │ 1964.92 ms │ 99.9 ms │ 3364.03 ms │ └─────────┴─────────┴─────────┴─────────┴─────────┴────────────┴─────────┴────────────┘ ┌───────────┬─────┬──────┬─────────┬────────┬────────┬────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Req/Sec │ 0 │ 0 │ 30 │ 199 │ 99.5 │ 94.27 │ 30 │ ├───────────┼─────┼──────┼─────────┼────────┼────────┼────────┼─────────┤ │ Bytes/Sec │ 0 B │ 0 B │ 50.7 kB │ 336 kB │ 168 kB │ 159 kB │ 50.7 kB │ └───────────┴─────┴──────┴─────────┴────────┴────────┴────────┴─────────┘ Req/Bytes counts sampled once per second. 995 requests in 10.08s, 1.68 MB read @loige 14

23. ⛅ Tip 1-bis Also, find out your ceiling @loige 15

24. const { createServer } = require('http') createServer((req, res) => { if (req.method === 'GET' && req.url === '/') { res.writeHead(200, { 'Content-Type': 'text/plain' }) res.end('Hello Worldn') } else { res.statusCode = 404 res.end() } }) .listen(8080) @loige 16

25. autocannon -c 200 --on-port / -- node server.js Running 10s test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 17

26. autocannon -c 200 --on-port / -- node server.js Running 10s test @ http://localhost:8080/ 200 connections ┌─────────┬──────┬──────┬───────┬───────┬─────────┬─────────┬──────────┐ │ Stat │ 2.5% │ 50% │ 97.5% │ 99% │ Avg │ Stdev │ Max │ ├─────────┼──────┼──────┼───────┼───────┼─────────┼─────────┼──────────┤ │ Latency │ 3 ms │ 5 ms │ 11 ms │ 14 ms │ 5.51 ms │ 2.71 ms │ 80.63 ms │ └─────────┴──────┴──────┴───────┴───────┴─────────┴─────────┴──────────┘ ┌───────────┬─────────┬─────────┬────────┬─────────┬─────────┬─────────┬─────────┐ │ Stat │ 1% │ 2.5% │ 50% │ 97.5% │ Avg │ Stdev │ Min │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Req/Sec │ 21087 │ 21087 │ 34623 │ 35487 │ 33258.4 │ 4107.01 │ 21077 │ ├───────────┼─────────┼─────────┼────────┼─────────┼─────────┼─────────┼─────────┤ │ Bytes/Sec │ 3.29 MB │ 3.29 MB │ 5.4 MB │ 5.54 MB │ 5.19 MB │ 641 kB │ 3.29 MB │ └───────────┴─────────┴─────────┴────────┴─────────┴─────────┴─────────┴─────────┘ Req/Bytes counts sampled once per second. 333k requests in 10.1s, 51.9 MB read @loige 17

27. 🍾 Tip 2. Find your bottleneck @loige 18

28. Clinic.js clinicjs.org @loige 19

29. clinic doctor --autocannon [ -c 200 '/?data=Hello%20Shift' ] -- node server.js @loige 20

30. clinic flame --autocannon [ -c 200 '/?data=Hello%20Shift' ] -- node server.js @loige 21

31. clinic bubble --autocannon [ -c 200 '/?data=Hello%20Shift' ] -- node server.js @loige 22

32. 🎳 Tip 3. Understand your goals @loige 23

33. What do we optimize for? Throughput? Memory? Latency? @loige 24

34. 👁 Tip 4. Always "observe" @loige 25

35. I mean, in production! Logs - Metrics - Traces @loige 26

36. 🚀 Tip 5. Scale your architecture @loige 27

37. Performance != Scalability @loige 28

38. How can we scale a system by adding resources? @loige 29

39. The " "Scale Cube x-axis cloning z-axis partitioning y-axis functional decomposition @loige 30

40. The " "Scale Cube x-axis cloning z-axis partitioning y-axis functional decomposition @loige 30

41. Cloning Reverse proxy 31 Inside the same server Load Balancer Using multiple server @loige

42. The modulecluster Master process Worker process Worker process Worker process 32 @loige

49. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } @loige 33

50. const cluster = require('cluster') const numCPUs = require('os').cpus().length if (cluster.isMaster) { // Fork workers for (let i = 0; i < numCPUs; i++) { cluster.fork() } } else { // Worker code require('./server.js') } 3-4x req/sec (8 core) @loige 33

51. You could also use Check out ! Worker Threads piscina @loige 34

52. Cloning is the easiest strategy to scale a service... ... as long as your application is Stateless @loige 35

53. API Gateway Functional decomposition a.k.a. "Micro-services" 36 /products /cart cart DB products DB @loige

54. API Gateway Functional decomposition a.k.a. "Micro-services" 37 /products /cart Functional decomposition can also be combined with cloning! cart DB products DB @loige

55. Node.js is great for microservices @loige 38

56. Microservices can also help with scaling the organisation! @loige 39

57. Microservices add complexity Observability Deployments Versioning Integration @loige 40

58. Partitioning Service and Data Partitioning along Customer Boundaries Shard partitioning /products/[A-L]/ /products/[M-Z]/ DB 2 41 DB 1 @loige

59. Partitioning is generally used to scale databases and SaaS software geographically @loige 42

60. Summary @loige 43

61. Summary 🛫 Establish a baseline @loige 43

62. Summary 🛫 Establish a baseline 🍾 Find your bottleneck @loige 43

63. Summary 🛫 Establish a baseline 🍾 Find your bottleneck 🎳 Understand your goals @loige 43

64. Summary 🛫 Establish a baseline 🍾 Find your bottleneck 🎳 Understand your goals 👁 Always "observe" @loige 43

65. Summary 🛫 Establish a baseline 🍾 Find your bottleneck 🎳 Understand your goals 👁 Always "observe" 🚀 Scale your architecture (cloning, decomposition & partitioning) @loige 43

66. Thank you! Special thanks to , , , , , , , , , , , , , , , , , , , , , Icons and SVGs by @StefanoAbalsamo @matteocollina @dagonzago @NullishCoalesce @DublinSvelte @KViglucci @gjohnson391 @lucamaraschi @laurekamalandua @giltayar @mrm8488 @adrirai @haraﬁse @EugeneWare @Jauny @tlivings @michaelcﬁne @leojino @shahidontech @Lordoomer @zsadigov @dottorblaster freepik.com loige.link/node-scale @loige 44

Shift Remote: JS - Node.js Scalability Tips - Luciano Mammino (FabFitFun)

More Related Content

Similar to Shift Remote: JS - Node.js Scalability Tips - Luciano Mammino (FabFitFun) (20)

More from Shift Conference (20)

Recently uploaded (20)

Shift Remote: JS - Node.js Scalability Tips - Luciano Mammino (FabFitFun)