Node, can you even in CPU intensive operations?
0 likes623 views
The document discusses the challenges of handling CPU-intensive operations in Node.js, highlighting the differences between blocking and non-blocking I/O operations as well as the role of the event loop. It introduces techniques like clustering and worker threads to leverage multi-core systems and perform parallel computations. The document concludes by suggesting that while Node.js is effective for I/O-bound tasks, it may not be the best choice for heavy computational tasks.
Node, can you even in CPU intensive operations?
- 1. NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 3. WHAT'S THE PLAN? ▸ SINGLE THREADED? ▸ NON BLOCKING I/O OPERATIONS ▸ PERFORMANCE TRAPS ▸ CPU-INTENSIVE OPERATIONS WITHOUT KILLING OUR APPS
- 4. ▸ HTTP REQUESTS ▸ DISK READS/WRITES ▸ DB I/O OPERATIONS NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 5. BLOCKING VS NON- BLOCKING I/O
- 6. ▸ NEXT OPERATIONS HAVE TO WAIT ▸ APPLICATION DOES NOTHING WHILE WAITING FOR THE RESULT ▸ TOP-DOWN - SYNCHRONOUS ▸ TIME = FUNCTION'S CALL + OPERATION’S TIME ▸ EXAMPLE: readFileSync BLOCKING I/O NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 8. ▸ NEXT OPERATIONS DON'T HAVE TO WAIT ▸ ASYNC ▸ CALLBACKS! ▸ TIME = FUNCTION'S CALL + CALLBACK'S HANDLING ▸ EXAMPLE: readFile NON-BLOCKING I/O NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 10. EVENT LOOP
- 11. “THE EVENT LOOP IS WHAT ALLOWS NODE.JS TO PERFORM NON-BLOCKING I/O OPERATIONS, DESPITE THE FACT THAT JAVASCRIPT IS SINGLE-THREADED, BY OFFLOADING OPERATIONS TO THE SYSTEM KERNEL WHENEVER POSSIBLE.”
- 12. ▸ SIMPLIFIED EXPLANATION! ▸ IMPLEMENTED IN LIBUV ▸ EVENT LOOP IS, WELL, AN EVENT LOOP ▸ STARTED WHEN NODE PROCESS STARTS ▸ ENDS WHEN THERE'S NOTHING MORE TO PROCESS EVENT LOOP NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 14. ▸ SYNCHRONOUS CODE IS EASY ▸ ASYNC NOT SO MUCH - ESPECIALLY IN A SINGLE THREADED ENVIRONMENT ▸ CALLBACKS MUST BE HANDLED SOMEHOW ▸ CAN'T HAPPEN ANYTIME ▸ WE NEED A MECHANISM THAT WILL QUEUE THESE CALLBACKS EVENT LOOP, WHAT FOR? NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 16. BENCHMARKS NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ API ▸ /userBlocking ▸ /userNonBlocking ▸ USER READ TIME = 75ms ▸ APACHE BENCHMARK: -c 10 -t 5
- 19. LIFE'S GOOD NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ API RUNNING HAPPILY ON PROD ▸ NON-BLOCKING I/O ONLY ▸ AWESOME PERFORMANCE OUT OF THE BOX ▸ $$$$
- 20. NEW REQUIREMENTS
- 22. NEW REQUIREMENTS NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ NEW ENDPOINT FOR SOME HYPER COMPLICATED CALCULATIONS ▸ CONSUMED BY OTHER TEAM ▸ BATCH JOB ▸ OUR ALGORITHM WILL PROCESS SOME DATA ▸ WE'LL EXPOSE THE ENDPOINT IN OUR EXISTING API
- 23. DEPLOY
- 27. LET’S FIND (AND HOPEFULLY FIX) THE PROBLEM
- 28. LET’S FIND THE PROBLEM NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ HIT CPU-CONSUMING ENDPOINT IN A LOOP ▸ RUN THE SAME BENCHMARK AS BEFORE (ab -c 10 -t 5)
- 31. CLUSTER MODE
- 32. "A SINGLE INSTANCE OF NODE.JS RUNS IN A SINGLE THREAD. TO TAKE ADVANTAGE OF MULTI-CORE SYSTEMS, THE USER WILL SOMETIMES WANT TO LAUNCH A CLUSTER OF NODE.JS PROCESSES TO HANDLE THE LOAD."
- 33. ▸ WE'LL HARNESS OUR CPU'S FULL POWER ▸ LOAD BALANCER OUT OF THE BOX ▸ LET'S IMPLEMENT IT! CLUSTER MODE NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 34. const cluster = require('cluster'); const numCPUs = require('os').cpus().length; if (cluster.isMaster) { console.log(`Master ${process.pid} is running`); for (let i = 0; i < numCPUs; i++) { cluster.fork(); } cluster.on('exit', (worker, code, signal) => { console.log(`worker ${worker.process.pid} died`); }); } else { // run http server console.log(`Process ${process.pid} started`); }
- 35. const cluster = require('cluster'); const numCPUs = require('os').cpus().length; if (cluster.isMaster) { console.log(`Master ${process.pid} is running`); for (let i = 0; i < numCPUs; i++) { cluster.fork(); } cluster.on('exit', (worker, code, signal) => { console.log(`worker ${worker.process.pid} died`); }); } else { // run http server console.log(`Process ${process.pid} started`); }
- 36. const cluster = require('cluster'); const numCPUs = require('os').cpus().length; if (cluster.isMaster) { console.log(`Master ${process.pid} is running`); for (let i = 0; i < numCPUs; i++) { cluster.fork(); } cluster.on('exit', (worker, code, signal) => { console.log(`worker ${worker.process.pid} died`); }); } else { // run http server console.log(`Process ${process.pid} started`); }
- 37. const cluster = require('cluster'); const numCPUs = require('os').cpus().length; if (cluster.isMaster) { console.log(`Master ${process.pid} is running`); for (let i = 0; i < numCPUs; i++) { cluster.fork(); } cluster.on('exit', (worker, code, signal) => { console.log(`worker ${worker.process.pid} died`); }); } else { // run http server console.log(`Process ${process.pid} started`); }
- 41. DEPLOY
- 45. LET’S FIND (AND FIX) THE PROBLEM
- 46. LET’S FIND THE PROBLEM (AGAIN) NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ HIT CPU-CONSUMING ENDPOINT IN A LOOP, BUT IN PARALLEL THIS TIME (5 SCRIPTS SIMULTANEOUSLY) ▸ RUN THE SAME BENCHMARK AS BEFORE (ab -c 10 -t 5) on /userNonBlocking
- 49. WORKER THREADS
- 50. "WORKERS (THREADS) ARE USEFUL FOR PERFORMING CPU-INTENSIVE JAVASCRIPT OPERATIONS. THEY WILL NOT HELP MUCH WITH I/O-INTENSIVE WORK. NODE.JS’S BUILT-IN ASYNCHRONOUS I/O OPERATIONS ARE MORE EFFICIENT THAN WORKERS CAN BE."
- 51. WORKER THREADS NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ ADDED IN NODE 10.5 ▸ ALLOWS TO EXECUTE JAVASCRIPT IN PARALLEL ▸ STABILITY: EXPERIMENTAL ▸ FLAG --experimental-worker NEEDED FOR NODE < 11.7 ▸ 'worker_threads' MODULE
- 53. const {Worker, isMainThread, parentPort} = require('worker_threads') if (isMainThread) { module.exports = async function timeConsumingOperationOnThreads(raw) { return new Promise((resolve, reject) => { const worker = new Worker(__filename, { workerData: raw }); worker.on('message', resolve) worker.on('error', reject) worker.on('exit', (code) => { if (code !== 0) { reject(new Error(`Worker stopped with exit code ${code}`)) } }); }); }; } else { const result = runSuperComplicatedAlgorithm() parentPort.postMessage({ result }) }
- 54. const {Worker, isMainThread, parentPort} = require('worker_threads') if (isMainThread) { module.exports = async function timeConsumingOperationOnThreads(raw) { return new Promise((resolve, reject) => { const worker = new Worker(__filename, { workerData: raw }); worker.on('message', resolve) worker.on('error', reject) worker.on('exit', (code) => { if (code !== 0) { reject(new Error(`Worker stopped with exit code ${code}`)) } }); }); }; } else { const result = runSuperComplicatedAlgorithm() parentPort.postMessage({ result }) }
- 55. const {Worker, isMainThread, parentPort} = require('worker_threads') if (isMainThread) { module.exports = async function timeConsumingOperationOnThreads(raw) { return new Promise((resolve, reject) => { const worker = new Worker(__filename, { workerData: raw }); worker.on('message', resolve) worker.on('error', reject) worker.on('exit', (code) => { if (code !== 0) { reject(new Error(`Worker stopped with exit code ${code}`)) } }); }); }; } else { const result = runSuperComplicatedAlgorithm() parentPort.postMessage({ result }) }
- 56. const {Worker, isMainThread, parentPort} = require('worker_threads') if (isMainThread) { module.exports = async function timeConsumingOperationOnThreads(raw) { return new Promise((resolve, reject) => { const worker = new Worker(__filename, { workerData: raw }); worker.on('message', resolve) worker.on('error', reject) worker.on('exit', (code) => { if (code !== 0) { reject(new Error(`Worker stopped with exit code ${code}`)) } }); }); }; } else { const result = runSuperComplicatedAlgorithm() parentPort.postMessage({ result }) }
- 60. DEPLOY
- 63. ALTERNATIVES
- 64. PARTITIONING NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ COMPLETE PART OF THE WORK => PUSH BACK TO EVENT LOOP ▸ NOT APPLICABLE TO ALL ALGORITHMS ▸ WILL TAKE LONGER TIME ▸ MIGHT NOT BE POSSIBLE WHEN USING EXTERNAL LIBRARIES ▸ HARDER TO REASON ABOUT
- 67. PROCESS.FORK NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ IMPOSSIBLE TO SHARE MEMORY BETWEEN PROCESSES ▸ EXPENSIVE IN TERMS OF RESOURCES ▸ NOT AS LIGHTWEIGHT AS THREADS ▸ WHY?
- 69. PROCESS.FORK CONT. NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ NEW WORKER THREAD * 500 = ~2000 ms ▸ NEW PROCESS * 500 = ~12300 ms
- 70. CHANGE YOUR ARCHITECTURE NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS? ▸ SEPARATE API ▸ MOVE TIME CONSUMING OPERATION TO A SEPARATE SERVICE WITH A QUEUE IN FRONT OF IT ▸ THEY WILL STILL SUFFER FROM LOW PERFORMANCE, JUST WON’T AFFECT OTHER ENDPOINTS
- 71. “(...) HOWEVER, IF YOUR SERVER RELIES HEAVILY ON COMPLEX CALCULATIONS, YOU SHOULD THINK ABOUT WHETHER NODE IS REALLY A GOOD FIT. NODE EXCELS FOR I/O-BOUND WORK, BUT FOR EXPENSIVE COMPUTATION IT MIGHT NOT BE THE BEST OPTION.”
- 72. ▸ FAVOR NON-BLOCKING (ASYNC) I/O OVER BLOCKING OPERATION ▸ VERTICAL SCALING (CLUSTER MODE) MIGHT NOT SOLVE YOUR PROBLEMS WITH CPU-INTENSIVE OPERATIONS... ▸ ...BUT WORKER THREADS CAN (JUST REMEMBER - THEY'RE STILL AN EXPERIMENTAL FEATURE!) ▸ NODE.JS EXCELS IN I/O BOUND AND MIGHT NOT BE THE BEST CHOICE FOR CPU-HEAVY OPERATIONS SUMMARY NODE, CAN YOU EVEN IN CPU INTENSIVE OPERATIONS?
- 73. THANKS