Building Microservices with Node.js: Explore microservices applications and migrate from a monolith architecture to microservices

Building Microservices with Node.js

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Rohit Rajkumar

Publishing Product Manager: Vaideeshwari Muralikrishnan

Book Project Manager: Shagun Saini

Senior Editor: Anuradha Joglekar

Technical Editor: K Bimala Singha

Copy Editor: Safis Editing

Proofreader: Anuradha Joglekar

Indexer: Subalakshmi Govindhan

Production Designer: Alishon Mendonca

DevRel Marketing Coordinators: Anamika Singh and Nivedita Pandey

Publication date: May 2024

Production reference: 1030424

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83898-593-6

www.packtpub.com

Contributors

About the author

Daniel Kapexhiu is a senior software developer working with the latest technologies in web, desktop, artificial intelligence, cloud computing, and application programming interfaces. He is in a continuously studying process from top universities in Italy, France, the UK, and the USA.

During the process of authoring this book, I have learned so many things, including collaborating with the amazing team at Packt.

I want to thank the people who have been close to me and supported me, especially my family, my friends, and the amazing staff at Packt.

About the reviewer

Samita Swagatika is an engineer dedicated to crafting flawless user experiences. With an 8+ year track record in software development and a degree in computer science, she specializes in full stack development with robust backend expertise. Her contributions to various projects have streamlined processes and enhanced user engagement. Currently serving as Health Engineer II at Best Buy Health, she has transformed innovative ideas into scalable solutions. Over the past six years, her focus has been on revolutionizing the healthcare domain. Beyond coding, she channels her creativity as a professional crocheter.

Table of Contents

Preface

Part 1: Understanding Microservices and Node.js

1

Introducing Microservices

Introduction to the microservices and decentralized architecture concepts

Microservices architecture

Decentralized architecture

Service boundaries and loose coupling

Service boundaries

Loose coupling

Independent development and deployment and polyglot architecture

Independent development and deployment

Polyglot architecture

Scalability and resilience and independent data management

Scalability and resilience

Independent data management

APIs and communication and CI

APIs and communication

CI

Summary

Quiz time

2

Exploring the Core Principles of Microservices

Overview of the core principles of microservices

Understanding the fundamentals and identifying business capabilities

Defining service contracts and decentralized decision making

Service contracts in microservices

Decentralized decision making in microservices

Prioritizing autonomy and ownership and designing for resilience

Prioritizing autonomy and ownership

Designing for resilience

Implementing communication strategies and ensuring scalability

Implementing communication strategies

Ensuring scalability

Implementing observability and continuously learning and improving

Implementing observability

Continuous learning and improving

Summary

Quiz time

3

Understanding Node.js Fundamentals: Building Blocks and Key Concepts

Asynchronous and non-blocking communication and event-driven architecture

Asynchronous and non-blocking communication

Event-driven architecture

The JavaScript ecosystem and server-side development

The JavaScript ecosystem

Server-side development with Node.js

Command-line applications and scalability and performance

Command-line applications

Scalability and performance

Cross-platform compatibility and community and support

Cross-platform compatibility

Community and support

Microservices and serverless architectures and their integration through APIs

What is microservices architecture?

What is a serverless architecture?

Integration through APIs

Integration in microservices and serverless architectures

Summary

Quiz time

4

Leveraging the JavaScript and Node.js Ecosystem for Microservices Development

Vast package management and developer productivity

Vast package management

Developer productivity

Community support and collaboration, and rapid innovation and updates

Community support and collaboration

Rapid innovation and updates

Versatility and full stack development, and cross-platform compatibility

Versatility and full stack development

Cross-platform compatibility

Integration and interoperability and support for modern web standards

Integration and interoperability

Support for modern web standards

Enterprise adoption and industry maturity and ecosystem growth and innovation

Enterprise adoption and industry maturity

What is ecosystem growth and innovation?

Summary

Quiz time

Part 2: Building and Integrating Microservices with Node.js

5

Knowing the Infrastructure of Microservices in Node.js

Service discovery and API gateways

Service discovery

API gateways

Load balancing and service orchestration

Load balancing

Service orchestration

Containerization and orchestration and centralized logging and monitoring

Containerization and orchestration

Centralized logging and monitoring

Distributed tracing and event-driven communication

Distributed tracing

Event-driven communication

Database integration and continuous integration and deployment

Database integration

CI/CD

Summary

Quiz time

6

Designing Microservices Architecture in Node.js

Things to consider before creating your microservice

Communication protocol and design APIs

Communication protocol

API design

Decentralized data management and data consistency

Authentication and authorization and error handling and fault tolerance

Authentication and authorization

Error handling and fault tolerance

Monitoring and tracing requests and containerization technologies

Monitoring and tracing requests

Containerization technologies

Summary

Quiz time

7

Integrating Microservices in Node.js Applications

Synchronous HTTP/REST communication and asynchronous messaging

Synchronous HTTP/REST communication

Asynchronous messaging

EDA and API gateways

EDA

API gateways

Service mesh and caching

Service mesh

Caching

Distributed tracing and database integration

Distributed tracing

Database integration

Monitoring and observability and error handling and resilience

Monitoring and observability

Error handling and resilience

Summary

Quiz time

8

Debugging Microservices in Node.js

Logging and debugging tools

Logging in microservices

Debugging tools

Debugging in containers and error handling

Debugging in containers

Error handling

Unit testing and remote debugging

Unit testing

Remote debugging

Instrumentation and tracing and environment and configuration

Instrumentation and tracing

Environment and configuration

Reproducing and isolating issues and debugging tools and libraries

Reproduce and isolate issues

Debugging tools and libraries

Summary

Quiz time

Part 3: Data Management in Microservices Using Node.js

9

Database Manipulation in Microservices with Node.js

Choosing the right database and database connections

Choosing the right database

Database connections in microservices

Data models and schemas and CRUD operations

Data models and schemas

CRUD operations

Transactions and data validation

Transactions in microservices

Data validation and sanitization

Error handling and optimizations

Error handling in microservices

Optimizations in microservices

Testing

Summary

Quiz time

10

API Communication and Data Contracts in Microservices

Defining API contracts and RESTful API design

Defining API contracts

RESTful API design

REST API libraries and API versioning

REST API libraries

API versioning

Authentication and authorization and data validation

Authentication and authorization

Data validation

Error handling and API documentation

Error handling

API documentation

API testing and API gateway

API testing

API gateway

Summary

Quiz time

11

Caching and Asynchronous Messaging in Microservices

Client-side caching and edge caching

Client-side caching

Edge caching

Microservice-level caching and database query caching

Microservice-level caching

Database query caching

Message queues and publish-subscribe

Message queues

Publish-subscribe (Pub/Sub)

Event-driven architecture

Summary

Quiz time

12

Ensuring Data Security with the Saga Pattern, Encryption, and Security Measures

Compensating actions and Saga orchestration

Compensating actions

Saga orchestration

Event-driven communication and Sagas with state

Event-driven communication

Sagas with state

Transport layer security (TLS) and data encryption at rest

TLS

Data encryption at rest

Encryption algorithms and key management

Encryption algorithms

Key management

Authentication, authorization, input validation, secure coding practices, and API rate limiting

Authentication

Authorization

Input validation

Secure coding practices

API rate limiting

Summary

Quiz time

Part 4: Monitoring and Logging in Microservices with Node.js

13

Monitoring Microservices in Node.js

Structured logging and log levels

Contextual information and centralized log management

Contextual information in logs

Centralized log management

Application-level metrics, distributed tracing, and health checks

Application-level metrics

Distributed tracing

Health checks

Threshold-based alerts and anomaly detection

Threshold-based alerts

Anomaly detection

Request tracing, request context propagation, and logging frameworks

Request tracing

Request context propagation

Logging frameworks

Summary

Quiz time

14

Logging in Microservices with Node.js

Choosing a logging framework and defining log levels

Choosing a logging library

Log levels

Structured logging, log transport, and storage

Structured logging

Log transport and storage

Log filtering, sampling, error handling, and exception logging

Context propagation, monitoring, and analyzing logs

Context propagation

Monitoring

Log analysis

Summary

Quiz time

15

Interpreting Monitoring Data in Microservices

Metrics analysis

Log analysis

Alerting and thresholds

Visualization and dashboards

Correlation and context

Summary

Quiz time

16

Analyzing Log Data in Microservices with Node.js

Log levels and severities

Request tracing, contextual information, and event sequencing and order

Request tracing

Contextual information

Event sequencing and order

Advantages and considerations of request tracing, contextual information, and event sequencing and order

Log format, structured logging, and log filtering and search

Log format

Structured logging

Log filtering and search

Advantages and considerations of log format, structured logging, and log filtering and search

Log aggregation, centralized log management, visualization, and log analysis tools

Log aggregation

Centralized log management

Visualization

Log analysis tools

Advantages and considerations of log aggregation, centralized log management, visualization, and log analysis tools

Correlation of log data with metrics and monitoring data

Summary

Quiz time

Final words

Index

Other Books You May Enjoy

Preface

Microservices are a popular architectural style for building scalable, resilient, and adaptable applications. They allow developers to decompose a complex system into smaller, independent, and loosely coupled services that communicate through well-defined interfaces. Microservices enable faster delivery, easier testing, and greater flexibility in choosing the right technologies for each service.

Node.js is a powerful and versatile platform for building microservices. It offers a fast and lightweight runtime environment, a rich set of libraries and frameworks, and a vibrant and supportive community. Node.js enables developers to write microservices in JavaScript, a ubiquitous and expressive language that runs on any platform. Node.js also supports asynchronous and event-driven programming, which is essential for handling concurrent requests and building reactive systems.

This book is a comprehensive guide to building microservices with Node.js. It covers the concepts, principles, and best practices of microservice architecture, as well as the tools and techniques for designing, developing, testing, deploying, and monitoring Node.js microservices. It also provides practical examples and case studies of real-world microservice applications built with Node.js.

Who this book is for

This book is intended for developers who have some experience with Node.js and want to learn how to build microservice applications with it. It assumes that you are familiar with the basics of Node.js, such as modules, callbacks, promises, and events. It also assumes that you have some knowledge of web development, such as HTTP, REST, JSON, and HTML. However, you do not need to be an expert in any of these topics, as the book will explain them as needed.

What this book covers

Chapter 1

, Introducing Microservices, introduces you to the microservices architecture. You’ll explore the various benefits of microservices as well as some challenges. Finally, you’ll learn about what it takes for successful implementation of microservices.

Chapter 2

, Exploring the Core Principles of Microservices, will take you through the core principles and mindset behind this architectural style to be able to learn to think in microservices, shifting from traditional monolithic thinking.

Chapter 3

, Understanding Node.js Fundamentals: Building Blocks and Key Concepts, will take you through the fundamentals of Node.js, providing you with a solid foundation for developing server-side applications, command-line tools, and other JavaScript-based solutions using the Node.js runtime.

Chapter 4

, Leveraging the JavaScript and Node.js Ecosystem for Microservices Development, will teach you the importance of using the JavaScript and Node.js ecosystems in microservices development to empower developers, drive innovation, provide extensive resources, and foster collaboration.

Chapter 5

, Knowing the Infrastructure of Microservices in Node.js, will introduce you to the infrastructure of microservices in Node.js. You’ll learn about the foundational components and technologies required to build, deploy, and manage a microservices architecture using Node.js. You’ll also learn how to carefully select and integrate these components and tools based on your specific requirements. Finally, you’ll know about some important considerations when designing and implementing the infrastructure for your microservices architecture.

Chapter 6

, Designing Microservices Architecture in Node.js, will show you how to design microservices in Node.js by breaking down a monolithic application into smaller, independent services that can be developed, deployed, and scaled individually. Designing microservices involves finding the right balance between granularity and complexity. It’s essential to carefully plan and evaluate your requirements to ensure that microservices provide tangible benefits in terms of scalability, maintainability, and agility.

Chapter 7

, Integrating Microservices in Node.js Applications, will teach you how to integrate microservices in Node.js. You’ll learn how to establish communication and coordination between different services to create a cohesive and functioning system. Finally, you’ll explore the specific requirements of your system, the communication patterns that best suit your needs, and the tools and libraries available in the Node.js ecosystem.

Chapter 8

, Debugging Microservices in Node.js, will explore how to debug microservices in Node.js, which is considered challenging due to their distributed nature and interaction with other services. You’ll learn how to identify and resolve issues or errors that occur within the services using a systematic and methodical approach, combined with the appropriate tools and techniques.

Chapter 9

, Database Manipulation in Microservices with Node.js, will take you through the Create, Read, Update, Delete (CRUD) operations performed on databases or data storage systems to manipulate data in microservices. You’ll know about the best practices for data security based on the requirements of your microservices and compliance standards. By implementing these steps, you can effectively manipulate data within your Node.js microservices and ensure proper interaction with the underlying database or data storage system.

Chapter 10

, API Communication and Data Contracts in Microservices, will teach you how to establish a clear contract for data exchange and define the interface through which services interact with each other during communication in microservices through APIs. By following the practices explored, you can establish effective communication between your microservices using APIs to enable decoupling, scalability, and flexibility in your architecture, allowing individual services to evolve independently while maintaining seamless interactions.

Chapter 11

, Caching and Asynchronous Messaging in Microservices, will introduce you to two important techniques used in microservices architecture to improve performance, scalability, and decoupling: caching and asynchronous messaging. You’ll learn how to store frequently accessed data in a cache as well as enable loose coupling and scalability in microservices by decoupling services through message queues or publish-subscribe patterns via asynchronous messaging.

Chapter 12

, Ensuring Data Security with the Saga Pattern, Encryption, and Security Measures, will introduce you to some essential aspects to consider when designing and implementing microservices: the Saga pattern, data encryption, and security.

Chapter 13

, Monitoring Microservices in Node.js, will teach you the importance of monitoring microservices in Node.js for maintaining their health, identifying performance issues, detecting errors, and ensuring overall system reliability. You’ll learn about the metrics that provide quantitative data about the behavior and performance of your microservices. You’ll also learn about alerting, which ensures that you are promptly notified of critical issues or abnormal behavior in your microservices. Finally, you’ll explore some tracing and debugging tools that can help diagnose and troubleshoot issues in your microservices.

Chapter 14

, Logging in Microservices with Node.js, will introduce you to a crucial aspect of microservices architecture, logging, which helps capture important information and events within your Node.js services. By implementing effective logging practices in your Node.js microservices, you can improve debugging.

Chapter 15

, Interpreting Monitoring Data in Microservices, will show you how to interpret monitoring data in microservices by analyzing the metrics, logs, and other monitoring information to gain insights into the health, performance, and behavior of your Node.js microservices.

Chapter 16

, Analyzing Log Data in Microservices with Node.js, will explore interpreting logs in microservices architecture, which involves analyzing the log data generated by your Node.js microservices to gain insights into their behavior, troubleshoot issues, and monitor their health.

To get the most out of this book

You need to have knowledge of an operating system such as Windows, Linux, or macOS, as well as knowledge of Node.js.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Building-Microservices-with-Node.js

. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/

. Check them out!

Code in Action

Code in Action videos for this book can be viewed at (https://packt.link/oBs87

)

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: The /debug/health endpoint is responsible for providing the health status of the microservice.

A block of code is set as follows:

// Import required modules

const express = require('express');

const app = express();

// Debug endpoint to get the health status of the microservice

app.get('/debug/health', (req, res) => {

Any command-line input or output is written as follows:

npm install winston

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "You can view and manage your monitors from the Monitors page…

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata

and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com

.

Share Your Thoughts

Once you’ve read Building Microservices with Node.js, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page

for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781838985936

Submit your proof of purchase

That’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Understanding Microservices and Node.js

In this part, you will get an overview of microservices and Node.js. We will start by understanding the theory of microservices. We will continue by looking at the environment of Node.js and its core principles.

The part contains the following chapters:

Chapter 1

, Introducing Microservices

Chapter 2

, Exploring the Core Principles of Microservices

Chapter 3

, Understanding Node.js Fundamentals: Building Blocks and Key Concepts

Chapter 4

, Leveraging the JavaScript and Node.js Ecosystem for Microservices Development

1

Introducing Microservices

Microservices, or microservices architecture, is an architectural style for designing and building software applications as a collection of small, independent, and loosely coupled services. Microservices offer benefits such as scalability, agility, independent development, and improved fault tolerance. However, they also introduce challenges such as service orchestration, distributed data management, and increased complexity in system design and testing. The successful implementation of microservices requires careful consideration of the specific application requirements and a well-defined architectural strategy.

In this book, we are going to learn about microservices in general and how to architect and develop microservices in Node.js. The book is suitable for backend developers, full-stack developers, software architects, and frontend developers who want to get into the world of backend development and extend their capabilities. You will learn, in an in-depth manner, the major tips and tricks to learn how to build microservices architecture using Node.js. At the end of this book, you will be able to conceptualize, plan, and architect microservices using Node.js, as well as develop and debug them. These are the major skills that companies want their employees to have in order to design the perfect solution for every problem every time.

We’ll start this chapter with an introduction to microservices and decentralized architectures. We’ll also learn about some key concepts in microservices, such as service boundaries, loose coupling, scalability, resilience, and independent data management. Finally, we’ll overview some important abilities in microservices, including independent development and deployment, polyglot architecture, APIs, and continuous integration (CI).

By the end of this chapter, you’ll have learned the fundamentals of microservices and why they are so useful.

In this chapter, we’re going to cover the following main topics:

Introduction to the microservices and decentralized architecture concepts

Service boundaries and loose coupling

Independent development and deployment and polyglot architecture

Scalability and resilience and independent data management

APIs and communication and CI

Introduction to the microservices and decentralized architecture concepts

In this section, we’ll learn about two important concepts: microservices and decentralized architecture.

Microservices is an architectural style and approach to building software applications as a collection of small, loosely coupled, and independently deployable services. Meanwhile, in decentralized architecture, components or services are distributed across multiple nodes or entities.

Both microservices architecture and decentralized architecture promote modularity, scalability, fault tolerance, and autonomy. While microservices focus on building applications as a collection of small services, decentralized architecture focuses on distributing processing and decision making across multiple nodes. These architectural approaches can be combined to build highly scalable, resilient, and flexible systems that can adapt to changing requirements and handle complex workloads.

Let’s start with the microservices architecture.

Microservices architecture

In a microservices architecture, the application is broken down into multiple small services, each responsible for a specific business capability. These services are developed, deployed, and managed independently, communicating with one another through well-defined application programming interfaces (APIs) or message-based protocols.

Figure 1.1 shows a typical microservices architecture compared to a typical monolithic architecture.

Figure 1.1: A typical microservices architecture

In Node.js, microservices are typically developed using lightweight frameworks such as Express.js or Fastify. Each microservice is a separate application with its own code base and can be developed, deployed, and scaled independently. Microservices can be written in different programming languages such as Java and