ElasticSearch Cluster

What is ElasticSearch Cluster?

Elasticsearch cluster is a group of elastic search nodes that are connected to each other. Standalone instances are good with development and testing, but in a production environment it is recommended to use elasticsearch clusters because there are more advantages than a standalone.

Advantages of Elasticsearch Cluster

• Speed of Elasticsearch is able to execute complex queries fast.
• It stores real world complex entities to the structured JSON files.
• It's a Distributed system. Because of that it allows to scale horizontally. And it also provides the ability to extend resources and balance the loading between the nodes in a cluster.
• It records all the changes in data transactions on multiple nodes.
• Elasticsearch is API-driven, so actions can be performed using a simple RESTful API.

What is Node and What is Cluster?

Any time that you start an instance of Elasticsearch, you are starting a node. A collection of connected nodes is called a cluster. If you are running a single node of Elasticsearch, then you have a cluster of one node. Every node in the cluster can handle HTTP and Transport traffic by their default. When it's using the transport layer is used specifically for communication between nodes.

Node types and their roles

Master-eligible node

• This node has the master role, which makes it eligible to be elected as the master node, It controls the cluster.

Data node

• This node has the data role. It Stores data and performs data related operations such as CRUD, search, and aggregations.
• A data role can fill any of the Specific data node roles.

Ingest node

• This node has the ingest role. Ingest nodes are able to apply an ingest pipeline to a document in order to transform and enrich the document before indexing. That means it does the processing part before indexing happened.
• Even with a heavy ingest load, it makes sense to use dedicated ingest nodes and to not include the ingest role from nodes that have the master or data roles.

Remote-eligible node

• This node has the remote Cluster and Client roles, it makes it eligible to act as a remote client.

Machine learning node

• This node has the machine learning role.
• It provides machine learning features, For that there must be at least one machine learning node in your cluster.

Transform node

• This node has the transform role.
• There must be at least one transform node in your cluster to use it

ES Cluster Installation

1. Java needs to be installed in pc! If not Download and install OpenJDK
2. If you are using oracle jdk check the version is supported one or not (check elastic site)
3. Download Elasticsearch zip and unzip it anywhere(using same location for all clusters will help to config)
4. Rename and Update the Directory as Node-1
5. Inside the config folder, locate and open elasticsearch.yml file using vs code or any other text editor(i used vs code)
6. Uncomment and Update following commands

• Cluster.name:
• Node.name:node-1
• Http.port:9201
• Discovery.seed_hosts[‘127.0.0.1’]
• cluster.initial_master_nodes:[“node-1”,“node-2”,“node-3”]

7. Duplicate the directory 2 times and rename as Node-2 and Note-3

8. Edit Duplicated Node-2 and Node-3 files again by updating .yml file

• node.name: node-2 | node.name: node-3
• http.port: 9202 | http.port: 9203

9. Run Elasticsearch for each 3 nodes

10. Run Browser and reach each node using localhost:9201, localhost:9202 and localhost:9203.

Node-1

• Locate the bin folder of Node-1 and open the PowerShell with administrator privileges
• Execute .\elasticsearch.bat

Node-2

• Locate the bin folder of Node-2 and open the PowerShell with administrator privileges
• Execute .\elasticsearch.bat

Node-3

• Locate the bin folder of Node-3 and open the powershell with administrator privileges
• Execute .\elasticsearch.bat

Node-1 Elasticsearch view ()

localhost:9201        

Node-1 Elasticsearch view

http://localhost:9201/_cat/nodes?h=ip,port,heapPercent,name        

Node-2 Elasticsearch view

localhost:9202        

Node-2 Elasticsearch view

http://localhost:9202/_cat/nodes?h=ip,port,heapPercent,name        

Node-3 Elasticsearch view

localhost:9203        

Node-3 Elasticsearch view

http://localhost:9203/_cat/nodes?h=ip,port,heapPercent,name        

11. Download Postman and install it!

• Select free version
• Run postman
• Get the data using all nodes localhost urls.

Postman Testing

Node-1

Node-2

Node-3

Cluster node return

Check Cluster Stats

Check Cluster Health

Reference