Soa with consul
2 likes421 views
While we move towards microservices architecture, and are managing too many services, the issue of service discovery arises. This becomes a bigger issue when we are autoscaling. So we need a system to be able to resolve a service to a specific machine in our deployment.
![Server nodes
con g.json
{
"server": true,
"data_dir": "/var/consul",
"encrypt": "2346rxJm2aFuMNxWCKWnvPBA==",
"log_level": "INFO",
"enable_syslog": true,
"bind_addr": "172.31.0.195",
"start_join": ["172.31.12.223", "172.31.28.140"],
"ui": true
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-8-320.jpg)
![services.json
{
"services": [
{
"id": "application@ip-172-31-21-186",
"name": "application",
"address": "172.31.21.186",
"port": 8080,
"checks": [
{
"name": "Application/Apache on port 8080",
"tcp": "localhost:8080",
"interval": "10s",
"timeout": "1s"
}
]
}
]
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-11-320.jpg)
![/v1/catalog/nodes
[
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "baz",
"Address": "10.1.10.11",
"TaggedAddresses": {
"lan": "10.1.10.11",
"wan": "10.1.10.11"
},
"Meta": {
"instance_type": "t2.medium"
}
} ...
]](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-14-320.jpg)
![/v1/catalog/services
{
"consul": [],
"redis": [],
"postgresql": [
"primary",
"secondary"
]
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-15-320.jpg)
![/v1/catalog/service/<service>
[
{
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "192.168.10.10",
"Meta": {
"instance_type": "t2.medium"
},
"CreateIndex": 51,
"ModifyIndex": 51,
"ServiceAddress": "172.17.0.3",
"ServiceEnableTagOverride": false,
"ServiceID": "32a2a47f7992:nodea:5000",
"ServiceName": "foobar",
"ServicePort": 5000,
"ServiceTags": [
"tacos"
]
}](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-16-320.jpg)
![/v1/catalog/node/<node>
{
"Node": {
"ID": "40e4a748-2192-161a-0510-9bf59fe950b5",
"Node": "foobar",
"Address": "10.1.10.12"
},
"Services": {
"consul": {
"ID": "consul",
"Service": "consul",
"Port": 8300
},
"redis": {
"ID": "redis",
"Service": "redis",
"Tags": [
"v1"
],
"Port": 6379](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-17-320.jpg)
![/v1/health/checks/<service>
[
{
"Node": "foobar",
"CheckID": "service:redis",
"Name": "Service 'redis' check",
"Status": "passing",
"Notes": "",
"Output": "",
"ServiceID": "redis",
"ServiceName": "redis"
}
]](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-20-320.jpg)
![GET /v1/kv/<key>
[
{
"CreateIndex": 100,
"ModifyIndex": 200,
"LockIndex": 200,
"Key": "zip",
"Flags": 0,
"Value": "dGVzdA==",
"Session": "adf4238a-882b-9ddc-4a9d-5b6758e4159e"
}
]
PUT /v1/kv/<key>
DELETE /v1/kv/<key>](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-24-320.jpg)
![Node Lookups
<node>.node[.datacenter].<domain>
$ dig @127.0.0.1 -p 8600 foo.node.consul ANY
; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 foo.node.consul ANY
; (1 server found)
;; truncating comment sections for brevity
;; QUESTION SECTION:
;foo.node.consul. IN ANY
;; ANSWER SECTION:
foo.node.consul. 0 IN A 10.1.10.12
;; AUTHORITY SECTION:
consul. 0 IN SOA ns.consul. postmaster.consul. 139283](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-26-320.jpg)
![Service Lookups
[tag.]<service>.service[.datacenter].<domain>
$ dig master.redis.service.consul
; <<>> DiG 9.10.3-P4-Ubuntu <<>> master.redis.service.consul
;; truncating comment sections for brevity
;; QUESTION SECTION:
;master.redis.service.consul. IN A
;; ANSWER SECTION:
master.redis.service.consul. 0 IN A 172.31.28.140
;; Query time: 2 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Feb 01 10:36:28 UTC 2017
;; MSG SIZE rcvd: 88](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-27-320.jpg)
![Other nodes can discover the leader node using
the session info API
$ curl http://localhost:8500/v1/kv/<key>
[
{
"Session": "4ca8e74b-6350-7587-addf-a18084928f3c",
"Value": "Ym9keQ==",
"Flags": 0,
"Key": "<key>",
"LockIndex": 1,
"ModifyIndex": 29,
"CreateIndex": 29
}
]
If the key has no associated Session, then there is
no leader](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-31-320.jpg)
![The nodes would be watching the key to check if it
was released and try to become the leader
$ curl
http://localhost:8500/v1/session/info/4ca8e74b-6350-7587-addf-a1
[
{
"LockDelay": 1.5e+10,
"Checks": [
"serfHealth"
],
"Node": "consul-primary-bjsiobmvdij6-node-lhe5ihreel7y"
"Name": "dbservice",
"ID": "4ca8e74b-6350-7587-addf-a18084928f3c",
"CreateIndex": 28
}
]](https://image.slidesharecdn.com/soa-170209160155/85/Soa-with-consul-32-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to Soa with consul (20)
Ad
Recently uploaded (20)
Soa with consul
- 1. Service Oriented Architecture with Consul Rajesh Sharma @broncha VantageBit
- 3. Problems when you scale a SOA/microservices Host resolution (service discovery) Con guration updates
- 5. Consul It is a tool for discovering and con guring services in your infrastructure, with features like Service Discovery Health Checking Key/Value Store
- 6. Installation Just download a binary le and you are good to go.
- 7. Con guration
- 8. Server nodes con g.json { "server": true, "data_dir": "/var/consul", "encrypt": "2346rxJm2aFuMNxWCKWnvPBA==", "log_level": "INFO", "enable_syslog": true, "bind_addr": "172.31.0.195", "start_join": ["172.31.12.223", "172.31.28.140"], "ui": true }
- 9. Client nodes
- 10. con g.json { "server": false, "data_dir": "/var/consul", "encrypt": "2346rxJm2aFuMNxWCKWnvPBA==", "log_level": "INFO", "enable_syslog": true, "bind_addr": "172.31.21.186", "start_join": ["172.31.28.140","172.31.12.223","172.31.0.195 }
- 11. services.json { "services": [ { "id": "application@ip-172-31-21-186", "name": "application", "address": "172.31.21.186", "port": 8080, "checks": [ { "name": "Application/Apache on port 8080", "tcp": "localhost:8080", "interval": "10s", "timeout": "1s" } ] } ] }
- 13. Catalog API
- 14. /v1/catalog/nodes [ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "baz", "Address": "10.1.10.11", "TaggedAddresses": { "lan": "10.1.10.11", "wan": "10.1.10.11" }, "Meta": { "instance_type": "t2.medium" } } ... ]
- 15. /v1/catalog/services { "consul": [], "redis": [], "postgresql": [ "primary", "secondary" ] }
- 16. /v1/catalog/service/<service> [ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "192.168.10.10", "Meta": { "instance_type": "t2.medium" }, "CreateIndex": 51, "ModifyIndex": 51, "ServiceAddress": "172.17.0.3", "ServiceEnableTagOverride": false, "ServiceID": "32a2a47f7992:nodea:5000", "ServiceName": "foobar", "ServicePort": 5000, "ServiceTags": [ "tacos" ] }
- 17. /v1/catalog/node/<node> { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Services": { "consul": { "ID": "consul", "Service": "consul", "Port": 8300 }, "redis": { "ID": "redis", "Service": "redis", "Tags": [ "v1" ], "Port": 6379
- 18. Health Check API
- 19. /v1/health/node/<node> [ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "CheckID": "serfHealth", "Name": "Serf Health Status", "Status": "passing", "Notes": "", "Output": "", "ServiceID": "", "ServiceName": "" }, { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check", "Status": "passing", "Notes": "",
- 20. /v1/health/checks/<service> [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check", "Status": "passing", "Notes": "", "Output": "", "ServiceID": "redis", "ServiceName": "redis" } ]
- 21. /v1/health/service/<service> [ { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Service": { "ID": "redis", "Service": "redis", "Tags": null, "Address": "10.1.10.12", "Port": 6379 }, "Checks": [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check",
- 22. /v1/health/service/<service>?passing [ { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Service": { "ID": "redis", "Service": "redis", "Tags": null, "Address": "10.1.10.12", "Port": 6379 }, "Checks": [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check",
- 23. The Key-Value Store API
- 24. GET /v1/kv/<key> [ { "CreateIndex": 100, "ModifyIndex": 200, "LockIndex": 200, "Key": "zip", "Flags": 0, "Value": "dGVzdA==", "Session": "adf4238a-882b-9ddc-4a9d-5b6758e4159e" } ] PUT /v1/kv/<key> DELETE /v1/kv/<key>
- 25. The DNS Interface Instead of making HTTP API requests to Consul, a host can use the DNS server directly via name lookups redis.service.us-east-1.consul master.rabbitmq.service.us-east-1.consul slave.redis.service.consul Consul will listen on 127.0.0.1:8600 for DNS queries in the consul. domain We can use dnsmasq to route DNS queries for consul. domains to 127.0.0.1:8600 Unhealthy nodes are automatically ltered
- 26. Node Lookups <node>.node[.datacenter].<domain> $ dig @127.0.0.1 -p 8600 foo.node.consul ANY ; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 foo.node.consul ANY ; (1 server found) ;; truncating comment sections for brevity ;; QUESTION SECTION: ;foo.node.consul. IN ANY ;; ANSWER SECTION: foo.node.consul. 0 IN A 10.1.10.12 ;; AUTHORITY SECTION: consul. 0 IN SOA ns.consul. postmaster.consul. 139283
- 27. Service Lookups [tag.]<service>.service[.datacenter].<domain> $ dig master.redis.service.consul ; <<>> DiG 9.10.3-P4-Ubuntu <<>> master.redis.service.consul ;; truncating comment sections for brevity ;; QUESTION SECTION: ;master.redis.service.consul. IN A ;; ANSWER SECTION: master.redis.service.consul. 0 IN A 172.31.28.140 ;; Query time: 2 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Feb 01 10:36:28 UTC 2017 ;; MSG SIZE rcvd: 88
- 28. Distributed lock and Leader election
- 29. Pick a key service/<servicename>/leader` Create a session $ curl -X PUT -d '{"Name": "rabbitmq-service"}' http://localhost:8500/v1/session/create # response { "ID": "4ca8e74b-6350-7587-addf-a18084928f3c" }
- 30. Acquire a session for given key $ curl -X PUT -d <body> http://localhost:8500/v1/kv/<key>?acquire=<session> If return value is true key was acquired and this node is the leader Else some other node already acquired that key and is the leader
- 31. Other nodes can discover the leader node using the session info API $ curl http://localhost:8500/v1/kv/<key> [ { "Session": "4ca8e74b-6350-7587-addf-a18084928f3c", "Value": "Ym9keQ==", "Flags": 0, "Key": "<key>", "LockIndex": 1, "ModifyIndex": 29, "CreateIndex": 29 } ] If the key has no associated Session, then there is no leader
- 32. The nodes would be watching the key to check if it was released and try to become the leader $ curl http://localhost:8500/v1/session/info/4ca8e74b-6350-7587-addf-a1 [ { "LockDelay": 1.5e+10, "Checks": [ "serfHealth" ], "Node": "consul-primary-bjsiobmvdij6-node-lhe5ihreel7y" "Name": "dbservice", "ID": "4ca8e74b-6350-7587-addf-a18084928f3c", "CreateIndex": 28 } ]
- 33. Questions?