Automating Networks by using API
0 likes190 views
The document discusses network automation and the use of APIs to programmatically configure network devices. It provides examples of using a REST API built with Sinatra and interacting with databases to automate routine network configuration tasks. Key points covered include an overview of network APIs, illustrations of how networking tasks can be automated through programming, and important considerations for popularizing network automation including standardization, library support, and growing an ecosystem of network engineers adopting these techniques.
![[Cisco Connect 2018 - Vietnam] Joseph yap journey to the multi cloud](https://cdn.slidesharecdn.com/ss_thumbnails/josephyapjourneytothemulticloud-180504042215-thumbnail.jpg?width=560&fit=bounds)
More Related Content
Similar to Automating Networks by using API (20)
Recently uploaded (13)
Automating Networks by using API
- 1. 0 Copyright IDC Frontier Inc. All rights reserved. Supporting the FutureYour Innovative Partner 0 Automating Networks by using API/Webs -story of a Network Engineer struggling with Programming- IDC Frontier Inc. UX Development Dept. Issei Inoue Feb 24, 2016
- 2. 11 (C) IDC Frontier Inc. All Rights Reserved. Agenda ・What Is Network Automation, Network API? ・Case Study (Entering Settings to a Router by Web/APIEntry) ・Software Design for Network API ・Tools Used (Sinatra, MySQL, ActiveRecord, NETCONF, JavaScript, jQuery, etc.) ・Required Skills and Issues
- 3. 22 (C) IDC Frontier Inc. All Rights Reserved. What is Automating Networks ?
- 4. 33 (C) IDC Frontier Inc. All Rights Reserved. client HTTP Rest API Sinatra DB Jobs Setting devices Workers Router Worker Netconf Worker Netconf Worker Netconf MySQL Ruby CRM Application form 【Conventional Network Operation】 【Automated Network Operation】 CLI Router client
- 5. 44 (C) IDC Frontier Inc. All Rights Reserved. What is Network API? • APIs for network devices They can be programatically configured by CLI/GUI, however, APIs are used to make configuration more easily programmable • APIs for network services run by operators, etc. End users use APIs to set cloud networks such as IaaS ex) AWS CreateVpc, AcceptVpcPeeringConnection , etc. In the first place, what are "network APIs"? The former is made by manufacturers, and the latter is made by operators.
- 6. 55 (C) IDC Frontier Inc. All Rights Reserved. What is Network Automation • SDN • Creation of virtual networks on physical networks • NFV • Virtualisation of network functions on x86 servers • White box • Switch that separates hardware from software • Configuration management tools such as Ansible/Chef are used for management • Automatic configuration by grouping together several hundred switches (Zero Touch Provisioning) • Automation of configuration tasks (AutoProvisioning) In the first place, what is "automation"? The use of software? This time, this is the main theme * Although SDNs or NFVs make AutoProvisioning easier they are defined as something else here
- 7. 66 (C) IDC Frontier Inc. All Rights Reserved. Programming of Tasks client HTTP Rest API Sinatra DB Jobs Setting devices Job registration Workers Router Worker Netconf Worker Netconf Worker Netconf Not executed Job confirmation Reflection of settings Tasks done by people are put into logical patterns and loaded into a program. MySQL Ruby Process ⇒⇒⇒⇒ API Information ⇒⇒⇒⇒ DB +----+-----------+-------------+------------+------+--------+--------+--------------------+----+ | id | vpn_num | domain |username | vlan | vrf | zone | segment |job | +----+-----------+-------------+------------+------+--------+--------+--------------------+----+ | 72 | NPVN00619 | 70000000619 |issei06192 | 20 | 619 | Henry | 192.168.20.254/24 | 1 | | 71 | NPVN00619 | 70000000619 |issei0619 | 19 | 619 | Pascal | 192.168.19.254/24 | 1 | | 67 | NPVN00618 | 70000000618 |issei0618 | 618 | 18 | Tesla | 192.168.18.254/24 | 1 | | 68 | NPVN00618 | 70000000618 |issei0618 | 617 | 18 | Tesla | 192.168.60.254/24 | 1 | | 65 | NPVN00111 | 70000000111 |issei01 | 111 | 110 | Tesla | 192.168.111.254/24 | 1 | | 58 | NPVN00615 | 70000000615 |issei0615 | 15 | 6 | Pascal | 192.168.15.254/24 | 1 | | 59 | NPVN00615 | 70000000615 |issei0615-2 | 152 | 6 | Tesla | 192.168.152.254/24 | 1 | | 64 | NPVN00615 | 70000000615 |issei0615-4 | 154 | 6 | Pascal | 192.168.154.254/24 | 1 | | 57 | NPVN00608 | 70000000608 |Issei | 50 | 60 | Pascal | 192.168.1.0/24 | 1 | | 63 | NPVN00615 | 70000000615 |issei0615-3 | 615 | 6 | Henry | 192.168.156.254/24 | 1 | | 62 | NPVN01615 | 70000001615 |issei1615 | 16 | 15 | Tesla | 192.168.16.254/24 | 1 | | 56 | NPVN00612 | 70000000612 |issei0612 | 612 | 612 | Tesla | 192.168.61.254/24 | 0 | +----+-----------+-------------+------------+------+--------+--------+--------------------+----+ there are quite a few provisioning tools like these
- 8. 77 (C) IDC Frontier Inc. All Rights Reserved. Illustration of Linking APIAPI PXC(Percona XtraDB Cluster) netconfnetconf (Standby) IP-VPN (Active) 3 replication east-regionwest-region
- 9. 88 (C) IDC Frontier Inc. All Rights Reserved.
- 10. 99 (C) IDC Frontier Inc. All Rights Reserved. Tools I used ・Sinatra ・Flask ・Django Language ・Ruby ・Python ・Go ・PHP ・Bash DB ・Percona ・ActiveRecord ・MongoDB ・Drizzle ・MariaDB ・Percona ・Redis Web framework Configuration management provisioning tools ・Ansible ・Puppet ・Chef ・Fabric Job management ・RabbitMQ ・Celery ・rundec ・cron ・(Bash) ・(MySQL) Web server ・Apache ・Nginx ・Webrick ・ ・ ・ ・ ・ ・ ・ ・ ・Web UI O/R mapping ・Rack ・WSGI ・PSGI ・Netconf ・Bash ・API ・JavaScript ・jQuery ・Ajax *This is not to say that I recommend these tools. I might even go so far to say that any tool would be good enough.
- 11. 1010 (C) IDC Frontier Inc. All Rights Reserved. API Design Method API Path Argument Return Value Addition of new records Post /privateconnect record Success/Fail VPN settings Post /privateconnect Job id Success/Fail Acquisition of information of VRFs to be set Get /routinginstance Job id xml-based VRF information Acquisition of information of interface to be deleted Get /interface_stats Job id xml-based interface information Deletion of records Delete /privateconnect_record Job id Success/Fail Deletion of VPN settings Delete /privateconnect_setting Job id Success/Fail Acquisition of 10 latest records in JSON format Get /privateconnect.json n/a VPN information in JSON format APIs and Classes/methods for each action
- 12. 1111 (C) IDC Frontier Inc. All Rights Reserved. Tips For Rest APIs API visibility is also important If possible. I'd like notation methods to be unified, too. It would be very difficult though. spinal-case snake-case camel-case dot How to join - (hyphen) _ (underscore) Uppercase characters .(dot) Examples /v1/idcf-inoue /v1/idcf_inoue /v1/idcfInoue /v1/idcf.inoue Main Corporation That Apply These Google LinkedIn Facebook Twitter Instagram slack YouTube CloudStack OpenStack Facebook Others Apparently good in terms of SEO What's the latest fashion? This is often the case with JavaScript, jQuery and Ruby. Not seen a lot There are several notation methods to connect two or more words.
- 13. 1212 (C) IDC Frontier Inc. All Rights Reserved. UI Illustration 1. Extraction of information 2. Addition of settings 3. Deletion of settings Information targeted in full text searches can be easily extracted mysql> select * from private_connects; +----+-----------+-------------+----------------+-------------+------+--------+-----------+--------+--------------------+---------------------+---------------------+----------+ | id | vpn_num | domain | account | username | vlan | vrf | bandwidth | zone | segment | created_at | updated_at | job_done | +----+-----------+-------------+----------------+-------------+------+--------+-----------+--------+--------------------+---------------------+---------------------+----------+ | 72 | NPVN00619 | 70000000619 | inoue06192 | issei06192 | 20 | 619 | 1000 | Henry | 192.168.20.254/24 | 2015-06-19 10:38:01 | 2015-06-19 10:39:15 | 1 | | 71 | NPVN00619 | 70000000619 | inoue0619 | issei0619 | 19 | 619 | 1000 | Pascal | 192.168.19.254/24 | 2015-06-19 10:34:24 | 2015-06-19 10:36:51 | 1 | | 67 | NPVN00618 | 70000000618 | inoue0618 | issei0618 | 618 | 18 | 1000 | Tesla | 192.168.18.254/24 | 2015-06-18 04:23:04 | 2015-06-18 06:10:22 | 1 | | 68 | NPVN00618 | 70000000618 | inoue0618 | issei0618 | 617 | 18 | 1000 | Tesla | 192.168.60.254/24 | 2015-06-18 06:13:04 | 2015-06-18 06:15:08 | 1 | | 65 | NPVN00111 | 70000000111 | inoue01 | issei01 | 111 | 110 | 1000 | Tesla | 192.168.111.254/24 | 2015-06-15 10:42:35 | 2015-06-17 01:58:20 | 1 | | 58 | NPVN00615 | 70000000615 | inoue0615 | issei0615 | 15 | 6 | 1000 | Pascal | 192.168.15.254/24 | 2015-06-15 06:51:00 | 2015-06-15 06:51:18 | 1 | | 59 | NPVN00615 | 70000000615 | inoue0615-2 | issei0615-2 | 152 | 6 | 1000 | Tesla | 192.168.152.254/24 | 2015-06-15 07:23:34 | 2015-06-15 07:29:34 | 1 | | 64 | NPVN00615 | 70000000615 | inoue0615-4 | issei0615-4 | 154 | 6 | 1000 | Pascal | 192.168.154.254/24 | 2015-06-15 10:32:31 | 2015-06-15 10:33:01 | 1 | | 57 | NPVN00608 | 70000000608 | Inoue | Issei | 50 | 60 | 1000 | Pascal | 192.168.1.0/24 | 2015-06-12 05:20:00 | 2015-06-15 06:53:06 | 1 | | 63 | NPVN00615 | 70000000615 | inoue0615-3 | issei0615-3 | 615 | 6 | 1000 | Henry | 192.168.156.254/24 | 2015-06-15 10:01:53 | 2015-06-15 10:02:54 | 1 | | 62 | NPVN01615 | 70000001615 | iinoue1615 | issei1615 | 16 | 15 | 1000 | Tesla | 192.168.16.254/24 | 2015-06-15 09:50:14 | 2015-06-15 09:50:49 | 1 | | 56 | NPVN00612 | 70000000612 | inoue0612 | issei0612 | 612 | 612 | 1000 | Tesla | 192.168.61.254/24 | 2015-06-12 05:15:41 | 2015-06-12 05:16:47 | 1 | | 73 | NPVN00619 | 70000000619 | %,$,#,&,?,!etc.| issei06193 | 193 | 619 | 1000 | Henry | 192.168.193.254/24 | 2015-06-19 12:32:39 | 2015-06-19 12:36:37 | 1 | | 74 | NPVN00619 | 70000000619 | inoue06194 | issei06194 | 0 | 619 | 1000 | Tesla | 192.168.0.254/24 | 2015-06-19 12:39:18 | 2015-06-19 12:39:39 | 1 | | 76 | NPVN01619 | 70000000619 | inoue06195 | issei06195 | 195 | 650000 | 1000 | Henry | 172.16.0.254/24 | 2015-06-19 12:45:35 | 2015-06-19 12:45:52 | 1 | | 83 | NPVN00624 | 70000000624 | inoue0624 | issei0624 | 625 | 60 | 1000 | Pascal | 192.168.25.254/24 | 2015-06-24 12:23:50 | 2015-06-24 12:24:00 | 1 | | 82 | NPVN00624 | 70000000624 | inoue0624 | issei0624 | 624 | 60 | 1000 | Tesla | 192.168.60.254/24 | 2015-06-24 11:09:01 | 2015-06-24 11:12:49 | 1 | | 85 | NPVN00626 | 70000000626 | inoue0626 | issei0626 | 626 | 26 | 1000 | Pascal | 192.168.26.254/24 | 2015-06-25 23:55:10 | 2015-06-25 23:55:10 | 0 | +----+-----------+-------------+----------------+-------------+------+--------+-----------+--------+--------------------+---------------------+---------------------+----------+ 18 rows in set (0.00 sec)
- 14. 1313 (C) IDC Frontier Inc. All Rights Reserved. 1. Extraction of information 2. Addition of settings 3. Deletion of settings Existing line information is shown to keep erroneous entry to a minimum UI Illustration Static route Next Hop 192.168.10.0/24 192.168.1.100
- 15. 1414 (C) IDC Frontier Inc. All Rights Reserved. 1. Extraction of information 2. Addition of settings 3. Deletion of settings Erroneous entry is kept to a minimum by displayed the usage status of the interface targeted for deletion UI Illustration
- 16. 1515 (C) IDC Frontier Inc. All Rights Reserved. Difficult and Important Points to Popularize Automation • Integration and standardization of input interface Bash Use of expect, general-purpose in a sense?? Netconf RPC-based, annotated in XML SOAP API Outdated.. Suited to complex input/output? Rest API The latest de facto standard URL resource support • Substantiation of libraries, substantiation of sample code Netconf is supported on the product side, which is meaningless even if supported Having a mechanism so that Netconf is used is important. • Software ecosystem comprising network engineers Even if manufacturers create plugins and libraries, they will not be popularized if they are not passed onto users as well
- 17. 1616 (C) IDC Frontier Inc. All Rights Reserved. Main Libraries Located in Github, Etc. Juniper https://github.com/Juniper/net-netconf https://github.com/Juniper/netconf-perl https://github.com/Juniper/netconf-java https://github.com/Juniper/netconf-php https://github.com/leopoul/ncclient Cisco https://github.com/jtimberman/ruby-cisco https://github.com/nickpegg/ciscolib Brocade https://github.com/brocade/ncclient https://github.com/brocade/brocade （OpenStack Plugin） https://github.com/BRCDcomm/BVC （VyattaController ） https://github.com/zapman449/brocade_switchshow_aliases （Fiber switches） Alaxala https://github.com/sumikawa/netconf Ideally, there should be standard APIs (APIs that allow all switches share a common configuration.), however, this would be difficult in a number of ways. I would at least like official libraries from manufacturers Cumulus https://github.com/CumulusNetworks/cumulus-linux-ansible-modules https://github.com/CumulusNetworks/cumulus-linux-chef-modules https://github.com/CumulusNetworks/net-next https://github.com/CumulusNetworks/quagga https://github.com/CumulusNetworks/cumulus-cl-interfaces-puppet https://github.com/OpenRTMFP/Cumulus （MonaServerusedSW） https://github.com/cotdsa/cumulus http://cumulusnetworks.com/blog/cumulus-linux-2/ It would be better to register project names (account names) early....φ(・ω・｀ )
- 18. 1717 (C) IDC Frontier Inc. All Rights Reserved. Topics and Talk About Hardships
- 19. 1818 (C) IDC Frontier Inc. All Rights Reserved. Setting Netconf on Rubyfor Junos Netconf was difficult. XML format as it is a bit troublesome for human eyes. However, In Junos, ”show configuration | display xml” is really handy! When using with Ruby, hyphens are taken as a regular expression, so it is recommended to use the send method. RPC is difficult. Sometimes the program cannot get unlocked... RestAPI is the best . It is easy for beginners. interfaces { ”interface” { unit ”vlan” { description NPVN00615:70000000615:inoue0615; vlan-id 15; family inet { filter { input 1G; } address 192.168.15.253/24 { vrrp-group 163 { virtual-address 192.168.15.254; priority 150; } } } } } } Junos ConfigXML format
- 20. 1919 (C) IDC Frontier Inc. All Rights Reserved. Checking Netconf information on Rubyfor Junos Show related information is in XML so the output cannot fit on single screen. show interfaces xe-2/0/0.111 detail results 3 scrolls
- 21. 2020 (C) IDC Frontier Inc. All Rights Reserved. BIG-IP Rest API (Ver11.5 or later) $ curl -sk -H "Authorization: Basic xxxxxxxxxxxxxx" https://x.x.x.x/mgmt/tm/ltm/node -H 'Content-Type: application/json' -X POST -d '{"address": "192.168.0.1","description": "testdescription","name": "testname"}' {"kind":"tm:ltm:node:nodestate","name":"testname","fullPath":"testname","generation":36,"selfLink":"https://localhost/mgmt/t m/ltm/node/testname?ver=11.5.1","address":"192.168.0.1","connectionLimit":0,"description":"testdescription","dynamicRatio": 1,"logging":"disabled","monitor":"default","rateLimit":"disabled","ratio":1,"session":"monitor-enabled","state":"checking"}[ Node addition Addition of member to pool $ curl -sk -H "Authorization: Basic xxxxxxxxxxxxxx" https://x.x.x.x/mgmt/tm/ltm/pool/~Common~pool_test/members -H 'Content-Type: application/json' -X POST -d '{"name": "testname:80"}' {"kind":"tm:ltm:pool:members:membersstate","name":"testname:80","fullPath":"testname:80","generation":38,"selfLink":"https ://localhost/mgmt/tm/ltm/pool/~Common~pool_test/members/testname:80?ver=11.5.1"} Setting Sync $ curl -sk -H "Authorization: Basic xxxxxxxxxxxxxx" https://x.x.x.x/mgmt/tm/ltm/pool/~Common~pool_test/members -H 'Content-Type: application/json' -X POST -d '{"name": "testname:80"}' {"kind":"tm:ltm:pool:members:membersstate","name":"testname:80","fullPath":"testname:80","generation":38,"selfLink":"https ://localhost/mgmt/tm/ltm/pool/~Common~pool_test/members/testname:80?ver=11.5.1"}
- 22. 2121 (C) IDC Frontier Inc. All Rights Reserved. libraries that were useful Ruby NetAddr Package Programming broadcast addresses or gateway addresses (end addresses) from scratch was more difficult than I expected. For example, it was surprisingly difficult to programmatically generate the gateway address 192.168.1.254 from the specified value from 192.168.1.0/24 using just the IPAddr Class. I have solved the problem by using the NetAddr library. https://rubygems.org/gems/netaddr/versions/1.5.0 http://www.rubydoc.info/gems/netaddr/1.5.0/NetAddr netaddr1 = NetAddr::CIDR.create('192.168.1.0/24') # broadcast address calculation bc_address = netaddr1.last # Convert to numerical form and subtract 1 gateway_int = NetAddr::CIDR.create(bc_address).to_i - 1 # That numerical value is returned to address notation gateway = NetAddr.i_to_ip(gateway_int) # /24 section only is extracted netmask = netaddr1.netmask p bc_address # => "192.168.1.255" p gateway # => "192.168.1.254" p gateway + netmask # => 192.168.1.254/24 various handy methods are available, such as output in ARPA format or range designation and IPv6 compatibility
- 23. 2222 (C) IDC Frontier Inc. All Rights Reserved. Many other issues… • What happens when a user enters in non-IP address format? • What happens when logically invalid items are entered? • In what format are errors returned to users? • How is rollback performed when processing ends midway? • Where and how are operation logs saved? • What happens when a target device is in the middle of maintenance or malfunctions? • How do you go about making a redundant system or DR? What about data maintenance? • How do you perform maintenance on the code when devices are replaced? • Generation management, testing, CI (Continuous Integration) • How do you transfer knowledge? There is no end to software development... How far can we anticipate errors?
- 24. 2323 (C) IDC Frontier Inc. All Rights Reserved. Summary Well, it is not easy… But How would you like jobs been automated? Would you like networks to be manageable like applications? We can explore new services. We can control network to make it more stable and secure With Network Automation,
- 25. 2424 (C) IDC Frontier Inc. All Rights Reserved. Summary New things come into existence by sharing various people's intelligence together. That is the Internet. Networks cannot be created by one person, nor one company. What is important is connecting people and things. A next generation Internet may be born by the combination of networks and software!
- 26. 2525 (C) IDC Frontier Inc. All Rights Reserved. Finally, a little about me... http://qiita.com/inoueissei https://github.com/inoueissei https://github.com/netops-coding https://www.facebook.com/inoue.issei https://twitter.com/inoueissei IDC Frontier Inc. Technology Development Division UX Development Dept. Issei Inoue I'm not very good at disseminating information. However, I'm going to do my best from here on.
- 27. 2626 (C) IDC Frontier Inc. All Rights Reserved. Thank you for your kind attention.
- 28. 2727 Supporting the FutureYour Innovative Partner (C) IDC Frontier Inc. All Rights Reserved.