Module 9: CDB Technical Intro
Download as PPTX, PDF2 likes4,260 views
The document provides an introduction to the ConfD Database (CDB), which is a hierarchical, ACID-compliant database used by ConfD to store configuration and operational data modeled with YANG. CDB stores data both in-memory and persistently on disk, supports transactions, replication, and automatic schema upgrades. Managed objects use CDB APIs to read configuration data at startup and subscribe to relevant configuration changes via a subscription socket.
![20©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Read out Configuration at Startup
Keypaths
• Each element in CDB is addressed by its path
cdb_get_duration(sock, &dur, “/dhcp/defaultLeaseTime”)
• Paths may include keys
cdb_get_ipv4(sock,&ip,”/dhcp/SubNets/subNet{10.0.0.0 255.0.0.0}/lowAddr”)
cdb_get_ipv4(sock, &ip, ”/dhcp/SubNets/subNet{%s %s}/lowAddr”, net, mask)
cdb_get_ipv4(sock, &ip, ”/dhcp/SubNets/subNet{%x %x}/lowAddr”, net, mask)
• Index may be used in place of keys
n = cdb_num_instances(sock, "/dhcp/SubNets/subNet");
for (i=0; i<n; i++) {
cdb_cd(sock, "/dhcp/SubNets/subNet[%d]", i);
do_subnet(…);
}](https://image.slidesharecdn.com/09cdbtechnicalintro-150213141226-conversion-gate02/85/Module-9-CDB-Technical-Intro-20-320.jpg)
![22©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Enter Event Loop
• Event loop needs to monitor subscription socket
• Typically using poll()/select()
while(1) {
set[0].fd = subsock;
set[0].events = POLLIN;
set[0].revents = 0;
poll(set, 1, -1); // Monitor one socket, wait forever
if (set[0].revents & POLLIN) {
cdb_read_subscription_socket(subsock, sub_points, &reslen);
// React on changes here…
// sub_points[] contains triggering subscription points,
// reslen how many
}
cdb_sync_subscription_socket(subsock, CDB_DONE_PRIORITY);
}
• How to handle CDB subscription notifications will be covered in the training module
“CDB Subscribers”](https://image.slidesharecdn.com/09cdbtechnicalintro-150213141226-conversion-gate02/85/Module-9-CDB-Technical-Intro-22-320.jpg)
Ad
More Related Content
What's hot (20)
Viewers also liked (14)
Ad
Similar to Module 9: CDB Technical Intro (20)
![[db tech showcase Tokyo 2016] E34: Oracle SE - RAC, HA and Standby are Still ...](https://cdn.slidesharecdn.com/ss_thumbnails/lqy2zqfxqmif8xeztxtr-signature-44327555e43d6a00760901a1f10c738a3c30da5dd494b81e3f7ae75f088cd2d2-poli-160815043717-thumbnail.jpg?width=560&fit=bounds)
Ad
Recently uploaded (20)
Module 9: CDB Technical Intro
- 1. 1©2015 Tail-f Systems all rights reservedFebruary 13, 2015 CDB Technical Introduction
- 2. 2©2015 Tail-f Systems all rights reservedFebruary 13, 2015 CDB – ConfD Database NETCONF SNMP REST CLI Web CDB Database Data Provider API CDB API Management Agent API App AppApp ConfD Core Engine App External DatabaseOperational Data App • Hierarchical database • ACID test compliant • Atomicity, consistency, isolation, durability • Fast, lightweight, fault-tolerant • Compact binary XML format • Memory resident with journal in persistent storage • Schema automatically derived from YANG • Supports 1:N data replication • Supports automatic schema version up/downgrades • CDB can hold some or all data • Configuration data and/or Operational data • External database can be used in addition to or instead of • Automatic loading of initial data • Applications read data, then subscribe to relevant configuration changes • Subscription notifications with priority level ordering • Multiple datastores per NETCONF standards • Startup, running, candidate, operational
- 3. 3©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Hierarchical • CDB uses an Hierarchical (XML) data model • Ideal for configuration data, which is tree structured • No Object-Relational-Mapping (ORM) required / dhcp/ aaa/ voip/ defaultLe… maxLeas… SubNets/ subNet/ mask range lowAddr highAddr logFacility net
- 4. 4©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Object Relational Mapping (ORM) Hierarchical Relational / dhcp/ aaa/ voip/ defaultLe… maxLeas… SubNets/ subNet/ mask range lowAddr highAddr logFacility net
- 5. 5©2015 Tail-f Systems all rights reservedFebruary 13, 2015 ACID The four properties that define a transaction: • Atomicity • Transactions are indivisible, all-or-nothing • Consistency • Transactions are all-at-once • {create A, create B} and {create B, create A} are identical • Independence • Transactions do not interfere with each other • Durability • Committed data remains in the system even in case of a restart, etc
- 6. 6©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Database Storage MO = Managed Object (i.e. configuration consumer) JOURNAL START TRANSACTION WRITE 14 TO /some/leaf/somewhere WRITE “hello” TO /some/other/leaf COMMIT START TRANSACTION WRITE -1 TO /foo/bar ABORTED START TR ON DISK Values stored binary Paths stored as sequences of 32-bit hashes ConfD CDB IN MEMORY MO MO MO
- 7. 7©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Data Model Becomes Database Schema Network Element NETCONF SNMP REST CLI Web CDB Database Data Provider API CDB API Management Agent API OSS/BSS App AppApp ConfD Core Engine App External DatabaseOperational Data App Management Agent API Auto-rendered management interfaces and data schema YANG Data Models
- 8. 8©2015 Tail-f Systems all rights reservedFebruary 13, 2015 JOURNAL ConfD CDB IN MEMORY MO MO MO Database Replication ACTIVE NODE Journal records JOURNAL ConfD CDB IN MEMORY MO MO MO STANDBY NODE
- 9. 9©2015 Tail-f Systems all rights reservedFebruary 13, 2015 CDB Start • CDB in-memory contents blank at this point • Read up schema from disk • Start transaction • Read up data from disk • Data on disk consistent with schema? • If not: Trigger automatic schema upgrade (see next page) • CDB doesn’t differentiate between up- or downgrade, it just notices that the schema is different • CDB does not rely on version in the namespace declaration • Validate • Commit
- 10. 10©2015 Tail-f Systems all rights reservedFebruary 13, 2015 CDB Automatic Schema Upgrade • Elements with default value added to schema • Default values added to new database • Elements removed from schema • Values removed from new database • Elements with trivial type changes • Value promoted to new type in new database • Data is converted to string representation, then to new type • Elements with complex type changes • No value in new database • All other elements • Copied as is to new database Check the devel.log for messages about what conversions were made
- 11. 11©2015 Tail-f Systems all rights reservedFebruary 13, 2015 CDB Programmatic Schema Upgrade Then add your own upgrade clients • Complement to the automatic schema upgrade • Connect in start phase 0 • Compute values for the new database • Read any old or new database values • Compute new values • Write (overwrite) new values • If the schema is changed cautiously, CDB automatic schema upgrade might be able to handle everything
- 12. 12©2015 Tail-f Systems all rights reservedFebruary 13, 2015 CDB Schema Upgrade Examples • 32-bit flag field becomes 64-bits • New flags should be initialized to all zeros • Automatic upgrade sufficient • New Maximum Transfer Unit (MTU) field added • Should be 1500 for Eth interfaces, 48 for ATM interfaces • Upgrade client reads InterfaceType value, classifies it and writes the appropriate MTU value • InterfaceType changed from string to enumeration • All strings that match enumeration value are auto upgraded • Remaining values left unset: set by upgrade client • Time field changed from seconds to nanoseconds • Upgrade client reads the old value, multiplies by 109 and overwrites the automatically promoted value
- 13. 13©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Data Stores • Each data store is a full copy of the configuration • Each data store can be mapped to CDB or external data base • Running can be split across CDB and several data bases • 5 combinations (S+Rro, S+R, R, R+C, Rro+C) • Operational data store • Store status, performance or application data CandidateStart-up Running Operational
- 14. 14©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Data Stores • Start-up • If enabled, start-up configuration is copied to (overwriting) the running configuration at system start • If not enabled, configuration in running remains unchanged from last time system was running • The first time the system starts, configuration is read from factory default files ( *_init.xml ) • Running • Holds the currently running configuration • Candidate • Data store for future configuration under construction • Copied to running to make it take effect
- 15. 15©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Sessions • Session Start (login) • Display values • Update values • Request Commit • Lock data store (running, candidate, start-up) • Run validation sequence • Run Transaction (next slide) • Wait for acknowledgement from standby node(s) • Inform subscribers of changes • Unlock data store • Session Continues
- 16. 16©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Transactions
- 17. 17©2015 Tail-f Systems all rights reservedFebruary 13, 2015 The Life of a Managed Object • Subscribe to relevant configuration changes • Read out configuration at startup • Enter event loop • If subscription event, react to configuration change RESTCLINETCONF SNMP Management Agent API Managed Object API MO MO CDB ConfD Core Engine CDB API Data Provider API Web
- 18. 18©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Read out Configuration at Startup • Initialization • cdb_connect() • cdb_start_session() • cdb_set_namespace() • Read elements • cdb_get_u_int32() • cdb_get_bool() • cdb_get_str() • cdb_get_ipv4() • cdb_get_enum_hash() • … (~30 more) • Read many • cdb_get_object() • cdb_get_values() • Count elements • cdb_exists() • cdb_num_instances() • Move in data tree • cdb_cd() • cdb_pushd() • Finalization • cdb_end_session() • cdb_close()
- 19. 19©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Read out Configuration at Startup Sessions • CDB protected with Single Writer- Multiple Reader lock • No configuration changes can occur between cdb_start_session() and cdb_end_session() • Don’t keep the session open unnecessarily, delays configuration changes • Timeout after some minutes Namespaces • Each data model is defined in an XML Namespace • ConfD handles multiple data models/namespaces • Each subsystem should be separate namespace • Generate header file with namespace hash constant sock = socket(PF_INET, SOCK_STREAM, 0); cdb_connect(sock, CDB_READ_SOCKET, &confd_ip_port, size); cdb_start_session(sock, CDB_RUNNING); cdb_set_namespace(sock, dhcpd__ns);
- 20. 20©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Read out Configuration at Startup Keypaths • Each element in CDB is addressed by its path cdb_get_duration(sock, &dur, “/dhcp/defaultLeaseTime”) • Paths may include keys cdb_get_ipv4(sock,&ip,”/dhcp/SubNets/subNet{10.0.0.0 255.0.0.0}/lowAddr”) cdb_get_ipv4(sock, &ip, ”/dhcp/SubNets/subNet{%s %s}/lowAddr”, net, mask) cdb_get_ipv4(sock, &ip, ”/dhcp/SubNets/subNet{%x %x}/lowAddr”, net, mask) • Index may be used in place of keys n = cdb_num_instances(sock, "/dhcp/SubNets/subNet"); for (i=0; i<n; i++) { cdb_cd(sock, "/dhcp/SubNets/subNet[%d]", i); do_subnet(…); }
- 21. 21©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Triggers when anything in scope changes Subscribe to Configuration Changes subsock = socket(PF_INET, SOCK_STREAM, 0); cdb_connect(subsock, CDB_SUBSCRIPTION_SOCKET, &confd, size); cdb_subscribe(subsock, 3, dhcpd__ns, &spoint, "/dhcp"); cdb_subscribe_done(subsock); / dhcp/ aaa/ voip/ defaultLe… maxLeas… SubNets/ subNet/ mask range lowAddr highAddr logFacility net Managed Objects typically have several subscription points
- 22. 22©2015 Tail-f Systems all rights reservedFebruary 13, 2015 Enter Event Loop • Event loop needs to monitor subscription socket • Typically using poll()/select() while(1) { set[0].fd = subsock; set[0].events = POLLIN; set[0].revents = 0; poll(set, 1, -1); // Monitor one socket, wait forever if (set[0].revents & POLLIN) { cdb_read_subscription_socket(subsock, sub_points, &reslen); // React on changes here… // sub_points[] contains triggering subscription points, // reslen how many } cdb_sync_subscription_socket(subsock, CDB_DONE_PRIORITY); } • How to handle CDB subscription notifications will be covered in the training module “CDB Subscribers”
- 23. 23©2015 Tail-f Systems all rights reservedFebruary 13, 2015