![17
location/pdx.yaml
profile::server::time_servers: "time.pdx.example.com"
common.yaml
profile::server::time_servers:
- 0.pool.ntp.org
- 1.pool.ntp.org
lookup('profile::server::time_servers', {merge => 'unique'})
[
'time.pdx.example.com’,
'0.pool.ntp.org',
'1.pool.ntp.org’,
]
Unique
strategy](https://image.slidesharecdn.com/hiera-180914121642/85/CCF-1-Taking-the-reins-of-your-data-with-Hiera-5-17-320.jpg)
CCF #1: Taking the reins of your data with Hiera 5
- 2. A brief introduction to Casual Config Fridays seminars and the idea behind them
- 3. S M T W T F S 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 < October >
- 4. CONFIG TRAINING A comprehensive introduction to the Puppet infrastructure Hostgroups And we can keep specializing… It’s everything about grouping it-puppet-hostgroup-webstuff webstuff/backendwebstuff/frontend webstuff/frontend/atlas webstuff/frontend/cms Welcome to the training Why are we here? Puppet infrastructure Get to know the infrastructure and all the components that allow us to offer Configuration management at CERN. High availability As part of the training you’ll learn how to define alias to ensure that at any moment multiple machines can attend your users’ requests. Handle secrets in your configuration No one wants to share a password with people who shouldn’t have access. Let’s use secrets! Configure just once Learning how to configure your services is as important as defining configuration in a way that you can replicate it easily across your machines. Learn the mechanism to share code Why spend time on tasks that other people in the community have solved before? Solve your questions We’ll cover what we think is important for you to know, but we are here to solve your questions. Don’t hesitate to raise your hand if you have any! Environments Differences between them 10 production qa new Production Machines in this environment will get all the config from the master branch of the repositories and is designed for production machines that focus on stability. QA This environment is usually used by a portion of the production machines to test new shared changes. All the code will come from qa branches. New At any point you can create a new environment to test changes in a more isolated way. 4 1
- 5. Introduction to the basic concepts of Hiera, usage examples and description of the priority chain
- 6. /haɪra/ 06 First things first Yeah, pretty much everyone says it wrong…
- 7. 07 Hiera 101 What it is and how it works A key/value store abstraction based in YAML files A key lookup resolution mechanism A hierarchical data organization A data composition mechanism (defaults, overrides, merges…)
- 8. 08 punch/puppet/ps.yaml --- osrepos_epel_exclude_pkgs: - puppetserver pluginsync_filter: - archive - puppetdbquery - puppetserver cernpuppet::puppetdb::puppetdb_server: "constable.cern.ch" A simple Hiera file A simple Hiera file
- 9. Understanding the hierarchy The global layer from the most to the least specific Fully qualified domain name Data can be assigned to a specific hostname Subhostgroups Searched from most to least specific Hostgroup and environment Different data can be assigned to, for example, production and qa Hostgroup and operating system Each hostgroup has hieradata available for operating sytem Toplevel hostgroup Has the last lookup in the hostgroup tree
- 10. 10 Environment Data specific to the Puppet environment Operatingsystem Global operating system variables Hardware vendor Specific to the vendor (or maintainer) of the hardware Module Data for the module Common Global data Datacenter Target the different datacenters (i.e. meyrin)
- 11. Learning to fetch your data using automatic lookups, useful functions and command line tools
- 12. 12 Getting your data Two different lookup mechanisms Automatic lookup Puppet automatically looks for class parameters values using the fully qualified name when those are not explicitly provided Explicit lookup Uses Hiera to retrieve the value for a key, allowing data validation and different strategies to define how to fetch data
- 13. 13 Automatic class parameter lookup hg_webserver/manifests/backend.pp class hg_webserver::backend { include ::yum } webserver/backend.yaml --- yum::clean_old_kernels: false yum/manifests/init.pp class yum ( Boolean $clean_old_kernels = true, ) { ... }
- 14. 14 code/manifest/webserver/frontend.pp class hg_webserver::frontend { $backend_url = lookup('backend_url') $backend_port = lookup('backend_port', { 'default_value' => 80, 'merge' => 'first', }) } webserver/backend.yaml --- backend_port: 8080 backend_url: "mywebserver.cern.ch" The lookup function
- 15. 15 Merging data Four different merge behaviors First No merge. First found, first used Unique (array merge) Combines any number of array and scalar values into an array Hash Combines keys and values of any number of hashes to return a merged hash Deep Similar to hash, but if the same key exists in multiple source hashes, Hiera recursively merges them
- 16. 16 location/pdx.yaml profile::server::time_servers: "time.pdx.example.com" common.yaml profile::server::time_servers: - 0.pool.ntp.org - 1.pool.ntp.org lookup('profile::server::time_servers', {merge => 'first'}) 'time.pdx.example.com’ First strategy
- 17. 17 location/pdx.yaml profile::server::time_servers: "time.pdx.example.com" common.yaml profile::server::time_servers: - 0.pool.ntp.org - 1.pool.ntp.org lookup('profile::server::time_servers', {merge => 'unique'}) [ 'time.pdx.example.com’, '0.pool.ntp.org', '1.pool.ntp.org’, ] Unique strategy
- 18. 18 lookup('site_users', {merge => 'hash'}) { "ash" => { group => "common", uid => 502, shell => "/bin/zsh" } "bob" => { group => "ops", uid => 1000 }, "jen" => { group => "ops", uid => 503, shell => "/bin/zsh" }, } groups/ops.yaml site_users: bob: group: ops uid: 1000 jen: group: ops shell: /bin/zsh uid: 503 common.yaml site_users: ash: group: common shell: /bin/zsh uid: 502 bob: shell: /bin/bash uid: 501 Hash strategy
- 19. 19 lookup('site_users', {merge => 'deep'}) { "ash" => { group => "common", uid => 502, shell => "/bin/zsh" } "bob" => { group => "ops", uid => 1000, shell => "/bin/bash" }, "jen" => { group => "ops", uid => 503, shell => "/bin/zsh" }, } groups/ops.yaml site_users: bob: group: ops uid: 1000 jen: group: ops shell: /bin/zsh uid: 503 common.yaml site_users: ash: group: common shell: /bin/zsh uid: 502 bob: shell: /bin/bash uid: 501 Deep strategy
- 20. 20 Deprecated functions Time to update your manifests hiera_array (141 uses) hiera (4381 uses) hiera_hash (258 uses) hiera_include (6 uses)
- 21. Definition of not-so-static Hiera values using variables and functions interpolations
- 22. 23 Interpolating variables Four different possible sources Puppet variables Most common way. Get’s the value from a Puppet variable Trusted hash Accurate values extracted from the node’s certificate Facts hash Contains all node’s facts. Structured ones are shown up as a nested structure
- 23. 22 webserver/backend.yaml --- # Puppet variable interpolation server_name: "%{servername}" # Facts hash interpolation smtpserver: "mail.%{facts.networking.domain}" # mail.cern.ch # Trusted hash interpolation webserver::frontend::backend_url: "%{trusted.hostname}.cern.ch" Interpolating variables
- 24. 24 Interpolating functions Lookups and beyond lookup Looks up a key using Hiera, and interpolates the values into a string scope An alternative way to interpolate a variable. Not generally useful literal A way to write a literal percent sign (%) without accidentally interpolating something alias Looks up a key using Hiera, and uses the value as a replacement for the enclosing
- 25. 25 webserver/backend.yaml --- # lookup interpolation webserver::backend::database_server: "%{lookup('mysql::public_hostname')}" # scope interpolation smtpserver: "mail.%{facts.domain}" smtpserver: "mail.%{scope('facts.domain')}" # literal interpolation server_name_string: "%{literal('%')}{SERVER_NAME}" # alias interpolation original: - 'one' - 'two' aliased: "%{alias('original’)}" Interpolating functions
- 26. Upgrading and cleaning our code with Hiera 5 and defining custom strategies
- 27. 27 Global, environment and module Three different layers of configuration Global layer Define all the levels of the hierarchy Environment layer Merged with the global layer in our deployment Module layer Allows to set defaults for a module’s class parameters
- 28. 28 Implementing Hiera 5 The hiera.yaml format Module level data is defined in a hiera.yaml file Must include the version (v5) The hierarchy key configures the data hierarchy The defaults key define default values for the backend and datadir keys
- 29. 29 hiera.yaml --- version: 5 defaults: datadir: 'data' data_hash: 'yaml_data' hierarchy: - name: 'Full Version' path: '%{facts.os.name}-%{facts.os.release.full}.yaml' - name: 'Major Version' path: '%{facts.os.name}-%{facts.os.release.major}.yaml' - name: 'Operating System Family' path: '%{facts.os.family}-family.yaml' - name: 'common' path: 'common.yaml' A hiera.yaml example
- 30. 30 Defining strategies The lookup_options key Any data source can set a lookup_options key This key controls the merge behavior of other keys Puppet lookups will first check for lookup_options The lookup_options keys are merged by Puppet using hash merge before deciding a merge behavior
- 31. 31 Defining strategies code/data/common.yaml lookup_options: ntp::servers: merge: unique "^profile::(.*)::users$": merge: deep ntp::servers: "ntp.cern.ch" code/data/rhel-7.yaml lookup_options: "^profile::(.*)::users$": merge: hash # Actual values after the hash_merge { "ntp::servers" => { merge => "unique" } "^profile::(.*)::users$" => { merge => "hash" }, }
- 32. Things to remember… Just an small summary 32
- 34. 34 Some useful links Click & Go Migrating to Hiera 5 Best module to use us an example of best practices Automatic class parameter lookup Hiera documentation in configdocs
- 35. 35 And some more… Again, Click & Go Interesting talk on Hiera by Hendrik Lindberg Further options for deep strategy behaviour More information on interpolation Deprecated functions and alternatives