Creating SaltStack State data with Pyobjects
Download as PPTX, PDF5 likes2,717 views
Pyobjects is an alternative renderer that allows you to author SaltStack state data in pure Python using a Pythonic API. This presentation takes an in-depth look at the motivation behind creating the SaltStack Pyobjects renderer and cover how to use it, and best practices.
Ad
More Related Content
What's hot (19)
Viewers also liked (15)
Ad
Similar to Creating SaltStack State data with Pyobjects (20)
Ad
Recently uploaded (20)
Creating SaltStack State data with Pyobjects
- 1. Creating State data with Pyobjects A high level Pythonic API Evan Borgstrom #SaltConf15 Site Reliability, LinkedIn
- 2. Pyobjects? • It’s an alternative renderer for your state data • High level “Pythonic” API • Part of SaltStack since 2014.7.0
- 3. Context • Long time user & contributor • Ended up with a very large state tree and became frustrated with YAML + Jinja • Began initial brainstorming late 2013 • Finally inspired to start on Pyobjects at SaltConf 2014
- 4. Motivation • 6500+ lines over hundreds of states (plus pillars) • YAML no longer felt like markup due to all of the Jinja syntax
- 7. + #!py
- 12. Basics
- 13. Batteries available • Fully fledged Python environment • Import whatever you need to compile your data • Here be dragons: – Blocking operations – Monkey patching (gevent, etc)
- 14. No boilerplate • Built-in run() function takes care of collecting and returning data. • No need to import import anything from Pyobjects, everything is included • Should be as natural as writing YAML
- 15. State objects • One per State available on the minion • CapWords version of the name – file -> File – postgres_user -> PostgresUser – ssh_known_hosts -> SshKnownHosts • Exposes each state function as a function – File.managed, Pkg.installed, etc…
- 17. Including • include() function takes a variable number of arguments • One per state to include • Can include any state, not just pyobjects
- 18. Extending • The extend() function is used in place of the ID being passed to the state. • Works exactly the same as the extend declaration in YAML
- 19. Importing • State trees will eventually become complex enough that you’ll want to create reusability • Pyobjects provides an augmentation to the import statement to allow salt:// URIs
- 20. Context Managers / Requisites
- 22. Maps
- 23. Maps 101
- 24. Targeting using grains • Preferred way of building configuration • Allows for dynamically changing values without conditionals • Matches against the inner class name • The grain defaults to os_family – Add __grain__ to the class to change it
- 25. Merging with Pillar data • Often times you’ll want to change configuration under some special conditions • Pillars provide the perfect platform for this • Specify the pillar name in the merge attribute in the main class
- 26. Best Practices
- 27. Pyobjects compliments YAML • Pyobjects only makes sense when you need to introduce logic into your state generation • YAML should still be used when do you need programmatic functions
- 28. Move logic out of templates
- 29. Pillars + Maps • Maps provide default values for all of your baseline settings • Pillars allow customization using Salt’s targeting system • Keeps all of the configuration with the code
- 31. Lazy loading of modules • The loader is lazy (pull #20274) • Pyobjects triggers loading of all state modules • Execution time on busy minions can be decreased
- 32. Reactor files • Reactor files are prime candidates to benefit from the separation of logic & presentation • Need to support the Local, Runner & Wheel prefixes in a Pythonic way
- 33. Map support for complex Grains • Currently Grains are limited to matching against the syntax supported by Python class names • Will be adding support through another “dunder” attribute that allows for complex grain matching – All numbers – Patterns
- 34. FIN. Please provide session feedback in the SaltConf15 mobile app @borgstr / #SaltConf15
Editor's Notes
- #3: Welcome to Pyobjects. Question: How many people have used Pyobjects? Other renders? Even knew you could change the way your states are interpreted? This talk will cover some background (the “why” question), provide an introduction and look at best practices. Questions @ the end.
- #4: Info on me. Used salt since 2011. (Background on FatBox + current work details) First version 0.7.0 (April 2011). Rolled out 0.8.8 to production. First contribution 2012. (Worked lots on mysql & virt modules) Pyobjects was originally called NaCl (sodium chloride)
- #5: All aspect of our systems were covered by states. YAML is a great place to start. Natural, easy to pickup. Jinja is a great way to remove some boilerplate, but soon it becomes a burden. (too much syntax, hard to manage as your state tree grows) Once your states become sufficiently complex you begin to write code. With YAML+Jinja this means your presentation and logic become mixed. The salt docs says “Simplicity, Simplicity, Simplicity”
- #8: Looking for a better way. Change renderer with shebang. Renders introduced in 0.8.8 Many different renderers ship with salt. Pure python seems like a great fit. (Tons in std lib, good support for data structures). Thought provoking -- Lots of ideas. Pydsl… (too much like javascript. Provides too many ways to do the same thing. Didn’t feel Pythonic to me [backslashes to continue lines])
- #9: The Python renderer ends up being too verbose since you need to replicate the YAML structure. Need run(). Using pure Python removes the mix so you can use just code to build your data, but it’s not a natural interface as you need to manually compose complex dictionary structures.
- #10: An API for the basic Python renderer. Pyobjects aims to bridge the gap by providing an interface that allows you to use rich, expressive syntax in place of complex dictionaries
- #11: 1/3 fewer lines More expressive. Utilizes all of Python’s awesomeness. (meta-classes, context managers) Pythonic
- #14: Eval’d inside of the minion process.
- #15: Un-Pythonic. I know…
- #16: Core of Pyobjects
- #17: Each object is a StateFactory. The StateFactory knows which methods are valid thanks to the Salt loader. Each of the methods on the StateFactory creates a State object The first argument is the id of the state. (which also automatically becomes the name, just like YAML). The kwargs become the parameters to the state, the same as YAML.
- #18: Include() simply adds references to the states into the final compiled dictionary given to Salt, nothing specific to pyobjects is going on.
- #19: Instead of creating a new state, it simply extends an existing one. The state you’re extending must be included first! (same as YAML) Contrived example, use watch_in or a context manager instead for requisites.
- #20: All objects that are imported are put in the global state, regardless of where you import them in your code. First version is more like “from X import *” Behind the scenes: Loader is used to fetch the file from the URI specified The fetched file is passed through Pyobjects Relevant objects are pulled out of the final compiled code
- #21: Talk about requisites in general. Both of these blocks are equivalent. The objects returned from any of the methods on the StateFactory and State objects are able to be used as a context manager. You can assign and use later if you want. Context managers are not restricted to states defined in Pyobjects. You can pass anything to the StateFactory (ie. Pkg(“something-defined-elsewhere”))
- #22: In the spirit of Python. All of the dunder objects are available Pyobjects provides shortcut methods for them to keep syntax clean
- #24: Extends from Map. Meta classes for targeting. Class name matches on the grain. Grain’s default to ‘os_family’, __grain__ changes that.
- #26: Talk about how building pillar data through top files is very flexible and the way you want to provide per-machine configuration overrides.
- #29: Example from the mine documentation. Possible with YAML. Best practice because we’re separating code from presentation. With YAML you’re still mixing them. Keep your templates clean.
- #32: Lazy loading pull by Thomas Jackson