Splunk FISMA for Continuous Monitoring

Editor's Notes

• #3: A quick level set – elevator pitch – what is Splunk.
• #4: By “logs” we mean logfiles, generated all across IT by applications, servers and network devices.They contain data vital for diagnosing service problems, detecting sophisticated security threats and demonstrating compliance, and more.Getting the data you need when you need it is labor-intensive, complex, and in many cases not possible.Virtualization and SaaS adoption is growing, but with increased abstraction, there is also added management complexity.Gartner (2009) predicts enterprise data will grow by 650% over the next three years, 80% of this being unstructured IT data.Traditional tools are silo-based, built on rigid schemas so unable to cope with increasingly dynamic data and too costly because they rely on custom parsers unique to specific data sources and vendors.
• #5: Dashboards let you extend the power of your data to wherever it’s needed, by role and on an authenticated basis. With Splunk you can create custom dashboards in minutes with the dashboard editor and make more sense of the huge volumes of data at your disposal. Combine pre-defined searches, charts, alerts and reports into a powerful dashboard. Or create mashups with other Web-based Apps, such as Tivoli, SAP, security consoles and more. Now your management, security analysts, auditors, developers and sysadmins are all empowered to get the visibility, information and intelligence they need.
• #7: A quick level set – elevator pitch – what is Splunk.
• #8: A quick level set – elevator pitch – what is Splunk.
• #9: Splunk was built on our founders’ frustrations running some of the world’s largest data centers and e-commerce sites. Companies like Infoseek, Yahoo, Disney, all of which had issues managing large geographically dispersed, complex, and highly dynamic infrastructures.While they were surrounded by the most state-of-the-art IT management technologies available, they found it nearly impossible to easily troubleshoot, secure and audit these IT silos in their environments. They knew there was a better way and they founded Splunk.The concept behind Splunk is simple: if Google could make it possible for users to search billions of pages of Web content, why couldn’t we do that for the datacenter? That’s what they built, an engine to search, alert, monitor and report on all “IT data”. Search and analyze all your IT data from one location in real-time. IT data such as all your logs, messages, configurations, metrics in virtual and non-virtual environments. With Splunk, silos of data are eliminated enabling organizations to make better use of their IT data.Traditional approaches have been built using a “schema first” mindset and attempt to normalize every data sourceto fit it into thispredetermined database schema. This approach is costly and rigid. New data sources require new schemas or custom adapters. Much of the data is simply not ‘seen’. IT data on the other hand is becoming more dynamic and increasingly prone to change. Splunk eats any type of IT data: no database, no schema, no DBA, no RDBMS license, no custom connector and it scales on inexpensive commodity servers.
• #11: Leverage distributed search to give each locale access to their own data, while providing a combined view to central teams back at headquarters. Whether to optimize your network traffic or meet data segmentation requirements, feel free to build your Splunk infrastructure as it makes sense for your organization. Further, each distributed search head automatically creates the correct app and user context while searching across other datasets. No specific custom configuration management is required; Splunk handles it for you.
• #12: Splunk isn’t the only technology that can benefit from collecting machine data, so let Splunk help send the data to those systems that need it. For those systems that want a direct tap into the raw data, Splunk can forward all or a subset of data in real time via TCP as raw text or RFC-compliant syslog. This can be done on the forwarder or centrally via the indexer without incrementing your daily indexing volume. Separately, Splunk can schedule sophisticated correlation searches and configure them to open tickets or insert events into SIEMs or operation event consoles. This allows you to summarize, mash-up and transform the data with the full power of the search language and import data into these other systems in a controlled fashion, even if they don’t natively support all the data types Splunk does.
• #13: Your logs and other machine data are important but often cryptic. You can extend Splunk’s search with lookups to external data sources as well as automate tagging of hosts, users, sources, IP addresses and other fields that appear in your machine data. This enables you to find and summarize machinedata according to business impact, logical application, user role and other logical business mappings. In the example shown, Splunk is looking up the server’s IP address to determine which domain the servicing web host is located in, and the customer account number to show which local market the customer is coming from. Using these fields, a search user could create reports pivoted on this information easily.
• #14: Splunk allows you to extend your existing AAA systems into the Splunk search system for both security and convenience. Splunk can connect to your LDAP based systems, like AD, and directly map your groups and users to Splunk users and roles. From there, define what users and groups can access Splunk, which apps and searches they have access to, and automatically (and transparently) filter their results by any search you can define. That allows you to not only exclude whole events that are inappropriate for a user to see, but also mask or hide specific fields in the data – such as customer names or credit card numbers – from those not authorized to see the entire event.
• #15: Example Splunk Apps.