Getting Started with Splunk
4 likes1,209 views
What is Splunk? At the end of this session you’ll have a high-level understanding of the pieces that make up the Splunk Platform, how it works, and how it fits in the landscape of Big Data. You’ll see practical examples that differentiate Splunk while demonstrating how to gain quick time to value.
Ad
More Related Content
What's hot (20)
Viewers also liked (9)
Ad
Similar to Getting Started with Splunk (20)
Ad
More from Splunk (20)
Recently uploaded (20)
Getting Started with Splunk
- 1. Copyright © 2013 Splunk Inc. June 11, 2015 Getting Started User Training Getting Started with Splunk Enterprise Ryan Ahlers – Splunk Engineer
- 2. Agenda • Splunk Enterprise Overview • Using Splunk (Live) – Installing, Indexing, Searching, Reports & Dashboards, Alerting • Deploying Splunk • Splunk Community (Apps, portals, docs, etc.) 2
- 4. Splunk Inc. 4 Public company, founded in 2004 Headquartered in San Francisco Universal Platform for Machine Data Any Machine Data Any Volume Deployments from 10MB to 350TB/day On Premise In the Cloud SAAS 9,000+ Customers in 100+ Countries 2/3 of the Fortune 100
- 5. What is Machine Data? Sources Order Processing Twitter Care IVR Middleware Error
- 6. Machine Data Contains Critical Insights Order ID Customer’s Tweet Time Waiting On Hold Product ID Company’s Twitter ID Order ID Customer ID Twitter ID Customer ID Customer ID Sources Order Processing Twitter Care IVR Middleware Error
- 7. Machine Data is Growing Exponentially Volume | Velocity | Variety | Variability GPS, RFID, Hypervisor, Web Servers, Email, Messaging, Clickstreams, Mobile, Telephony, IVR, Databases, Sensors, Telematics, Storage, Servers, Security Devices, Desktops Machine data is the fastest growing, most complex, most valuable area of big data
- 8. Universal Platform for Machine Data 8 Real-time indexing of ANY machine data Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID On- Premise Private Cloud Public Cloud Local Storage SAN NoSQL Explore Visualize ShareAnalyze Develop
- 9. Universal Platform for Machine Data 9 Real-time indexing of ANY machine data Online Services Web Services Servers Security GPS Location Storage Desktops Networks Packaged Applications Custom ApplicationsMessaging Telecoms Online Shopping Cart Web Clickstreams Databases Energy Meters Call Detail Records Smartphones and Devices RFID On- Premises Private Cloud Public Cloud Developer Platform Report and analyze Custom dashboards Monitor and alert Ad hoc search Any amount, any location, any source Schema-on-the-flyNo Database No need to filter data
- 10. Splunk Delivers Value Across IT and the Business IT Operations Security & Compliance Web Intelligence Application Management Developer Platform (Java, Python, JavaScript, PHP, SDKs, REST API) Business Analytics Industrial Data Small Data. Big Data. Huge Data.
- 11. Insights Across Roles & Departments Product Managers Sales Operations Executive Management Customer Service & SupportIT Management & Operations Marketing Managers 11
- 12. Scales to Hundreds of TBs/Day Enterprise-class Scale, Resilience and Interoperability Collect machine data from thousands sources via Splunk forwarders Compress and store data on Splunk Indexers Initiate searches and visualize results via Search Heads
- 13. Delivers Mission-critical Availability Data replication – maintain searchability even if servers go down Multi-site capable – maintain searchability even if a site goes down Search Affinity – optimized searches by fetching from the closest/fastest location REPLICATION Portland Datacenter New York Datacenter Clustering
- 14. Drastically Reduces Time-to-Value Over 600 apps available on splunkbase REST API XenApp XenDesktop Server, Storage, Network Server Virtualization Operating Systems Infrastructure Applications Mobile Applications Cloud Services Other Monitoring Ticketing/Help Desk Custom Biz Applications SDKs Web Framework
- 16. Install Splunk Splunk Home • WIN: Program FilesSplunk • Other: /opt/splunk (Applications/splunk) Start Splunk • WIN: Program FilesSplunkbinsplunk.exe start (services start) • *NIX: /opt/splunk/bin/splunk start www.splunk.com/download
- 17. Splunk Licenses Free Download Limits Indexing to 500MB/day • Enterprise Trial License expires after 60 days • Reverts to Free License Features Disabled in Free License • Multiple user accounts and role-based access controls • Distributed search • Forwarding to non-Splunk Instances • Deployment management • Scheduled saved searches and alerting • Summary indexing
- 18. Default installation on: http://localhost:8000 18 Splunk Console Browser Support • Firefox 10.x and latest • Internet Explorer 7, 8, 9 and 10 • Safari (latest) • Chrome (latest)
- 19. Indexing Demonstration Download the sample file, follow this link and save the file to your desktop, then unzip: http://bit.ly/UBPFWP (Exploring Splunk Book) To add the file to Splunk: – Click Add Data – Click Upload files from my computer. – Drag and drop you sample data zip file. – Add a new Index – Review and Finish. 19
- 20. Search & Alert Demonstration 20 Search App Field Extractions (Auto/Manual) Free-form Searching 130+ Commands
- 21. Report & Dashboard Demonstration 21
- 22. Settings Demonstration 22 For All of that Cool Stuff You Just Created (and more!) • Permissions • Saved Searches/Reports • Custom Views • Distributed Splunk • Deployment Server • License Usage….
- 23. Deploying Splunk
- 24. Splunk’s Core Components 24 A Splunk install can be one or all roles… Search HeadIndexerForwarder
- 25. Single Instance or Distributed? 25 < 150GB per Day > 1500GB per Day 6X2 Core CPUs/12GB RAM/800+ IOPs
- 26. Distributed Architecture Universal Forwarder 26 Collect and Forward Machine Data to Indexers May or May not be Required Agent or Agentless are both supported Overhead ~1% CPU, ~50MB RAM, ~256kb/sec
- 27. Distributed Architecture Indexer 27 Compresses, Index and Search up to 150GB/day Compressed Raw Data (~15% raw data size) Time Series Index (~35% raw data size) Executes Searches Scales Horizontally via Commodity Hardware 6X2 Core CPUs/12GB RAM/800+ IOPs
- 28. Distributed Architecture Search Head 28 Initiates Distributed Searches Publishes Reports/Dashboards/Apps Scales Horizontally via Commodity Hardware 4X4 Core CPUs/12GB RAM/2 x 300GB, 10,000 RPM SAS Raid 1
- 29. Scalability & High Availability 29 Forwarders load balance across Indexers Indexed data can be replicated across peers and different physical sites Search Heads can be Clustered to eliminate single point of failure and handle large search loads
- 30. Service Desk Event Console SIEM Send Data to Other Systems 30 Route raw data in real time or send alerts based on searches.
- 31. Integrate External Data 31 LDAP, AD Watch Lists CRM/ER P CMDB Correlate IP addresses with locations, accounts with regions Extend search with lookups to external data sources.
- 32. Integrate Users and Roles 32 Problem Investigation Problem Investigation Problem Investigation Save Searches Share Searches LDAP, AD Users and Groups Splunk Flexible Roles Manage Users Manage Indexes Capabilities& Filters NOT tag=PCI App=ERP … Map LDAP & AD groups to flexible Splunk roles. Define any search as a filter. Integrate authentication with LDAP and Active Directory.
- 33. Splunk’s Core Components 33 Time to start SPLUNKING!!! Search HeadIndexerForwarder
- 35. Support Through the Splunk Community 35 Browse and share Apps from Splunk, Partners and the Community splunkbase.splunk.com Splunkbase Community-driven knowledge exchange and Q&A answers.splunk.com 5 tracks, more than 40 sessions, the smartest Splunk users together conf.splunk.com .conf2014
- 36. Where to Go for Help 36 • Documentation – http://www.splunk.com/base/Documentation • Technical Support – http://www.splunk.com/support • Videos – http://www.splunk.com/videos • Education – http://www.splunk.com/goto/education • Community – http://answers.splunk.com • Splunk Book – http://splunkbook.com
- 37. Thank you November 12st, 2012 Thank You!!
Editor's Notes
- #5: Splunk Inc. is a public company, founded in 2004 with the goal of providing a universal platform for machine data. We have more than 9,000 global customers with deployments ranging from home use to massive enterprises indexing 250TB of data a day. So what is machine data?
- #6: Every machine on the planet, from internal servers and applications to call center hardware and the networks they run on to social media all generate some kind of structured or unstructured machine data.
- #7: Within that machine data are critical insights about the performance availability and value of business services provided to the end user. But there is a challenge with this Machine Data. It is growing exponentially.
- #8: But there is a big challenge with this Machine Data. It is growing exponentially both in volume as well as complexity. – And that’s where Splunk comes in.
- #9: No matter what type of machine data the systems powering your business produce, Splunk can collect and index that data in real-time; allowing you to immediately explore, Analyze and visualize that data, and achieving what we at Splunk refer to as “Operational Intelligence”. As machine data flows into Splunk, it is compressed and stored on local storage or a SAN for real-time investigation and alerting of incidents or Security challenges. NoSQL Datastores can be used to store longer-term machine data not required for real-time analysis or can be queried directly to correlate the petabytes of unstructured NoSQL Datastore data with the unlimited types of Machine data indexed by Splunk. In addition to a wizard-driven console for easily visualizing all of your data, Splunk provides a full development platform for collecting new Machine Data types or sharing data with third party products, ticketing systems or custom interfaces.
- #10: All of this is done without the limitation of a back end relational database. Which means the integrity of your data is never compromised, any field within your machine data is fair game for investigation, and scalability is never an issue.
- #11: Machine data is useful across both IT and the Business. Splunk use cases range from standard IT Operations management to Security and Business Analytics.
- #12: Splunk Dashboards are easy to generate and customize to meet the needs of a wide variety of roles within your organization. And no matter how much machine data is generated by your business processes and services, Splunk can scale to meet your needs.
- #13: Splunk Forwarders are lightweight components which collect Machine data throughout your environment. Forwarder deployment is highly customizable, you can have the forwarder remotely collect data or place the forwarder locally on hundreds of thousands of devices as some of our customers do. Forwarders automatically load-balance their collected machine data across a pool of Indexers, which scale horizontally on commodity hardware to adjust to your growing pool of Machine Data. Search Heads initiate map-reduced searches across the indexer tier, combine and return the results to the Splunk console or your interface of choice. Like Indexers, Search Heads can scale horizontally to meet your needs on commodity hardware.
- #14: Even if an entire datacenter were to go down, Splunk’s ability to replicate data across sites ensures a true high availability environment.
- #15: Finally, Splunk’s robust community of users and partners have created more than 500 apps which plug into Splunk Enterprise to get you up and running quickly. The vast majority of these apps are free. Some apps, which have dedicated SCRUM development and support teams are charged for. For Example: HUNK is an application which provides easy NoSQL Connectivity, Search, and Reporting capabilities across your NoSQL nodes The Exchange App provides out of the box monitoring of your email environment The VM Ware app monitors virtual environments. And Splunk’s Enterprise Security app provides Security Analysts with the ability to search across all Machine Data within an environment, not just data limited by a particular SIEM; which has propelled Splunk to a “Leader” in the Gartner Magic Quadrant for Security.