Splunk Enterprise for InfoSec Hands-On Breakout Session

Editor's Notes

• #2: - Real world problems and data - Tons of ways to do searches - Context-specific adjustments - Starting points - Because real data, real noise to filter. - Some quirks to the data
• #3: We’ll do a intro of Splunk for security. We’ll then move pretty quickly to our hands-on exercise in three phases where we cover web attacks, lateral movement, and DNS exfil The Splunk examples will go from very simple, to very complex – all in the course of an hour Afterwards you will have all of the material we have gone through, and the servers will stay up through tonight If you require some additional time, we can discuss that after the talk
• #5: Splunk excels at creating a data fabric Machine data: Anything with a timestamp, regardless of incoming format. Throw it all in there! Collect it. Store it in one place. Make it accessible for search/analytics/reporting/alerting. DETECTION NOT PREVENTION! ASSUME BREACH! So we need a place we can go to DETECT attacks. DETECT breaches. DETECT the “weird.” So if you had a place to see “everything” that happened… ….what would that mean for your SOC and IR teams?
• #6: The Splunk platform consists of multiple products and deployment models to fit your needs. Splunk Enterprise – for on-premise deployment Splunk Cloud – Fully managed service with 100% SLA and all the capabilities of Splunk Enterprise…in the Cloud Splunk Light – log search and analytics for small IT environments Hunk – for analytics on data in Hadoop The products can pull in data from virtually any source to support multiple use cases. Splunk Apps extend and simplify deployments by providing pre-packaged content designed for specific use cases and data types.
• #7: Our rapid ascent reflects the customer traction we have and value we deliver to customers – with thousands of security customers and 40% year-over-year growth, we are the fastest growing SIEM vendor in the market. 2011 was our first time in the MQ; In 2 short years we raced up to the top quadrant in the MQ.
• #9: The goal of the OWASP (Open Web Application Security Project ) Top Ten is to raise awareness about application security by identifying some of the most critical risks facing organizations today. This list is refreshed every three years – we’re currently waiting on the 2016 one. These were the top 10 through 2 for 2013. What is #1?
• #10: These are examples of injections flaws that occur when untrusted data is sent to an interpreter as part of a command or query. The attacker essentially tricks the interpreter into executing unintended commands or bypassing authorization/authentication. Most of us are familiar with SQL injection attacks, the risks they pose to our environments and maybe even the inner workings of the threat itself, because they have been around for about 10 years. However, the attack types can vary and mitigating them efficiently requires an understanding of the basic types of SQL injection attacks. SQL injection has been around so long…
• #11: …so long, that this is what Google looked like when SQL injection first became a topic of discussion: 1998. With 18 years of ways to prevent SQLi, you would think that this isn’t a problem anymore, right?
• #12: Wrong. Check out this survey conducted by Imperva – ostensibly to sell their web application firewalls, but still. This survey was just conducted last year…we can see that SQL injection attempts appear to happen in concentrated bursts, perhaps the result of specific campaigns. Regardless of when they happen, SQL injection attacks remain some of the most damaging to organizations in terms of records compromised and lost value. Attribution: http://www.imperva.com/docs/HII_Web_Application_Attack_Report_Ed6.pdf
• #13: A SQL injection needs just two conditions to exist – a relational database that uses SQL, and a user controllable input which is directly used in an SQL query. An attacker would get into your organization though unsanitized user input. Instead of entering a “real” username or password, the hacker will enter a SQL command (or a JSON string in the case of a NoSQL database). As a result, the malicious code, the example SQL query shown here, is then sent to the database. Cybercriminals use injection attacks against databases to export data, to delete accounts, create bogus accounts and modify data. Injection attacks can even be used to initiate a Denial of Service (DOS) attack. There have been some notable SQL injection attacks in our recent history that have exfiltrated entire user databases, prompting system-wide password resets, such as…
• #14: Here’s two big SQLi injection breaches that happened last year TalkTalk and vTech – resulting in the theft of about 10M records between the two of them – a lot of them in the vTech breach were kids – including names, genders, birthdates, and addresses.
• #15: And so far this year – we’ve had at least 9 major breaches covered in the media that involved SQLi. You can just google “SQL injection hall of fame” to pull this up. Attribution: The Code Curmudgeon Blog http://codecurmudgeon.com/wp/sql-injection-hall-of-shame/
• #16: We’re going to do some hands-on here, but before we do, let me set the stage a bit. How many of you have had your hands on Splunk before? OK – for the purposes of this discussion, you all are ninjas.
• #17: For the newbies – or for anyone who wants a little more help – the app we’re going to use this afternoon is very self-guided. You can literally just read along and see the search examples without having to type any Splunk searches.
• #18: This is what that looks like – the search is shown, and the results below.
• #19: You will notice that in the app there’s this “step by step” button. Don’t press it – what it does is it allows you to step through the search line by line – great for learning, but not great for this shared environment where you’re all hitting the same servers.
• #20: This is what that step by step looks like. You can use it later on your own time.
• #21: Now if you are a ninja you can simply cut and paste the searches into the Search bar within the app – that’s what I will do up here.
• #22: So what I will be doing is mousing over the search…
• #23: …and pasting it in to the search bar.
• #24: A little bit about our environment
• #26: The answer is Splunk apps! Apps come with pre-built content, like dashboards, reports, alerts and workflows that anyone can leverage as a starting point or template and adapt them accordingly to their environment. We have a community of 11,000+ Splunk customers and this is one of the 1,000+ apps on SplunkBase (stats as of June 2016). New Splunk users often try to reinvent the wheel, when it turns out it is already there. In this case, a Splunk engineer wrote an app that searches for SQLI, and then published it on SplunkBase. He provides a broader regex that we are going to leverage for our next example.
• #27: In that app one of the things provided is a regular expression pre-created for you to find common patterns in your data that could indicate a SQL injection attack. So all we need to do is call the macro when we do the search.
• #29: If you’d like, you can take a look at the dashboards in the SQL injection app on your own time.
• #30: In summary, why does SQL injection attacks continue to be one of the most common? BECAUSE IT STILL WORKS!! Don’t waste your previous cycles to reinvent the wheel, search for an existing app, or even provide one to the community. While there are great Web Application Firewalls and similar products that will sit inline and detect or even block these attacks, you will still benefit from looking through all of your data for any signs of SQL injection, and Splunk can help. If you have a specific question or use case that you want to run by us, come find us after the session.
• #33: What if they use that to pivot around the environment to get to other more sensitive systems?
• #34: Pass the hash – dump the hash tables from a system, then use those hashes to authenticate to other systems and move around the environment or proliferate malicious code.
• #35: Guess what – this is pretty common – the Sands breach and Sony breaches from last year used PtH to move around the environment and in both cases distribute destructive malware. I mention these two because they were prominent and well researched.
• #36: It used to be that PtH was pretty easy to find – and in some cases it still is. Basically we just look for certain Windows events, and if the user account is not domain or anonymous, or specifies an unusual domain – that’s suspicious. Today, a not-very-sophisticated actor might show up this way if we look for them…so let’s see how we can do that.
• #38: If you look at some of the whitepapers and other links in the app you will see that the “old” ways of detecting PtH are not enough. Instead we really need to look at user and entity behavior and find unusual network connections and login patterns in our data. Again – splunk can help here…
• #45: Recalling back to the magical UBA solution with the three high-level stages of the kill chain: We started with an intrusion through the use of SQL injection, expansion through lateral movement, and now we will discuss exfiltration via DNS.
• #46: Recalling back to the magical UBA solution with the three high-level stages of the kill chain: We started with an intrusion through the use of SQL injection, expansion through lateral movement, and now we will discuss exfiltration via DNS. Here is a Dave‘s credentials in clear text; <CLICK> And what it looks like encoded <CLICK> An attacker could send that encoded string, containing Dave‘s credentials via a DNS request bound for his own DNS server. By the way – there are other ways to exfil data, too – you can use TXT records and chunk data into that and tunnel it. Our examples here however will exfil data right in the domain name.
• #47: Raise your hand if you log your DNS data? Do you look at it? The point is: DNS data tends to be too chatty to log and often overlooked. The same goes for an ICMP content body.
• #48: There are a couple of retail breaches that we have called out here. We will be posting all session slides to Slideshare, so you can view these articles at your leisure. FrameworkPOS is a packaged tool that hackers can buy and put into their hacking arsenal along with dnscat. Just like the hacker community have their tools, our Splunk community does too!
• #49: URL Toolbox can parse URLs and complicated top level domains by dissecting a domain into its parts, such as subdomain, domain name and top level domain. It can also perform functions like shannon entropy, counting, suites, meaning ratio, bayesian analysis, etc. But our focus today will be on the use of Shannon Entropy.
• #50: The Shannon Entropy will give us a numeric representation for how much randomness there is in the letters / numbers used in a subdomain. It is a good indication of Domain Generation Algorithms (DGA) or of data exfiltration because compressed and/or encrypted data will have very high entropy.
• #51: Let’s all go to the DNS Exfil menu and select Detecting Data Exfiltration from the dropdown. Again, using pre-built content, we can parse out the query and domain components and calculate the Shannon Entropy for each subdomain. Since most data exfiltration occurs via subdomain, that’s where we will be focusing our search. If you recall back to the tornado slide, the attacker used encoded credentials as a subdomain for the DNS request.
• #52: Large volumes of data exfiltration tend to have relatively similar data length because it is chunking big segments of data. A low standard deviation will help indicate that this is large volume data transfer.
• #53: How many DNS requests would it take to exfiltrate a MB of data? How about a GB? This is why we look for high counts. Here, an 18K text file is being exfiltrated to chickenkiller.com and a 20MB+ zip file is being exfiltrated to mooo.com.
• #55: In summary, don’t overlook the value in your DNS and ICMP data. If you don’t log it, you could consider grabbing the data right off the wire with Splunk Stream. If you don’t have logs or Stream, at a minimum, trend your firewall byte counts.
• #58: We’re headed to the East Coast! 2 inspired Keynotes – General Session and Security Keynote + Super Sessions with Splunk Leadership in Cloud, IT Ops, Security and Business Analytics! 165+ Breakout sessions addressing all areas and levels of Operational Intelligence – IT, Business Analytics, Mobile, Cloud, IoT, Security…and MORE! 30+ hours of invaluable networking time with industry thought leaders, technologists, and other Splunk Ninjas and Champions waiting to share their business wins with you! Join the 50%+ of Fortune 100 companies who attended .conf2015 to get hands on with Splunk. You’ll be surrounded by thousands of other like-minded individuals who are ready to share exciting and cutting edge use cases and best practices. You can also deep dive on all things Splunk products together with your favorite Splunkers. Head back to your company with both practical and inspired new uses for Splunk, ready to unlock the unimaginable power of your data! Arrive in Orlando a Splunk user, leave Orlando a Splunk Ninja!