Ad
More Related Content
What's hot (20)
Viewers also liked (20)
Ad
Similar to ASGARD Splunk Conf 2016 (20)
Ad
Recently uploaded (20)
ASGARD Splunk Conf 2016
- 1. Copyright © 2016 Splunk Inc. Scaling Security Inves<ga<ons With Interac<ve Event Graphs & Spark Leo Meyerovich CEO/Co-‐founder, Graphistry Joshua PaKerson Principal Data Scien<st, Accenture Tech Labs
- 2. 36 Billion Alerts Per Year 100 Million Alerts Per Day 3 Billion Alerts Per Month How do we scale visibility around any individual alert? Alert counts from one enterprise security team ?
- 3. TALK: Architecture & Prac<ce Of Visualizing Events @ Scale Splunk/Spark/Graphistry à Security Event Graphs Fraud: Tracking Embezzlers Hunting: Daily Anomalies Shadow IT: Auditing Dropbox Ops: Outage Root CauseBotnet Deconstruction IR: Killchain Analysis
- 4. QUICK DEMO 4
- 5. About Graphistry 5 Mission Look and work across event systems with one intelligent layer Investigator BETA • Inves<ga<on <er with Splunk connector • Increase visibility & automa<on in inves<ga<ons • Handle increasingly large and diverse data sources Founding Team Spun out of UC Berkeley parlab in 2014 Core Technology Smart & scalable visual querying powered by GPUs, language design, and unsupervised learning
- 6. Silicon Valley • Digital Experiences • Ar<ficial Intelligence • Pla`orms & Systems Washington DC • Security Dublin • Ar<ficial Intelligence Sophia An8polis • Industry Innova<on (FS & Resources) Beijing • Industrial Internet Bangalore • Soeware Engineering Tel-‐Aviv • Security For more than 20 years, Accenture Labs has served as the tip of the spear for technology innovation at Accenture. Over the last 5 years Accenture Labs has: • Supported 300+ client engagements and hosted 1100+ client workshops • Published 200+ thought leadership pieces, filed 110+ patent applica<ons, and garnered 350+ Tier-‐1 media hits Accenture Labs: Expanding Global Presence 6 Copyright © 2016 Accenture All rights reserved.
- 7. Agenda 7 ASGARD End-‐to-‐End Architecture Big data rethink with Splunk, Spark, & Graphistry Hun<ng Demo: Notebooks for anomaly analysis Visual Science: Event graphs for scalable views (+ GPUs!) Incident Response Demo: Botnet outbreak
- 8. Who Visually Analyzes & How? 8 Escalation Chain Freeform Notebooks Premade Playbooks Search Apps Workflow automation SOC “triage” Response “dig” Forensics & Hunting “dig deep” Today’s Topic
- 9. Accenture ASGARD: Rethinking Cyber Security Analy<cs Hun<ng 9 Enable Incubate Discover Intellectual asset licensing Joint Ventures Products in- sourced for scale up Intellectual assets insourced for development Insourced ideas & technologies Out to Market Scale ASGARD Streaming Storage Analy8cs Visualiza8on Interac8on Copyright © 2016 Accenture All rights reserved.
- 10. Accenture Labs ASGARD Pla`orm 10 Copyright © 2016 Accenture All rights reserved. Ingest Event Processing Storage NotebooksQuery Layer Data Sources Visual Tier SQL Streaming py FIRST DEPLOYMENT • 100-‐400M events/day GOALS • Scalable • Interac8ve, Real-‐Time • Affordable THEMES • OSS Distributed In-‐Memory • GPUs • Events/Graphs
- 11. ASGARD Accelera<on Benchmarks 11 Everyday Scenario Time Period Without ASGARD With ASGARD ASGARD’s Speed Improvement 1 Network communica<on lookup, from one host (IP) to mul<ple hosts (IPs) 1 Day 3h 20m 13s 1m 44s 114 Times Faster 1 Week Not Feasible 4m 05s 2 Failed logon aKempts lookup for ac<ve directory 1 Day 18m 26s 1m 37s 10 Times Faster 1 Week 2h 13m 45s 3m 10s 41 Times Faster 3 Looking for malware (exe) in the Symantec logs 1 Day 3h 24m 36s 1m 37s 125 Times Faster 1 Week Not Feasible 1m 37s 4 Proxy Logs Lookup (looking for specific domain) 1 Day 4h 30m 13s 2m 54s 92 Times Faster 1 Week Not Feasible 1m 09s
- 12. Building For The Long-‐term: Innova<on Cycle 12 Customize, create, and iterate DATA SCIENCE ARCHITECTUREArchitecture Data Visualization Analytics Copyright © 2016 Accenture All rights reserved.
- 13. Hun<ng Demo (5min): Notebook For Daily Anomalies 13
- 14. Agenda ASGARD End-‐to-‐End Architecture Big data rethink with Splunk, Spark, & Graphistry Hun<ng Demo: Notebooks for anomaly analysis Visual Science: Event graphs for scalable views (+ GPUs!) Incident Response Demo: Botnet outbreak 14
- 15. Why Graph Visualiza<ons? 15
- 16. GOAL: Security Visualiza<on For The Data Era Scale visuals to modern enterprises 1 million devices under management Billions of events between them Reveal paKerns & outliers Explore at speed of thought Code less; easily pivot & drill Responsive: 10ms – 1s 16 Relevant Interac<ve
- 17. Lists Do Not Visually Scale Text search is a great star<ng point! 17 Do not scale Do not see the 30K+ events nor the IPs, users, nor how they relate…
- 18. Bar Charts Hide Rela<onships 18 ? • Good for summaries! • But not: rela<onships, paKerns, outliers • But not: individual items
- 19. Event Graphs: A Key Missing View 19 Unified Model • Describes en<<es & links, e.g., events • Mul<purpose: connect, see, interact Visual • Spot rela<onships, paKerns, outliers • Inspect individual items • Work at enterprise scale
- 20. Different Graphs for Different Scales, Ques<ons 20 Uni Ex: Network mapping “What services use this?” ip ip Hyper Ex: Incident Response “Did this escalate?” ip user event event user Mul< Ex: SSH trails “Is a user crossing zones?” ip user user ip ip
- 21. Graphistry’s GPU Pla`orm: Scale & Accelerate The Visual Analy<cs Tier 21 Optimized networking GPU analysis & MLGPU rendering (No JavaScript!) GovCloud
- 22. GPUs: Accelerate Every Component 10X+ Interac<ve Rendering 1+ million en<<es: 100X+ over D3.js Meaningful Viz: Layout & ML Smart clustering, coloring, sizing: 50X+ over Gephi Interac<ve Analy<cs Quickly drill down: 1 NVidia Tesla K80 = ~9 TFLOPS 22
- 23. Sample Speedup: Interac<ve Clustering 60X more data than Gephi Itera<ve clustering: pure GPU GPU in server via Node-‐OpenCL, Nvidia Docker 23 0.1 1 10 100 500K 1.0M 1.5M Graph Size: # Nodes + # Edges 20 Frames per second
- 24. Demo: Botnet Inves<ga<on (7min) 24
- 25. Lessons Learned ASGARD • Rethink security pla`orm for scale, speed, cost • Innova<on process for next-‐gen SIEM flow Graphistry • Event graphs: unify tools; explore behavior at scale • Inves<ga<on <er: increase visibility & streamline pivots
- 26. THANK YOU We’re hiring engineers + seeking innova<on partners! Leo Meyerovich [email protected] Joshua Patterson [email protected] G R A P H I S T RY