AI & ML in Cyber Security - Why Algorithms Are Dangerous
19 likes14,029 views
Every single security company is talking in some way or another about how they are applying machine learning. Companies go out of their way to make sure they mention machine learning and not statistics when they explain how they work. Recently, that's not enough anymore either. As a security company you have to claim artificial intelligence to be even part of the conversation. Guess what. It's all baloney. We have entered a state in cyber security that is, in fact, dangerous. We are blindly relying on algorithms to do the right thing. We are letting deep learning algorithms detect anomalies in our data without having a clue what that algorithm just did. In academia, they call this the lack of explainability and verifiability. But rather than building systems with actual security knowledge, companies are using algorithms that nobody understands and in turn discover wrong insights. In this talk I will show the limitations of machine learning, outline the issues of explainability, and show where deep learning should never be applied. I will show examples of how the blind application of algorithms (including deep learning) actually leads to wrong results. Algorithms are dangerous. We need to revert back to experts and invest in systems that learn from, and absorb the knowledge, of experts.
Ad
More Related Content
What's hot (20)
Similar to AI & ML in Cyber Security - Why Algorithms Are Dangerous (20)
Ad
More from Raffael Marty (20)
Ad
Recently uploaded (20)
AI & ML in Cyber Security - Why Algorithms Are Dangerous
- 1. Raffael Marty AI & ML in Cyber Security Why Algorithms Are Dangerous Cancun, Mexico March, 2018
- 2. A Brief Summary 2 • We don’t have artificial intelligence (yet) • Algorithms are getting ‘smarter’, but experts are more important • Stop throwing algorithms on the wall - they are not spaghetti • Understand your data and your algorithms • Invest in people who know security (and have experience) • Build “export knowledge” absorbing systems • Focus on advancing insights
- 3. 3 The master of Kennin temple was Mokurai. He had a little protégé named Toyo who was only twelve years old. Toyo saw how students entered the masters room each day and received instructions and guidance in Zen. The young boy wished to do zazen (meditation) as well. Upon convincing Mokuri, he went in front of the master who gave him the following koan to ponder: "You can hear the sound of two hands when they clap together," said Mokurai. "Now show me the sound of one hand."
- 4. Outline 4 An Example Let’s Get Practical Statistics, Machine Learning & AI Defining the Concepts 1 2 The Algorithmic Problem Understanding the Data and the Algorithms 3
- 6. 6 “Everyone calls their stuff ‘machine learning’ or even better ‘artificial intelligence’ - It’s not cool to use statistics!” “Companies are throwing algorithms on the wall to see what sticks - see security analytics market”
- 7. ML and AI – What Is It? 7 • Machine Learning – Algorithmic ways to “describe” data o Supervised - learning from training data o Unsupervised - optimization problem to solve (clustering, dim reduction) • Deep Learning – a ‘newer’ machine learning algorithm o Eliminates the feature engineering step o Verifiability / explainability issues • Data Mining – Methods to explore data – automatically and interactively • Artificial Intelligence – “Just calling something AI doesn’t make it AI.” ”A program that doesn't simply classify or compute model parameters, but comes up with novel knowledge that a security analyst finds insightful.”
- 8. What “AI” Does Today 8 •Kick a human's ass at Go •Design more effective drugs •Make Siri smarter
- 9. Machine Learning Uses in Security 9 • Supervised o Malware classification (deep learning poster child) o Spam identification o MLSec project on firewall data • Unsupervised o DNS analytics (domain name classification, lookup frequencies, etc.) o Threat Intelligence feed curation (IOC prioritization, deduplication, …) o Tier 1 analyst automation (reducing 600M events to 100 incidents)* o User and Entity Behavior Analytics (UEBA) * See Respond Software Inc.
- 10. The Algorithmic Problem Understanding the Data and the Algorithms 10
- 12. Famous AI (Algorithm) Failures 12 neil.fraser.name/writing/tank/ US government in October 2016 published a comprehensive report titled “Preparing for the future of artificial intelligence”
- 13. What Makes Algorithms Dangerous? 13 • Algorithms make assumptions about the data • Assume ‘clean’ data (src/dst confusion, user feedback, etc.) • Often assume a certain type of data and their distribution • Don’t deal with outliers • Machine learning assumes enough, representative data • Needs contextual features (e.g., not just IP addresses) • Assume all input features are ‘normalized’ the same way • Algorithms are too easy to use these days (tensorflow, torch, ML on AWS, etc.) • The process is more important than the algorithm (e.g., feature engineering, supervision, drop outs, parameter choices, etc.) • Algorithms do not take domain knowledge into account • Defining meaningful and representative distance functions, for example • e.g., each L4 protocol exhibits different behavior. Train it separately. • e.g., interpretation is often unvalidated - beware of overfitting and biased models. • Ports look like numerical features, they are not, same is true for IPs, processIDs, HTTP return codes, etc.
- 14. Models Are Just Approximations 14 High Bias - increasing the number of input features. How do you know in what case you operate? • ML explainability problem • Compute error margins model classifies future instances high error failure to generalize High Variance reduce the number of input features, increasing the number of training examples
- 15. Cognitive Biases 15 • How biased is your data set? How do you know? • Only a single customer’s data • Learning from an ‘infected’ data set • Collection errors • Missing data (e.g., due to misconfiguration) • What’s the context the data operates in? • FTP although generally considered old and insecure, isn’t always problematic • Don’t trust your IDS (e.g. “UDP bomb”)
- 16. Don’t Use Machine Learning If … 16 • Not enough or no quality labeled data • Don’t use for network traffic analysis - you don’t have labeled data - really, you don’t! • No well trained domain experts and data scientists to oversee the implementation • Not enough domain expertise to engineer good features • Need to understand what ML actually learned (explainability) Also remember • Data cleanliness issues (timestamps, normalization across fields, etc.) • Operational challenges (scalability and adaptability) of implementing machine learning models in practice
- 17. Adversarial Machine Learning 17 • An example of an attack on deep learning
- 19. Network Traffic - Finding Anomalies / Attacks 19 • Given: Netflow • Task: Find anomalies / attacks 2 2005-10-22 23:09:45.903 -1.000 UDP 192.168.0.2 62569 -> 192.168.0.255 8612 0 0 1 0 .A.... 0 0 0 3 2005-10-22 23:09:53.003 -1.000 UDP 192.168.0.2 52457 -> 192.168.0.255 8612 0 0 1 0 .A.... 0 0 0 4 2005-10-22 23:09:58.435 -1.000 ICMP 192.168.2.2 0 -> 192.168.2.1 3.3 0 0 2 0 .A.... 192 0 0 5 2005-10-22 23:10:00.103 -1.000 UDP 192.168.0.2 59028 -> 192.168.0.255 8612 0 0 1 0 .A.... 0 0 0 6 2005-10-22 23:10:03.839 -1.000 UDP 192.168.2.2 138 -> 192.168.2.255 138 0 0 2 0 .A.... 0 0 0 7 2005-10-22 23:10:04.971 -1.000 UDP 192.168.0.2 17500 -> 255.255.255.255 17500 0 0 1 0 .A.... 0 0 0 8 2005-10-22 23:10:04.971 -1.000 UDP 192.168.0.2 17500 -> 192.168.0.255 17500 0 0 1 0 .A.... 0 0 0 9 2005-10-22 23:10:07.207 -1.000 UDP 192.168.0.2 62319 -> 192.168.0.255 8612 0 0 1 0 .A.... 0 0 0 10 2005-10-22 23:10:14.311 -1.000 UDP 192.168.0.2 50273 -> 192.168.0.255 8612 0 0 1 0 .A.... 0 0 0 11 2005-10-22 23:10:21.403 -1.000 UDP 192.168.0.2 56243 -> 192.168.0.255 8612 0 0 1 0 .A.... 0 0 0 12 2005-10-22 23:10:25.267 -1.000 ICMP 192.168.0.2 0 -> 192.168.2.1 8.0 0 0 1 0 .A.... 0 0 0 13 2005-10-22 23:10:28.043 0.004 ICMP 192.168.0.2 0 -> 192.168.2.2 8.0 0 0 1 2 .A.... 0 1338 1 14 2005-10-22 23:10:28.499 -1.000 UDP 192.168.0.2 62390 -> 192.168.0.255 8612 0 0 1 0 .A.... 0 0 0 15 2005-10-22 23:10:35.019 -1.000 UDP 192.168.0.2 17500 -> 255.255.255.255 17500 0 0 1 0 .A.... 0 0 0 16 2005-10-22 23:10:35.019 -1.000 UDP 192.168.0.2 17500 -> 192.168.0.255 17500 0 0 1 0 .A.... 0 0 0 ?
- 20. Network Traffic - Deep Learning 20 • Solution: Deep Learning • No feature engineering - really? • Lots of data available • What are the labels? Most security problems can’t be solved with Deep Learning
- 21. Analytics Challenges 21 • Data cleansing • Wrong features -> wrong results • Distance functions (for unsupervised approaches) • Selecting the right algorithm (there is more than k-means!) • Algorithmic stability across iterations (iterative)? • Parameter choices for algorithm
- 22. VAST Challenge 2013 Submission – Spot the Problems? 22 dest port! Port 70000? src ports! http://vis.pku.edu.cn/people/simingchen/docs/vastchallenge13-mc3.pdf
- 23. Distance Functions 23 • Need a domain-centric similarity function • URLs (simple levenshtein distance versus domain based?) • Ports (and IPs, ASNs) are NOT numerical features • Treat user names as categories, not strings outlier?!
- 25. Illustration of Parameter Choices and Their Failures • t-SNE clustering of network traffic from two types of machines perplexity = 3 epsilon = 3 No clear separation perplexity = 3 epsilon = 19 3 clusters instead of 2 perplexity = 93 epsilon = 19 What a mess
- 26. Illustration of Parameter Choices and Their Failures • Dangerous clusters
- 27. Network Traffic - Unsupervised Attempt 27 The graph shows an abstract space with colors being machine identified clusters. Preparation: • Feature engineering • Distance functions (what’s similar?) • Algorithm parameters Hard Questions: • What are these clusters? • What are good clusters? • What’s anomalous? What are the attacks?
- 28. The Real Problems 28 • Missing context • Asset inventory • User information • Missing expert knowledge • Domain expertise Possible Solutions • Enable data exploration • Improved Human Computer Interfaces & visualization • (Bayesian) Belief Networks
- 29. In Summary 29
- 30. Summary • Build solutions for actual problems with real data that produce actionable insight • Encode expert knowledge - leverage experienced experts • Use simple systems - how about letting users give input? Push problem to the edge • Don’t start with the algorithms - EVER • Start with the problem at hand and choose the right approach (hardly ever ML) • From the problem gather the right data and context • Use ML for problems where you have a large corpus of well labeled data • Chose meaningful distance functions • Verify your models - use visualization to help with that • Measure through user feedback, what you have implemented makes sense and pleases users • Allow for expert supervision - feedback loops • Share your insights with your peers – security is not your competitive advantage
- 31. BlackHat Workshop 31 Applied Machine Learning for Identity and Access Management August 4,5 & August 6,7 - Las Vegas, USA ML | AI | IAM http://secviz.org
- 32. Questions? 32 http://slideshare.net/zrlram @raffaelmarty "You can hear the sound of two hands when they clap together," said Mokurai. "Now show me the sound of one hand."