AI for security or security for AI - Sergey Gordeychik
- 1. S e r g e y G o r d e y c h i k H T T P : / / S C A D A . S L @ S C A D A S L s e r g . g o r d e y @ g m a i l . c o m Security for AI or AI for Security? h t t p s : / / c y b e r w e e k . a e
- 2. Sergey Gordeychik § AI and Cybersecurity Executive • Abu Dhabi, UAE § Visiting Professor, Cyber Security • Harbour.Space University, Barcelona, Spain § Program Chair, PHDays Conference • www.phdays.com, Moscow § Cyber-physical troublemaker • Leader of SCADA Strangelove Research Team • www.scada.sl, @scadasl § Ex… • Deputy CTO, Kaspersky Lab • CTO, Positive Technologies • Gartner recognized products and services 2
- 3. Disclaimer Please note, that this talk is by Sergey and AISec group. We don't speak for our employers. All the opinions and information here are of our responsibility. So, mistakes and bad jokes are all OUR responsibilities. 3 Actually no one ever saw this talk before. https://en.wikipedia.org/wiki/Terms_and_Conditions_May_Apply
- 4. 4
- 7. 7
- 8. 8
- 9. 9
- 10. 10
- 12. 12 Spherical AI traveling in a vacuum?
- 13. 13 What is Cyber? What is Cybersecurity?
- 15. 15 OT/ICS/SCADA Security?! SCADA Security Basics: Integrity Trumps Availability, ISA/IEC 62443-2-1 standards (formerly ISA-99) https://www.tofinosecurity.com/blog/scada-security-basics-integrity-trumps-availability Marina Krotofil, Damn Vulnerable Chemical Process https://fahrplan.events.ccc.de/congress/2014/Fahrplan/system/attachments/2560/original/31CC_ 2014_Krotofil.pdf
- 16. 16 Machine Learning and AI? AI security
- 19. 19 James Mickens, Harvard University, USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible? https://www.youtube.com/watch?v=ajGX7odA87k
- 20. 20 Mission-centric Cybersecurity Gapanovich, Rozenberg, Gordeychik, Signalling cyber security: the need for a mission-centric approach https://www.railjournal.com/in_depth/signalling-cyber-security-the-need-for-a-mission-centric-approach a process that ensures control object operation with no dangerous failures or damage, but with a set economic efficiency and reliability under adversarial anthropogenic information influence
- 21. 21 But what about?... dangerous failures? economic efficiency? reliability level?
- 22. 22
- 23. 23 But what about?... dangerous failures? economic efficiency? reliability level? Build the Threat Model First!
- 24. 24 AI Threat Model Li, K. (n.d.). Reverse Engineering AI Models.
- 25. 25 But what about?... §Cloud §AUC/ROC §Privacy §IP protection §Federative learning §Insane androids?… 25 AI security
- 26. 26 NCC Group, Building safer machine learning https://www.nccgroup.trust/uk/about-us/newsroom-and-events/blogs/2018/august/building-safer-machine-learning-systems-a-threat-model/
- 27. 27 Epoch I AI in da Cloud
- 28. 28 Cloud - CyberSec as usual? §InfiniBand and SDN §Security of ML/GPU servers • Supply chain • BMC/Firmware • GPU is a new CPU §Virtualization §Containers https://giphy.com/gifs/glas-2017-26gswvT0Ocx01b6AU
- 29. 29 SDN/SD-WAN NEWS BYTES § A vendor says its solution has the capability of “stitching together” WAN and Ethernet networks § Service providers are using SD-WAN to provide network agility § An SD-WAN router has an artificial intelligence (AI)-based routing service § A vendor announced that it would be unifying its security and SD-WAN
- 30. SDN/SD-WAN Security § C. Yoon, S. Lee, H. Kang, etc. Flow Wars § J. Hizver. Taxonomic Modeling of Security Threats in Software Defined Networking • S. Lal, T. Taleb, A. Dutta. NFV: Security Threats and Best Practices § SD-WAN New Hope, https://github.com/sdnewhop/sdwannewhope
- 31. SD-WAN New Hop - Hack before you buy! http://www.scada.sl/search/label/sd-wan
- 32. BMC/IPMI/UEFI 32• Remotely Attacking System Firmware, Jesse Michael, Mickey Shkatov & Oleksandr Bazhaniuk, BH18
- 33. 33 ML in da Cloud? To find a ML Server in the Internet?
- 34. 34 GPGPU?
- 35. 35 Crypto currency on GPGPU in 2019? https://www.zoomeye.org/searchResult?q=%2Bport%3A%225555%22%20%2Bservice%3A%22http%22%20NVIDIA
- 36. 36 SNMPWALK
- 37. 37 DGX-1 § 8 Tesla V100-32GB § TFLOPS (deep learning) 1000 § CUDA Cores 40,960 § Tensor Cores 5,120 § $130,000 § Good hashcat rate :) NetNTLMv2: 28912.2 MH/s MD5: 450.0 GH/s SHA-256: 59971.8 MH/s MS Office 2013: 163.5 kH/s bcrypt $2*$, Blowfish (Unix): 434.2 kH/s https://hashcat.net/forum/thread-6972.html
- 38. 38 Other things?
- 39. 39 Supply chain is a pain
- 42. 42 I have only one question! http://www.demotivation.us/i-have-only-one-question-1267735.html Why it still enabled by default in 2019? What do you need a helmet for? How the complex password will help?!!
- 43. 43 Any bugs there? We don’t know yet
- 44. 44 GPGPU is a new CPU § GPU drivers vulns • 10x for Windows, few for Linux • CVE-2018-6249 • CVE-2018-6253 § GPU rootkit • Avoid detection • DMA (keylogger, passwords) • Project Maux Mk.II (2008) • Jellyfish PoC rootkit (2015) § GPU – specific vulnerabilities???? Rendered Insecure GPU Side Channel Attacks are Practical
- 46. 46 Docker Host security Hardening Docker daemon (CVE-2018-15664, CVE-2018-8115, etc) Container Images Patch management Configuration (CVE-2019-5021) Information leakage Trust Root access Running containers as Root Processes as Root CAP_SYS_ADMIN privilege Limit Compute Resources The issue was first discovered back in August 2015, patched in November, then accidentally re-opened three weeks later, in December 2015, only to be re- discovered again by a Cisco Umbrella researcher in January this year. https://vulnerablecontainers.org/
- 48. 48 ML/DL Frameworks §Vulnerabilities in frameworks • Management interfaces • Data processing • Integration • Patch management §Code security • Custom code • Model as malware https://towardsdatascience.com/deep-learning-framework-power-scores-2018-23607ddf297a
- 49. 49 Data processing § 3rd party packages dependencies § Obsolete code § Data handling vulnerabilities § Example • Remote code execution in Caffe via crafted image Kang Li & Qihoo 360 Team Seri0s Exposing Vulnerabilities in Deep Learning Frameworks
- 50. 50 From framework to Pipeline NVIDIA CLARA Platform
- 52. 52 Do DICOM Series Dream of /etc/passwd? http://www.scada.sl/2019/10/dicom-to-passwd-on-security-of-ml.html
- 53. 53 Tensorflow graphs as malware § The TensorFlow server is meant for internal communication only. It is not built for use in an untrusted network. § By default, ModelServer also has no built-in mechanism for authentication. § TensorFlow may read and write files, send and receive data over the network, and even spawn additional processes. https://data-flair.training/blogs/tensorflow-security/ https://github.com/tensorflow/tensorflow/blob/master/SECURITY.md
- 54. 54 Is it real? We don’t know yet
- 55. 55 Epoch 2 Notes on HUGE data
- 56. The Satellite Flies High… §1 PT of images daily §Different formats/sources/types §Different models §Different regions §Overfitting rulez! Multispectral sources NOAA 18/19 MetOp-A/B Terra Aqua Suomi NPP NOAA 20 (JPSS-1) FengYun-3A/B/C
- 57. Data questions §Data collection and privacy §Data integrity §Training cycle •Model integrity? §IP protection 57
- 58. Model Extraction Attacks 58 Tramèr, F. (2016). Stealing Machine Learning Models via Prediction APIs.
- 59. …binwalk + grep + strings 59 Nikhil Joshi, Rewanth Cool. GDALR: An efficient model duplication attack on black-box Machine Learning models https://static.ptsecurity.com/phdays/presentations/phdays-9-gdalr-an-efficient-model-duplication-attack-on-black-box-machine-learning-models.pdf
- 60. How the AI works?
- 63. Memorization in Neural Networks In experiments, we show that unintended memorization is a persistent, hard-to-avoid issue that can have serious consequences. Specifically, for models trained without consideration of memorization, we describe new, efficient procedures that can extract unique, secret sequences, such as credit card numbers 63 Carlini, Nicholas et al. “The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks.”
- 64. Data in the model and model as a data 64 The Lottery Ticket Hypothesis at Scale Jonathan Frankle, Gintare Karolina Dziugaite, Daniel M. Roy, Michael Carbin
- 65. Adversarial example: Being John Malkovich 65Ivan Evtimov, et al, "Robust Physical-World Attacks on Machine Learning Models”
- 66. CIFAR-10 classifier on Gaussian noise 66 (Goodfellow 2016) https://www.youtube.com/watch?v=CIfsB_EYsVI&t=1756s Justin Johnson, Adversarial Examples and Adversarial Training (“CleverHans, Clever Algorithms,” Bob Sturm) Pink box – something Уellow box – airplane one step FGSM
- 69. Adversarial Robustness??? 69 Adversarial Training Gaussian Data Augmentation Ensemble learning Ensemble of weak defenses does not lead to strong defense…
- 70. Adversarial Example Frameworks Fool your AI! But… Never trust it..
- 71. 71 Epoch 3 AI for Security
- 74. 74
- 75. 75https://www.vice.com/en_us/article/9kxp83/researchers-easily-trick-cylances-ai-based-antivirus-into-thinking-malware-is-goodware Martijn Grootenhttps://skylightcyber.com/2019/07/18/cylance-i-kill-you/ Skylight Cyber – “AI” antivirus bypass with copy Not a real chicken
- 76. 76 DARPA Cyber Grand Challenge 2016 …create automatic defensive systems capable of reasoning about flaws, formulating patches and deploying them on a network in real time… Network Capture Fuzzer SymEx1 Fuzzer Crash
- 77. 77 DARPA Cyber Grand Challenge 2016 …create automatic defensive systems capable of reasoning about flaws, formulating patches and deploying them on a network in real time… Network Capture Fuzzer SymEx1 Fuzzer Crash
- 78. 78 Epoch 5 As IS
- 79. 79 You should scan all these Internets for AI
- 81. AIFinger Project The goals of the project is to provide tools and results of passive and active fingerprinting of Machine Learning Frameworks and Applications using a common Threat Intelligence approach and to answer the following questions: ● How to detect ML backend systems on the Internet and Enterprise network? ● Are ML apps secure at Internet scale? ● What is ML apps security level in a general sense at the present time? ● How long does it take to patch vulnerabilities, apply security updates to the ML backend systems deployed on the Internet? sdnewhop.github.io/AISec/ github.com/sdnewhop/AISec Contributors: ● Sergey Gordeychik ● Anton Nikolaev ● Denis Kolegov ● Maria Nedyak
- 82. AIFinger Project Coverage ● Frameworks ○ TensorFlow ○ NVIDIA DIGITS ○ Caffe ○ TensorBoard ○ Tensorflow.js ○ brain.js ○ Predict.js ○ ml5.js ○ Keras.js ○ Figue.js ○ Natural.js ○ neataptic.js ○ ml.js ○ Clusterfck.js ○ Neuro.js ○ Deeplearn.js ○ Convnet.js ○ Synaptic.js ○ Apache mxnet ● Databases with ML Content ○ Elasticsearch with ML data ○ MongoDB with ML data ○ Docker API with ML data ● Databases ○ Elasticsearch ○ Kibana (Elasticsearch Visualization Plugin) ○ Gitlab ○ Samba ○ Rsync ○ Riak ○ Redis ○ Redmon (Redis Web UI) ○ Cassandra ○ Memcached ○ MongoDB ○ PostgreSQL ○ MySQL ○ Docker API ○ CouchDB ● Job and Message Queues ○ Alibaba Group Holding AI Inference ○ Apache Kafka Consumer Offset Monitor ○ Apache Kafka Manager ○ Apache Kafka Message Broker ○ RabbitMQ Message Broker ○ Celery Distributed Task Queue ○ Gearman Job Queue Monitor ● Interactive Voice Response (IVR) ○ ResponsiveVoice.JS ○ Inference Solutions ● Speech Recognition ○ Speech.js ○ dictate.js ○ p5.speech.js ○ artyom.js ○ SpeechKITT ○ annyang … and many more
- 85. 85 Databases
- 86. 86 Dockers
- 87. 87 NVIDIA DIGITS § Training logs § Datasets § Model design
- 88. 88 Tensorboard § … § Everything § + vulns The TensorFlow server is meant for internal communication only. It is not built for use in an untrusted network. Totally more than 120 results
- 90. AI incidents There is this company in China named SenseNets. They make artificial intelligence-based security software systems for face recognition, crowd analysis, and personal verification. And their business IP and millions of records of people tracking data is fully accessible to anyone. https://twitter.com/0xDUDE/status/1095702540463820800
- 91. TAY.AI
- 92. From human
- 93. On the Feasibility of Side-Channel Attacks with Brain-Computer Interfaces Visual Stimulus PIN Code BCI Internet of Brains?
- 94. 94 Epoch 5 To Be
- 95. 95 Summa Technologiae § Intellectronics • Artificial Intelligence + Neuro interfaces • Augmented intelligence § Phantomology • Virtual reality • Augmented Reality § Creation of the Worlds • research, cognition, management “Will it be possible to construct an electronic brain that will be an indistinguishable copy of a living brain one day?” “Most certainly it will, but no one is going to do it.”
- 96. 96 Social stasis “Smart” Sales? ”Smart” Culture? “Smart” Propaganda? “Smart” Live? https://medium.com/@jonathan_hui/gan-some-cool-applications-of-gans-4c9ecca35900
- 97. 97
- 98. 98 What can we do? For Researchers AI Cybersecurity is Green Field From SDN to Model Privacy, from Secure SDL to Adversarial Robustness For Enterprises Don’t trust AI if adversarial “input” is possible AI IS NOT spherical model traveling in a vacuum! For Governments Centralize data and annotation Force vendors to follow security best practices from the beginning Detect and control AI-based abuses
- 100. 100 Am I afraid?
- 101. 101
- 102. S e r g e y G o r d e y c h i k H T T P : / / S C A D A . S L @ S C A D A S L s e r g . g o r d e y @ g m a i l . c o m Security for AI or AI for Security?h t t p s : / / c y b e r w e e k . a e Ask a Question! Make the better AI