Preparing for the Cybersecurity Renaissance

Editor's Notes

• #7: HP ArcSight: https://www.protect724.hpe.com/docs/DOC-12978 Splunk (Hadoop Connect): http://www.splunk.com/en_us/solutions/solution-areas/big-data/splunk-hadoop-connect.html Export events from Splunk to Hadoop Browser directories and files in Hadoop Index data from Splunk in Hadoop for secure search, visualization, analytics Qradar: https://www.ibm.com/.../Integrating_QRadar_with_Hadoop.pdf
• #9: A starting point on files: PDF EXE JAVA PCAP PE Files RAR ZIP SWF MS Office RTF MHTML YARA Mach-O XOR
• #14: Every Hadoop platform gives you scalability and flexibility. Cloudera makes Hadoop fast, easy, and secure. Trap Questions: Spark: What matters to you in supporting Spark and Hadoop? Impala: How many BI users will you have? What additional budget have you allocated for Hive? Kudu: How do you plan to address operational data warehouse / time series use cases? Cloudera Navigator Optimizer: How do you know what data should be in Hadoop vs existing systems? Trap Questions: Cloudera Manager: How much downtime are you willing to accept during an upgrade? What if your operations tools fail during an outage? What does your team need to debug critical and latent issues? Cloudera Director: Where is your data being created? How do you plan to manage across environments? Are you prepared to train staff on both Amazon and on-premises Hadoop platforms? Expert Support: How can a core R&D group simultaneously respond to frequent customer issues and also build a culture of innovation? [only Cloudera has a back-line support team to address issues without bringing in R&D] Trap Questions: Navigator Encrypt/KeyTrustee: What is the impact of an information leak from intermediate MR results, log files, etc? Sentry/RecordService: How are you planning to secure access to sensitive data across Hive and Spark? Navigator: Do your governance needs extend beyond Hive? Manager: How will you keep end users from damaging your production environments?
• #15: Merge slide 12 and 10 without categorization on the vendors
• #16: Cybersecurity application framework Abnormal traffic patterns analyzing network flow and dns with machine learning Open data models for network Community to extend
• #18: Find the needle in the haystack. Note: slideshow mode builds image in the following sequence: 1.       You can see all the connections, but which are suspicious? 2.       Netflow data is sent through the advanced algorithms 3.       The algorithms identify the most likely candidates for further investigation 4.       The suspicious connections are ranked and given to a human to analyze 5.       The human determines what actions to take toward each item 6.       Feedback then makes the system grow smarter and more useful
• #20: Misconfigurations, network problems you didn’t know about. Security and misconfigurations account for an enormous part of what they do. Its all fire fighting. Things are either operational or security not both.
• #23: First and foremost we have the Apache Hadoop community. This ever growing community continues to grow as Hadoop continues to expand out of just batch storage and processing. As more businesses continue to use Hadoop, more use cases continue to emerge creating additional projects that allow for more robust data applications. With Hadoop you don’t just get the code your team built, you get the code the community built.
• #27: Link to account record in SFDC (for Cloudera employees only): https://na6.salesforce.com/0018000000oqdOF?srPos=0&srKp=001 A national security organization in the US offers real-time information, warnings and guidelines that strengthen our ability to protect against cyber attacks. Background: In today’s digital world, cyber security is a serious effort. A US national security organization is tasked with identifying potentially suspicious activity across the worldwide web, and they must make that information available to 700 commercial and federal organizations.  Challenge: More data lends itself to more accurate predictions, so logically the organization has been collecting massive volumes of cyber data in order to prevent cyber attacks. But their incumbent IBM Netezza environment was hitting scalability and performance limitations.  Solution: The organization deployed a Cloudera Enterprise platform with Sherpasurfing, an open source cyber security solution created by Six3 Systems. Cloudera runs data processing, staging and storage; the Cloudera environment is integrated with Netezza which performs analytics. The system is also integrated with HP ArcSite, Tableau and Centrifuge. Hadoop components in use include Accumulo, Hive, Flume, MapReduce and Sqoop. This organization is growing their Cloudera cluster to 100 PB.  Results: Since deploying Cloudera alongside Netezza, this organization can offer real-time information and warnings to other government organizations. They can also now show rich information on major malware outbreaks and provide guidelines to strengthen organizations’ protection against cyber attacks -- this is a net new capability that resulted from their Cloudera deployment.