Oracle Big data at work

Editor's Notes

• #2: Big data is the electricity of the 21st century – a new kind of power that changes everything it touches. But big data is still a bit of a mystery. Get four CIOs in a room and you’ll have 5 opinions about what big data is, what it does, and how to make it useful. The early days of electricity were the same. In fact, at one time, it was a mystery whether electricity created in the lab was the same thing as lightning in the sky.
• #3: On a hot and humid night in 1752, Benjamin Franklin ran a now-famous experiment of flying a kite in a thunderstorm. Franklin added a wire to the kite so it stuck out the top and attached a metal key to the bottom of the kite’s wet string. When he saw fibers of the string stiffen, he moved his hand close to the key and sparks jumped from the key to his knuckles. He then captured some of this electricity in a Leyden jar (an early kind of battery), brought it home and proved the stuff he caught was the same as the electricity he had generated in his workshop. More than just curiosity drove Franklin. He was trying to solve a problem – how to prevent lightning strikes from burning up buildings. Now that he had proven lightning was the same as the electricity in his experiments, he knew how to conduct it away. The first lightning rods were installed on tall buildings in Philadelphia that summer.
• #4: Over the years, the work of many other inventors, like Nikola Tesla and Thomas Edison, turned Franklin’s discovery into one of the most useful substances humanity has ever known. The electrification of everything opened the floodgates of innovation in business, government, and private life
• #5: Today, we are on the cusp of the same magnitude of transformation thanks to the datafication of everything. Datafication is a term coined by Ken Cukier, data editor at The Economist, and Viktor Mayer-Shönberger in their excellent book, Big Data: A Revolution That Will Transform How We Live, Work, And Think. Datafication is the capture and use of information in more daily activities, and we are seeing this happen everywhere.
• #6: The datafication of our thoughts and opinions. For example, in just one minute, Facebook users share over half a million pieces of content (Source: http://www.visualnews.com/2012/06/19/how-much-data-created-every-minute/) The datafication of things through sensors collecting information from cars, medical devices, stop lights, and factory equipment. “a single jet engine can generate 10TB of data in 30 minutes. With more than 25,000 airline flights per day, the daily volume of just this single data source runs into the Petabytes.” http://www.oracle.com/us/products/database/big-data-for-enterprise-519135.pdf Even the natural world is being datafied through the use of satellite imagery and climate sensing devices We’ve been datafying enterprise processes for decades. Now we’re datafying personal life activities, too. You no longer have to go to the bank to deposit a check; you can use your phone. Running on the treadmill at the gym? Chances are good you might be wearing a heart-rate monitor or a Nike Fuel Band to track your physical activity
• #7: All this new data offers tremendous potential to change the way our organizations do business Whether it be capturing the thoughts and opinions of our customers to create better marketing campaigns Using sensors to manage buildings, capital equipment, and improve maintenance and service costs Or streamlining processes based on new data insights, the possibilities are endless
• #8: And this is just the beginning Smart Devices are predicted to grow from 1.3B in 2013 to 12.5B in 2020 And data generated from “things” is growing at a rate of 22 times over 5 years, from 2011-2016 (Source: IDC 2011, Cisco,, Cloudera, and Machina Research http://blog.iobridge.com/2012/02/cisco-reports-mobile-internet-of-things-traffic-to-grow/) There’s lots of available data, representing huge new opportunities to create new value. So, what’s the problem?
• #9: The problem is the world’s ability to produce data has outstripped most organizations’ ability to use it. One of the largest airlines in the world, employing dozens of operational research analysts, throws away most of its fleet operational data at the end of the day because it’s so big there’s nowhere to put it and analyze it. (Source: a presentation by Jim Diamond, Managing Director of Operations & Research at American Airlines. Given at the Evanta CIO event in Dallas, TX 6/7/13) The same is true for many other businesses: the information they need to improve products and services already exists, they’re just not quite sure how to use it. According to a study we conducted with The Economist Intelligence Unit, only 12% of executives feel they understand the impact data will have on their organizations over the next three years.” (Source: http://www.oracle.com/webapps/dialogue/ns/dlgwelcome.jsp?p_ext=Y&p_dlg_id=13367869&src=7634271&Act=143 )
• #10: But don’t worry. This has happened before. It took 150 years to go from capturing lightning in a bottle to making electricity widely available and useful. This process is already happening with big data. Just faster. A lot faster. So this raises a new question: How do you make big data useful? Every company is asking themselves this question right now so they won’t get left behind. But for large organizations like yours, the question is slightly different. Your question is How do you bring big data into the enterprise to make it useful? You already have hundreds of millions, if not billions, of dollars invested in analytical technologies. How do you bring big data into the enterprise analytical environment to make it useful? To understand this, we need to look at how big data will improve what exists and create new possibilities.
• #11: Let’s start with what exists. You already have analytics you use to run the business – data warehouses, reports and dashboards. You carefully select and standardize the data you need to solve specific problems, like running a marketing campaign or billing customers.
• #12: This is a powerful way to reduce the time, cost and effort of standardizing and controlling processes to run the business. The data coursing through this enviroment will increase in volume and velocity – it will get bigger and arrive faster than today. This is the relational analytical environment we’re all familiar with. You’ll need more processing power to handle this, but that’s pretty straightforward.
• #13: But that’s not all. The datafication of everything opens a new possibility – the possibility to learn from the data in new ways. There’s a huge amount of available data, but it’s not always clear which data might be useful to you or what you might learn from it Examining the data in a non-relational environment and letting it tell YOU what you can learn from it, cuts the time, effort and cost of forming and testing hypotheses.
• #14: We call this the “Change the Business” approach because new ideas uncovered this way often lead companies to make changes, or pivot processes and systems in ways they wouldn’t have attempted otherwise.
• #15: The critical difference between the run-the-business and change-the-business environments boils down to one thing: To run the business, you organize data to make it do something specific; to change the business, you take data as-is to figure out what it can do for you. Relational technologies excel at the first, non-relational technologies at the second.
• #16: These two approaches are more powerful together than either alone. Combining the two gives you continuous innovation. This is Big Data At Work. This, also, has happened before. In the early days of commercial electricity, there was a huge standards war – alternating currrent vs direct current, AC vs DC. AC could be transmitted over long distances, but its high voltage made the wires deadly if you touched them. DC wires were safe to touch because of their low voltage, but could only deliver power a mile or two from the source. It was a bitter fight. But in the end, the solution was to make them work together. AC now comes out of every wall socket. DC comes out of every battery at the heart of every mobile device. The two together are far more powerful than either alone. But to bring a non-relational environment into the corporate fold, it has to have the same basic capabilities as the relational environment the company already counts on It has to acquire, manage,and analyze, whatever data happens to be in it, just like the traditional relational environment
• #17: To bring big data into the enterprise analytical environment you’re going to need all the standard enterprise capabilities for data acquisition, management, and analysis for relational, non-relational, and streaming environments (which can be both). For acquisition, this means you’ll need relational databases as well as NoSQL databases, plus super-small-footprint Java embedded in devices for real-time capture. For management, you’ll still need your relational data warehouses, and they’ll be complemented by Hadoop clusters plus real-time caching and event processing. For analysis, you’ll still use BI reports and dashboards, and they’ll be complemented by non-relational discovery plus real-time recommendations, alerts, and predictive analytics
• #18: But the real magic of big data at work is having these relational, non-relational, and streaming environments seamlessly deployed together. Only Oracle creates products for every aspect of this unified architecture. And these products can be deployed on-premise, in private or public clouds, depending on your needs. What if you could do that? What if you could have acquisition, management and analysis for any kind of data for any purpose you could imagine? What would it mean to you?
• #19: If you had Big Data At Work you could Get fast answers to new questions Predict more, and more accurately Create a reservoir of data for potential reuse Accelerate data-driven action Let’s look at a couple companies realizing the benefits of big data at work
• #20: Let’s begin with getting fast answers to new questions.
• #21: Problem Delphi Electronics and Safety , a division of leading global auto parts supplier Delphi Automotive, had a data analysis challenge: they needed to determine if the performance of certain parts were meeting contractual levels and if improvements were needed Seems straightforward, right? Not so much. Delphi receives huge amounts of warranty data generated by its customer. Every month, the automotive manufacturers (OEMs) delivers performance data including verbatim text descriptions of issues related to its 340,000 active parts in service in millions of vehicles worldwide That’s data from over a dozen different OEM systems, each with its own distinct format, as well as data from Delphi’s own parts databases, manufacturing systems, and industry data. Additionally, Delphi had to adhere to strict time guidelines to provide responses to performance issues with parts—including a complete analysis to support their response—or be financially penalized. The real challenge was the diversity of the issues. Delphi’s warranty engineers needed to quickly combine and explore a variety of customer data sets based on the issue under investigation. Warranty Engineers were spending more time manipulating data than getting answers from it. CLICK Solution In the first month alone, engineers discovered the root cause of three field performance issues that could have cost them lots of money. Since then, Endeca has paid for itself many times over But more importantly, their warranty analysts could now spend more time investigating issues and less time manipulating data. The shift in this work was so great that they had a new idea. They realized they could have a warranty strategy for each of 20,000 individual parts they manufacture and ship at a rate of 7 million pieces per month– an unprecedented innovation. This is the power of Big Data At Work – creating discoveries in a change-the-business environment which are then injected into run-the-business processes and applications to perform at a higher level.
• #22: Other customers have created different combinations of diverse data for discovery. Some draw data out of Hadoop clusters and combine it with data warehouse extracts. Others combine social media feeds with customer transaction data. In every case, they’re bringing together relational and non-relational data to reduce the time, cost, and effort of getting fast answers to new questions. Another way we unify run-the-business and change-the-business analytics is by fully integrating Endeca with Oracle Business Intelligence. You can automatically ingest OBI data into Endeca as well as publish analytics on non-relational data from Endeca in BI dashboards. This is what it means to have big data at work.
• #23: Predict more, more accurately.
• #24: Dell is a great example of a company using predictive analytics to improve their customer experience with targeted cross-sell and upsell offers. Dell brings together data from its website, social media channels, as well as offline customer data into a big data farm. This big variety of data then drives predictive analytics for promotions at the website, in the call center, in email campaigns, and even on-demand print materials. Since deploying the system, Dell has realized $132M in incremental revenue for FY12 They have also seen a 10% increase in revenue, and a 20% increase in profit margin per call at their call centers This also is the power of Big Data At Work – predictive analytics driven by machine-learning algorithms chewing on masses of diverse and changing data. This new non-relational technology is now integral to Dell’s cross-channel customer experience. Here, change-the-business analytics have become the way Dell runs its business.
• #25: They are currently using Oracle’s Real-Time Decisions in 15 countries and 30 languages in their call centers for technical service and sales, email correspondence, and social media to offer personalized, targeted product and service recommendations across multiple channels. Each of these channels is able to perform self-learning decisions to optimize the next best action. Real-Time Decisions is the non-relational predictive analytics technology making all these predictions. It learns from the responses and adjusts its algorithms to continuously improve. It also adjusts its algorithms to use new data sources Dell believes will be valuable.
• #26: Information is unlike almost anything else in that it is not used up when you use it. You can reuse it and even repurpose it infinitely. The reason this matters is that all of you are sitting on data assets with huge potential value. According to McKinsey, the vast majority of American companies store more data than the US Library of Congress. But most of it is locked away in separate buckets. What if you could pour it all together into a great big reservoir, ready to be tapped at any moment? That would be great, but how can you do this in a cost effective way when you don’t know what value the reservoir will produce?
• #27: We worked with a large, full-service bank faced with this exact problem. The bank had to comply with regulations requiring more data to support stress testing. But there was a problem. The bank could only pull 10-15% of the necessary data from their source systems, which took 16 different nightly extracts, resulting in multiple data marts. Plus, managers also suspected there would be new requirements to the stress tests which would start this whole process over again. So they needed to evolve their information architecture to support their run the business relational warehouse with a change the business data reservoir. The bank is reaping the benefits of lower costs thanks to the reduction in the number of data marts, duplicate data stores, and fewer extracts. They can now also work with all their data; not just the 10-15% they could access before Their big data solution added the missing 85%. And because they are not Hadoop experts, they appreciated the speed, time to value, and overall TCO of an appliance
• #28: The bank is reaping the benefits of lower costs thanks to the reduction in the number of data marts, duplicate data stores, and fewer extracts. They can now also work with all their data; not just the 10-15% they could access before Their big data solution added the missing 85%. And because they are not Hadoop experts, they appreciated the speed, time to value, and overall TCO of an appliance
• #29: They used a combination of Oracle’s Big Data Appliance with Cloudera’s distribution of Hadoop and Exadata running Oracle Database, and they seamlessly integrated those two together using Oracle’s Big Data Connectors and extreme network performance provided by Infiniband. For those of you not familiar with Hadoop, it’s a non-relational method of storing data and processing it. This bank filled the Big Data Appliance with data from legacy mainframes, operational databases, enterprise applications, and more. They created a great, big reservoir of diverse data that’s ready to be tapped and siphoned into the enterprise warehouse at a moment’s notice. Not only is this bank prepared for future changes to the stress tests, it’s now also prepared to do customer, product, and process analyses that would have been cost-prohibitive before.
• #30: One of the most critical day-to-day necessities of big data is to accelerate data-driven action In a constantly changing business environment, the value of real-time analytics can reduce fraud, and help your workforce create fast and accurate reporting based on the most up to date information
• #31: Another example is a major European bank that needed a big data at work solution This bank was looking to monetize the relationship they have with their existing customers by delivering a location-based service They do this with customers who have joined their shopping club looking for potential deals When a customer uses an ATM, the bank knows where they are. They can use this information to deliver a relevant, targeted advertisement or message. But in order to make the message relevant and targeted they have to know the customer well. So they build a model that incorporates internal (to the bank and customers) and external social media, purchase records and location information (where has the customer purchased items, interacted with the bank, used ATMs)
• #32: From this model of both relational and non-relational data, they can build a model of potential interests. Once the customer uses an ATM they take the location, what the model tells them, and a list of partner merchants with offers and discounts. They automatically send the offer that seems to be the best fit to the customer’s mobile phone. They monitor the success and failure of the offers to improve their knowledge of the customer and increase future success rates
• #33: An integrated big data solution will enable you to… Get fast answers to new questions Predict more, and more accurately Create a reservoir of data for potential reuse Accelerate data-driven action
• #34: Tightest integration of big data into business analytics Connector between BI Foundation and Hive Connector between R and RTD Integration between BI Foundation and EID Big data discovery in weeks not months Standard EID deployment times Predictive analytics for all big data environments In database predictive analytics (OAA) Machine learning predictive analytics (RTD) Best price/performance big data platform Lowest TCO enterprise Hadoop (BDA) Best price/performance big data warehousing (Exadata) Fastest connection between big data analytical environments Connectors between Hadoop and the data warehouse Infiniband connections between BDA, Exadata, Exalogic, and Exalytics
• #35: In the end, we must return to where we began. To electricity. In the early days of commercial electricity, the 1893 World’s Fair was lit up with about 200,000 electric lights. No one had ever seen anything like it. This was electricity at work. But outside the fair, the world was still mostly dark. Electric cars and power grids were still in the future. For all the bright spots of big data we’ve seen so far, the world is still mostly dark. It waits for your ideas, your new uses for big data. And we look forward to creating that new world with you – a world of Big Data at Work.