Predicting Customer Experience through Hadoop and Customer Behavior Graphs
23 likes•8,881 views
The document discusses the integration of Hadoop and customer behavior analytics in enhancing customer experience in the telecom industry. It highlights how telecom service providers can leverage customer data from various sources, such as social media and IoT, to improve service delivery and customer engagement. Additionally, it outlines a modern data architecture with Hadoop that supports real-time and predictive analytics for better decision-making and operational efficiency.
Predicting Customer Experience through Hadoop and Customer Behavior Graphs
- 1. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Predicting Customer Experience through Hadoop and Customer Behavior Graphs with Hortonworks and Apigee We do Hadoop.
- 2. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Your speakers… Sanjay Kumar General Manager, Telecom Hortonworks Sanjeev Srivastav VP Data Strategy Apigee
- 3. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Enhancing Customer Experience with Hadoop We Do Hadoop Sanjay Kumar General Manager, Telecom Hortonworks
- 4. © Hortonworks Inc. 2011 – 2015. All Rights Reserved The New Landscape of the Telecom Industry Service Providers Social Media & Mobile: Explosion of rich customer data through Social Media and Mobile Apps for customer sentiment & Interests Customer Expectation: With the cultural impact of web and mobile, customers are expecting greater levels of service and responsiveness Competitive Differentiation: As other service providers deliver similar levels of telecom service and coverage, other areas of service levels are needed New Digital Ecosystem: Greater value of Data on digital ecosystem for Customers and Partners driving Data Monetization Internet Of Things: Explosion of data from IOT with benefits aligned with insight not correlated to data volumes
- 5. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Service Provider Focus Service Providers Customer Experience Management - Enhance End-to-end Experience of Customer - Become Trusted Partner to Customer - Awareness of customer’s needs when and where needed New Business & Consumer Services - New Digital & Infrastructure Services - Data Monetization - M2M, IoT, Analytics-as-a service Network Optimization - Move to Software Driven Networks - Leverage Network Data Assets - Self optimizing and provisioning
- 6. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Traditional systems under pressure Challenges • Constrains data to app • Can’t manage new data • Costly to Scale Business Value Clickstream Geolocation Web Data Internet of Things Docs, emails Server logs 2012 2.8 Zettabytes 2020 40 Zettabytes LAGGARDS INDUSTRY LEADERS 1 2 New Data ERP CRM SCM New Traditional
- 7. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Tomorrow: A Data-Centric Model for Your Business DATA-CENTRIC Limitations: • Multiple copies of data • Difficult cross-system integration • Upper-limit on data volumes before harming performance Advantages: • One version of the data • No need for cross-app integration • System scales linearly APP-CENTRIC App1 App 2 App 3 App 4 App 5 App 6 App Centric will break down with x10, x100,x1000… Need to shift to Data Centric
- 8. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Modern Data Architecture emerges to unify data & processing Modern Data Architecture • Enable applications to have access to all your enterprise data through an efficient centralized platform • Supported with a centralized approach governance, security and operations • Versatile to handle any applications and datasets no matter the size or type Clickstream Web & Social Geoloca3on Sensor & Machine Server Logs Unstructured SOURCES Existing Systems ERP CRM SCM ANALYTICS Data Marts Business Analytics Visualization & Dashboards ANALYTICS Applications Business Analytics Visualization & Dashboards ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° HDFS (Hadoop Distributed File System) YARN: Data Operating System Interactive Real-TimeBatch Partner ISVBatch Batch MPP EDW
- 9. © Hortonworks Inc. 2011 – 2015. All Rights Reserved HDP delivers a completely open data platform Hortonworks Data Platform 2.3 Hortonworks Data Platform provides Hadoop for the Enterprise: a centralized architecture of core enterprise services, for any application and any data. Completely Open • HDP incorporates every element required of an enterprise data platform: data storage, data access, governance, security, operations • All components are developed in open source and then rigorously tested, certified, and delivered as an integrated open source platform that’s easy to consume and use by the enterprise and ecosystem. YARN: Data Operating System (Cluster Resource Management) 1 ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ApachePig ° ° ° ° ° ° ° ° ° ° HDFS (Hadoop Distributed File System) GOVERNANCE BATCH, INTERACTIVE & REAL-TIME DATA ACCESS Apache Falcon ApacheHive Cascading ApacheHBase ApacheAccumulo ApacheSolr ApacheSpark ApacheStorm Apache Sqoop Apache Flume Apache Kafka SECURITY Apache Ranger Apache Knox Apache Falcon OPERATIONS Apache Ambari Apache Zookeeper Apache Oozie
- 10. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Only HDP delivers a Centralized Architecture HDP is uniquely built around YARN serving as a data operating system that provides multi-tenant Resource Management, consistent Governance & Security and efficient Operations services across Hadoop applications. Hortonworks Data Platform YARN Data Operating System • A centralized architecture of consistent enterprise services for resource management, security, operations, and governance. • The versatility to support multiple applications and diverse workloads from batch to interactive to real-time, open source and commercial. Key Benefits • Multiple applications on a shared data set with consistent levels of service: a multitenant data platform. • Provides a shared platform to enable new analytic applications. • Delivers maximum cost efficiency for cluster resource management. Fewer servers fewer nodes. Storage YARN: Data Operating System Governance Security Operations Resource Management Existing Applications New Analytics Partner Applications Data Access: Batch, Interactive & Real-time
- 11. © Hortonworks Inc. 2011 – 2015. All Rights Reserved HDP: Any Data, Any Application, Anywhere Any Application • Deep integration with ecosystem partners to extend existing investments and skills • Broadest set of applications through the stable of YARN-Ready applications Any Data Deploy applications fueled by clickstream, sensor, social, mobile, geo-location, server log, and other new paradigm datasets with existing legacy datasets. Anywhere Implement HDP naturally across the complete range of deployment options Clickstream Web & Social Geoloca3on Internet of Things Server Logs Files, emails ERP CRM SCM hybrid commodity appliance cloud Over 70 Hortonworks Certified YARN Apps
- 12. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Social Media Sentiment The View from the Customer Call Center Interaction Quality of Service Lifestyle & Interests Clickstream Geolocation Web Data Internet of Things Docs, emails Server logs Streaming: Network Probes, Click Stream, Sensor, Location Batch: Call Detail Records On-Line: Customer Sentiment Unstructured: Txt, Pictures, Video, Voice2Text
- 13. © Hortonworks Inc. 2011 – 2015. All Rights Reserved DELIVERY The Destination: Data-Centric Operations Clickstream Geolocation Web Data Internet of Things Docs, emails Server logs Streaming: Network Probes, Click Stream, Sensor, Location Batch: Call Detail Records On-Line: Customer Sentiment Unstructured: Txt, Pictures, Video, Voice2Text Personal Data Analysis & Customer Insight Services To Customer & Partners Hadoop Distribution with Yarn: Allows central source of data across all mediums of ingestion and interaction Existing & Legacy Systems can Contribute and Participate: May extend the life of existing and legacy systems from enriched data New Applications interact with Data Lake, not each other: Next Generation Apps build around data and can deliver to customers and partners
- 14. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Hadoop Driver: Enabling the data lakeSCALE SCOPE Data Lake Definition • Centralized Architecture Multiple applications on a shared data set with consistent levels of service • Any App, Any Data Multiple applications accessing all data affording new insights and opportunities. • Unlocks ‘Systems of Insight’ Advanced algorithms and applications used to derive new value and optimize existing value. Drivers: 1. Cost Optimization 2. Advanced Analytic Apps Goal: • Centralized Architecture • Data-driven Business DATA LAKE Journey to the Data Lake with Hadoop Systems of Insight
- 15. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Journey to Enhanced Customer Experience Real-time Event Action based on Customer Context & Location Collection of Data for Customer Insights (Data Science) Actionable Insights through Customer Models (BI) Current State of the Customer (Dynamic Customer Profile - NPS)
- 16. © Hortonworks Inc. 2011 – 2015. All Rights Reserved HDFS Raw Event Storage Customer Experience Journey – Data Lake for Collection 1 ° ° ° ° ° ° ° HBase Processed Event Storage ° ° ° ° ° ° ° ° N ° Mul3tenant Processing: YARN (Hadoop Opera.ng System) Metadata Management HCatalog Hive / Tez (Interac.ve Query) ISV (YARN Apps, i.e. HPA / LASR) Slider (Always-‐on Services) Message Queues Log Files Web Services JMS Update Data Lake Network Probe Events ODBC / JDBC Rest API Native API External Customer Data References
- 17. © Hortonworks Inc. 2011 – 2015. All Rights Reserved HDFS Raw Event Storage 1 ° ° ° ° ° ° ° HBase Processed Event Storage ° ° ° ° ° ° ° ° N ° Mul3tenant Processing: YARN (Hadoop Opera.ng System) Metadata Management HCatalog Hive / Tez (Interac.ve Query) ISV (YARN Apps, i.e. HPA / LASR) Slider (Always-‐on Services) Message Queues Log Files Web Services JMS Update Data Lake Network Probe Events ODBC / JDBC Rest API Native API External Customer Data References Customer Experience Journey – Actionable Insights
- 18. © Hortonworks Inc. 2011 – 2015. All Rights Reserved HDFS Raw Event Storage 1 ° ° ° ° ° ° ° HBase Processed Event Storage ° ° ° ° ° ° ° ° N ° Mul3tenant Processing: YARN (Hadoop Opera.ng System) Metadata Management HCatalog Hive / Tez (Interac.ve Query) ISV (YARN Apps, i.e. HPA / LASR) Slider (Always-‐on Services) Message Queues Log Files Web Services JMS Update Data Lake Network Probe Events ODBC / JDBC Rest API Native API External Customer Data References State of the Customer - Net Promoter Score - Appetite for Info score - Target Advertising Profile - # Dropped calls today/month - Sentiment / Churn Score HBase /Accumulo Real-‐3me Serving ° ° ° ° ° ° ° ° N In-‐Line Memory (Spark) Update Customer Profile and Scores Customer Experience Journey – Current State of Customer
- 19. © Hortonworks Inc. 2011 – 2015. All Rights Reserved HDFS Raw Event Storage Customer Experience Journey – Real-time Event Interaction 1 ° ° ° ° ° ° ° HBase Processed Event Storage ° ° ° ° ° ° ° ° N ° Mul3tenant Processing: YARN (Hadoop Opera.ng System) Metadata Management HCatalog Hive / Tez (Interac.ve Query) ISV (YARN Apps, i.e. HPA / LASR) Slider (Always-‐on Services) HBase /Accumulo Real-‐3me Serving ° ° ° ° ° ° ° ° N Streaming Event Processor: Storm Machine Learning (Spark) Indexing (Lucene) Rules Processing (Drools) In-‐Line Memory (Spark) Message Queues Log Files Web Services JMS Enrich Events with Customer info And Score Matrix Update Data Lake Real-time Intelligent Action - Marketing Promotions - Next Best Action - Dynamic Network Provisioning Network Probe Events ODBC / JDBC Rest API Native API Messaging Platrom: KaLa Update Customer Profile and Scores External Customer Data References State of the Customer - Net Promoter Score - Appetite for Info score - Target Advertising Profile - # Dropped calls today/month - Sentiment / Churn Score
- 20. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Hortonworks for Customer Experience Functional Area Solution Components / Use Case Description Problem Addressed Business Benefits Customer Experience Management Central Data Lake for 360 Customer View Visibility of customer household view across services and accounts through ingestion of account service and event data into a central Data Lake with views into granular customer’s service experience - Silo view of customer in different systems - Social media unstructured data does not fit into existing EDW - Incorporating customer call center discussions into customer view - Complete view into customer experience across all services - Reduction in Customer Churn Dynamic Customer Profile Summarized instant view of customer across service identifiers and customer key performance metrics and ‘net promoter scores’. Used for immediate view of customer profile - How to react to customer contact & events based on their experience - What is the customer’s experience level and context at this moment in time - Next Best Action based on customer’s experience with service provider (Retail /Call Center) - Greater targeted marketing/ advertising Real-time Next Best Action Real-time streaming of network event data to identify customer location - How to determine next best action when and where they are most appropriate to customer - Marketing and CEM analysis is after the fact; need for real-time - Context sensitive promotions 10x customer acceptance - Improves customer experience levels and customer retention
- 21. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Customer Experience Management & Marketing Journey HDP Landing Zone HDP DataLake Real-‐.me Streaming HDP DataLake Dynamic Customer Profile 360 Customer Awareness & Household View Loca3on Based CEM & Real-‐3me customer response Next Best Ac3on Customer Sen3ment Customer Aware Loca3on Based Promo3ons Mul3-‐channel Customer Scoring Models
- 22. Telco Customer Care using Hadoop & Customer Behavior Analytics Sanjeev Srivastav / [email protected]
- 23. Agenda The changing landscape in customer care • Interactions across multiple channels • Key experience management strategies Data lakes for ongoing analysis and action Visualize and predict • Sequences of activity across call center and operations data • Next-best-actions, churn, etc. based on activity sequences 23
- 24. 24 TheChangingLandscape! inCustomerCare! 24 Traditional models of customer behavior were largely based on customer profiles. New focus on behavior models.
- 25. 25 DataLakes! 25 Investments in storing customer engagement data across various interaction channels
- 26. 26 Visualize&Predict! 26 Granular customer activity across channels
- 27. Enterprises see interaction fragments 27 Email Analytics Mobile Analytics Web Analytics Retail Analytics Social Analytics Call Center AnalyticsTime
- 28. Understand each customer’s journey 28
- 29. Identify common interactions and influences 29
- 30. Contextual and individualized actions 30 Customer journey insights Individual and contextual actions Real time ?
- 31. Use case – call center + operations data Predict the customers likely to contact the call center Predict the likely reason of the next call Avert the next call 31
- 32. Use case – call center + operations data Predict the customers likely to contact the call center Use sequence of events + customer profile Predict the likely reason of the next call Illustrate with individual customer experience Avert the next call Act during call or after call 32
- 33. Use case – digital channel abandonment Interaction sequences ending in abandonment of a digital channel Profile of customers and relevant metrics Interactions after abandonment Evaluate customer preferences Likelihood of abandonment Influence engagement with customers 33
- 34. Responding to trends: Key technology strategy Data Lake for cross-channel experience Journey analytics Predictive analytics Actionability § Batch and real-time engagement § APIs and interfaces with existing systems § Models for recommendations and targeting § Next best actions § Customer journey visualization § Time and sequence query interface § Raw data, not summarized or aggregates § Time sequences preserved Users Apps Developers Models + APIs Data + APIs Backend End–to-end System 34
- 35. Apigee Insights Demo • Customer interaction across various “channels” – Web site – IVR – Call center agents – Retail stores • Visualize – “Channel surfing” – Customer satisfaction • Act – Segmentation, based on customer experience 35
- 36. Visualize: repeated calls to a call center • Calls center data • Operations data • Network data 36 IVR change 25 Escalation 226,359 (66,532 dropoff) Network 9,340 Education 18,588 (6,119 dropoff) Internal issue 9,121 Equipment issue 9,121 Internal issue 661 (blank) 576 Network 528 No sound 27 No pic 29 Education 107 (35 dropoff) Equipment issue 765 (243 dropoff)
- 37. Visualize: Customer profiles with path analysis 37
- 38. Predict: using customer profiles and events Event sequences before t (training) Prediction at time=t Events after t (test) 38 Machine learning @scale Ok to have small number of historical events that correlate with predicted events
- 39. Apigee Insights platform and tools Machine learning (ML) Segment customers Individualized actions ML feedback § APIs for customer engagement and recording customer responses § Engine for hyper-personalization; in-context recommendations § Tool for business analysts; data interfaces to business systems § Models of behavior (Apigee GRASP + 3rd party models) § Flexible modeling time windows Users Apps Developers Models + APIs Data + APIs Backend End–to-end System 39
- 40. Next Steps… Download the Hortonworks Sandbox Learn Hadoop Build Your Analytic App Try Hadoop 2 More about Apigee & Hortonworks http://hortonworks.com/partner/apigee Contact us: [email protected]
- 41. © Hortonworks Inc. 2011 – 2015. All Rights Reserved Q&A