Self Service Analytics and a Modern Data Architecture with Data Virtualization (US)
1 like260 views
The document discusses G2O's expertise in data strategy, modern data platforms, and master data management, emphasizing the importance of high-quality, integrated customer data for analytics and personalization. It highlights the value of data virtualization for creating a self-service data architecture that enables real-time access to disparate data sources without the need for data replication. Finally, it addresses the challenges organizations face in achieving a data-driven approach and positions data virtualization as a critical solution for modern data management needs.
Self Service Analytics and a Modern Data Architecture with Data Virtualization (US)
- 1. Self Service Analytics November 2020
- 2. © 2019 g2o LLC; proprietary and confidential Uttam Channegowda Practice Director Big Data and Data Engineering g2o Speakers Click icon to add picture Paul Moxon SVP Data Architectures Denodo Technologies
- 3. about g2o 3 400+ analysts, designers, developers, engineers, researchers, and strategists Largest digital experience and technology shop based in Ohio technology, data, experience design We stay close to our clients for speed, efficiency, and deep collaboration 25+ YEARS EXPERIENCE © 2020 g2o, LLC; proprietary and confidential
- 4. strategically organizing and architecting your data 4© 2020 g2o, LLC; proprietary and confidential Data s trategy Accelerated by best-practice templates, automation, and a deep partner network, our experts help you prioritize data efforts to capture opportunities and support your business goals. Modern data platforms We assess, reimagine, and re-platform your data environments, so you can economically and sustainably leverage advanced capabilities to turn data from a cost center to a growth engine. Mas ter data management We help you ensure the integrity of your essential data assets – especially your customer and product data, to support guided selling, dynamic pricing, and personalization. S ingle view of the cus tomer We help you achieve a singular and complete representation of your customer data, and analyze customer behavior, so you can better target and personalize customer interactions.
- 5. 1 2 3 4 5 modern data platform analytics and actionable insights seamless customer experience automation and personalization tools optimization with on-going measures and KPIs 5 components of a data-driven organization 5
- 6. 4 common gaps • customer data lacks quality that is needed for analysis or personalization • customer data is not organized or accessible to support analytics or drive experiences • customer data is siloed across multiple systems and the customer view is incomplete • customer data is not integrated into other systems that can personalize the customer experience where organizations struggle 6
- 7. self-service users are always waiting on their data © 2019 g2o LLC; proprietary and confidential data warehouses MDM platforms data lakes Plagued by gaps in data governance limited to a small subset of core data, not easily accessible to business analysts the up-front effort of developing a schema pushed to the data consumption team
- 8. © 2019 g2o LLC; proprietary and confidential 80% Data Preparation
- 9. drivers of self-service analytics data democratization an integral part of being a data-driven organization disruptive events COVID-19, Japan earthquake, Asian tsunami data-driven innovation according to Forrester, between 60% and 73% of all data within an enterprise goes unused for analytics © 2019 g2o LLC; proprietary and confidential
- 10. 1. Not all data needs to be integrated 2. Data quality is in the eye of the beholder 3. Combining datasets does not always need to be an IT project thinking differently about data © 2019 g2o LLC; proprietary and confidential
- 11. © 2019 g2o LLC; proprietary and confidential Sometimes, being directionally correct is good enough
- 12. Find the right balance where IT’s charter to govern and secure data can peacefully coexist with the business’ need for speed to market The reality is that shadow IT will continue to exist and truly does serve a purpose for specific analytics use cases controlled chaos © 2019 g2o LLC; proprietary and confidential
- 13. 1. Data abstraction 2. Zero replication, zero relocation 3. Real-time information 4. Self-service data services 5. Centralized metadata, security & governance 6. Location-agnostic architecture for multi-cloud, hybrid acceleration data virtualization as a self- service architecture © 2019 g2o LLC; proprietary and confidential
- 14. © 2019 g2o LLC; proprietary and confidential data virtualization is not just for self-service, it’s also a first-class citizen when it comes to modern data platform architectures
- 15. 15 Gartner – The Evolution of Data Architectures This is a Second Major Cycle of Analytical Consolidation Operational Application Operational Application Operational Application IoT Data Other NewData Operational Application Operational Application Cube Operational Application Cube ? Operational Application Operational Application Operational Application IoT Data Other NewData 1980s Pre EDW 1990s EDW 2010s2000s Post EDW Time LDW Operational Application Operational Application Operational Application Data Warehouse Data Warehouse Data Lake ? LDW Data Warehouse Data Lake Marts ODS Staging/Ingest Unified analysis › Consolidated data › "Collect the data" › Single server, multiple nodes › More analysis than any one server can provide ©2018 Gartner, Inc. Unified analysis › Logically consolidated view of all data › "Connect and collect" › Multiple servers, of multiple nodes › More analysis than any one system can provide ID: 342254 Fragmented/ nonexistent analysis › Multiple sources › Multiple structured sources Fragmented analysis › "Collect the data" (Into › different repositories) › New data types, › processing, requirements › Uncoordinated views
- 16. 16 Gartner – Logical Data Architecture “Adopt the Logical Data Warehouse Architecture to Meet Your Modern Analytical Needs”. Henry Cook, Gartner April 2018 DATA VIRTUALIZATION
- 17. 17 Data Virtualization – A Data Fabric Layer Consume in business applications Combine related data into views Connect to disparate data sources 2 3 1 DATA CONSUMERS DISPARATE DATA SOURCES Enterprise Applications, Reporting, BI, Portals, ESB, Mobile, Web, Users Databases & Warehouses, Cloud/Saas Applications, Big Data, NoSQL, Web, XML, Excel, PDF, Word... Analytical Operational Less StructuredMore Structured CONNECT COMBINE PUBLISH Multiple Protocols, Formats Query, Search, Browse Request/Reply, Event Driven Secure Delivery SQL, MDX Web Services Big Data APIs Web Automation and Indexing CONNECT COMBINE CONSUME Share, Deliver, Publish, Govern, Collaborate Discover, Transform, Prepare, Improve Quality, Integrate Normalized views of disparate data “Data virtualization integrates disparate data sources in real time or near-real time to meet demands for analytics and transactional data.” – Create a Road Map For A Real-time, Agile, Self- Service Data Platform, Forrester Research, Dec 16, 2015
- 18. 18 How Does It Work? Development Lifecycle Mgmt Monitoring & Audit Governance Security Development Tools and SDK Scheduled Tasks Data Caching Query Optimizer JDBC/ODBC/ADO.Net SOAP / REST WS U Customer 360 View Virtual Data Mart View J Application Layer Business Layer Unified View Unified View Unified View Unified View A J J Derived View Derived View J JS Transformation & Cleansing Data Source Layer Base View Base View Base View Base View Base View Base View Base View Abstraction
- 19. 19 Data Virtualization Connects the Users to the Data That They Need 1. Data Virtualization allows you to connect to (almost) any data source 2. You can combine and transform that data into the format needed by the consumer 3. The data can be exposed to the consumers in a format and interface that is usable by them • Typically consumers use the tools that they already use – they don’t have to learn new tools and skills to access the data 4. All of this can be done without copying or moving the data • The data stays in the original sources (databases, applications, files, etc.) and is retrieved, in real-time, on demand Cliffs Notes version (TL;DR)
- 20. 20 Data Source Connectivity Relational Databases • MS SQL*Server (JDBC, ODBC): 2000, 2005, 2008, 2008R2, 2012, 2014, 2016, 2017 • Oracle (JDBC): 8i, 9i, 10g, 11g, 12c, 18c, 19c • Oracle E-Business Suite (JDBC): 12 • IBM DB2 (JDBC): 8, 9, 10, 11, 12 for LUW; 9,10 for z/OS, AS400 • Informix (JDBC): 7, 12 • Sybase Adaptive Server Enterprise (JDBC): 12, 15 • MySQL (JDBC): 4, 5 • PostgreSQL (JDBC): 8, 9, 10, 11 • Denodo Platform (JDBC): 5.5, 6.0, 7.0, 8.0 - For multi-location architecture deployments • MS Access (ODBC) • Apache Derby (JDBC): 10 • Generic (JDBC) In-Memory Databases • SAP HANA (JDBC): 1 • Oracle TimesTen (JDBC): 11g • Oracle 12c In-Memory • Redis In-memory Cache Parallel databases and appliances • GreenPlum (JDBC): 4.2 • HP Vertica (JDBC): 7, 8 • Oracle Exadata (JDBC): X5-2 • ParAccel 8.0.2 (using ParAccel 2.5.0.0 JDBC3g/SSL driver) • Netezza (JDBC): 4.6, 5.0, 6.0, 7.0 • SybaseIQ (JDBC) 12.x, 15.x • Teradata (JDBC): 12, 13, 14, 15 • Yellowbrick Multi-Dimensional Sources • SAP BW (BAPI/XMLA): 3.x • SAP BI 7.x (BAPI): 7.x • Mondrian (XMLA): 3.x • IBM Cognos TM1 • MS SQL Server Analysis Services 200x • Essbase (XMLA): 9, 11 Cloud Databases and Data Warehouses • Amazon Redshift (JDBC) • Amazon Athena (JDBC) • Amazon Aurora (JDBC) • Amazon DynamoDB • Amazon RDS (JDBC) • Azure Cosmos DB • Azure SQL Database • Azure Synapse Analytics (fka SQL Data Warehouse) • Databricks Delta Lake • Google Cloud SQL • Google BigQuery (JDBC) • MongoDB Atlas • Snowflake (JDBC) Data Lake Storage • Amazon S3 • Azure Data Lake Storage • Azure Data Lake Storage Gen 2 • Azure Blob Storage • Google Cloud Storage • Parquet (Distributed File System Connector) • Avro Big Data • Apache Hive (JDBC): 0.12, 1.1.0, 1.1.0 for Cloudera 1.2.1 and for Hortonworks 2.0.0 • MapR-XD, MapR-DB, MapR-ES, Hive, and Drill for MapR 6.1 • Amazon Elastic Map-Reduce (EMR) • Apache HBase (using DenodoConnect connector) • Impala (JDBC): 2.3 • Google BigTable • Spark SQL (JDBC): 1.5, 1.6 • Presto (JDBC) • Databricks 2.x NoSQL • MongoDB • Cassandra Web Services • SOAP • REST (XML, RSS, ATOM, JSON) • OData v2 and v4 Packaged Applications • SAP ERP/ECC (BAPIs and RFC tables) • Oracle E-Business Suite 12 • Siebel • SAS (SAS JDBC Driver): 7 and higher Flat and Binary Files • CSV, pipe-delimited, Regular expression-parsed • MS Excel xls 97-2003 • MS Excel xlsx 2007 or later • MS Access • XML • JSON • SAS Files (SAS7BDAT) All files can be locally accessible or in remote filesystems, through FTP/ SFTP/FTPS, and in clear, zipped and/or encrypted format. Active Directory as source or leveraging security • LDAP v3 • Microsoft Active Directory 2003, 2008 Cloud, SaaS, Web Sources with Simplified OAuth Security • Amazon • Google • Google Sheets • Facebook • LinkedIn • MS SharePoint (by using the OData connector) • MS Dynamics 365 Business Central/Customer Engagement • Marketo • ServiceNow • Salesforce (SOQL) • NetSuite • Twitter via APIs with simplified OAuth integration (1.0, 1.0a and 2.0) • Workday Indexes and unstructured content • CMS, file systems, pdf, word, text, email servers, knowledge bases, indexes • Elastic Search 6.4, 6.7 Streaming/Messaging Systems • MQSeries • SonicMQ • ActiveMQ • TIBCO EMS • Kafka Messaging • Spark Streams • IBM Streams Semantic Repositories • Semantic repositories in Triple Stores/RDF accessed through SPARQL endpoints. • Neo4j Graph Database Denodo SDK for Custom Connectors • CouchDB • Lotus Domino Web Automation • Denodo’s ITPilot automates extraction from web pages Mainframe • IMS • IBM IMS native drivers: 8, 9 • IMS Universal Drivers: 11 Hierarchical databases • Adabas (SOA Gateway and Denodo’s SOAP connector): 5, 6 Legacy • Microsoft FoxPro (ODBC) The following data sources have been successfully tested with Denodo using JDBC and ODBC drivers, WS/SOAP and WS/REST, and DenodoConnect adapters (not exhaustive list): • Apache Solr • IBM BigInsights • Pivotal HAWQ
- 21. 21 Protocols and Formats • SQL Based access via JDBC, ODBC and ADO.NET • Web Services • SOAP (XML/JSON) • REST (JSON/XML) • OData 2 & 4 • GraphQL • Open API (a.k.a Swagger) • Web Parts (for SharePoint), Portlets • Kafka and JMS listeners for message queues • Denodo Scheduler for batch process and ‘ETL lite’ Security Options • Authentication using LDAP or Active Directory • Kerberos for Single Sign-On (SSO) • OAuth, OAuth 2.0 (JWT) • SAML • SSL/TLS • WS-Security, X.509 certificates • Two-Factor Authentication – via identity providers Okta, Duo, etc. BI/Reporting tools • Microstrategy, Cognos, Business Objects, Oracle OBIEE • Tableau, Qlikview, Spotfire, Microsoft PowerBI • Excel Analytical Tools/Languages • SAS, Statistica, SPSS, MatLab • R, Python, Java, Scala, etc. • Azure ML Studio, Apache Zeppelin and Jupyter analytics notebooks Portals • SharePoint, Enterprise portals, Web/mobile apps Enterprise Service Bus • Oracle Service Bus, Azure Service Bus, TIBCO Active Matrix Bus ETL tools • SAP Data Services, Informatica Powercenter, IBM Data Stage, Talend ETL API Management tools • CA (Layer 7), TIBCO Mashery, Apigee Publishing Options
- 22. 22 Decoupling Business and IT IT: Flexible Source Architecture Business: Flexible Tool Choice IT can now move at slower speed without affecting the business Business can now make faster and more sophisticated decisions as all data accessible by any tool of choice
- 23. 23 Multi-cloud future is a reality: • Risk mitigation • Mix and match of best of breed tools and technologies • Multi-cloud architectures include a mix of on-premise databases as well • Organizations won’t be moving to the cloud overnight and need a layer that eases the transition Data Virtualization Enables Cloud Modernization
- 24. 24 • Data Virtualization has reached the ‘Plateau of Productivity’ • Alternatives are still not mature enough for mainstream • Data Lakes still rely on ETL and security remains a challenge • ‘No code’ data tools for self-service (e.g. data Prep tools) have governance and security issues also. Data Virtualization is Mainstream…
- 25. 25 Gartner and Forrester Research Evaluations Why Denodo? Forrester Wave: Enterprise Data Virtualization, Q4 2017Forrester Wave: Enterprise Data Fabric, Q2 20202020 Gartner Magic Quadrant for Data Integration Tools
- 26. 26 Publication Date – 25th August 2020 Gartner Critical Capabilities for Data Integration Tools Denodo is the only product with 5.0 score in Data Virtualization category
- 27. Q&A
- 28. What’s your Next? Request a Discovery Session Learn how to put Data Virtualization to work in your organization! pages.denodo.com/g2orequest.html REGISTER NOW
- 29. Thank you for joining us! © 2020 g2o LLC; proprietary and confidential