How to Swiftly Operationalize the Data Lake for Advanced Analytics Using a Logical Data Fabric (ASEAN)
1 like169 views
The document discusses how to operationalize a data lake for advanced analytics using a logical data fabric approach, emphasizing the challenges faced by enterprises in unifying disparate data sources. It highlights the benefits of employing a data fabric architecture, which includes faster data access, improved analytics capabilities, and streamlined data management. A case study of Logitech is presented, demonstrating how the Denodo platform facilitated a successful data migration and enhanced analytics through virtualization.
How to Swiftly Operationalize the Data Lake for Advanced Analytics Using a Logical Data Fabric (ASEAN)
- 1. How to Swiftly Operationalize the Data Lake for Advanced Analytics Using a Logical Data Fabric Chris Day Director, APAC Sales Engineering [email protected]
- 2. 4 - VentureBeat AI, July 2019 87% of data science projects never make it into production.
- 3. No Unified View of Enterprise Data Current State of Enterprise Architecture Business wants all enterprise data, integrated, and up-to-date IT responds by loosely stitching together disparate data sources Business Wants All of the Data, Now – So IT creates 100s to 1000s of brittle direct connections and replicates large volumes of data Inventory System (MS SQL Server) Product Catalog (Web Service -SOAP) BI / Reporting JDBC, ODBC, ADO .NET Web / Mobile WS – REST JSON, XML, HTML, RSS Log files (.txt/.log files) CRM (MySQL) Billing System (Web Service - Rest) ETL Portals JSR168 / 286, Ms Web Parts SOA, Middleware, Enterprise Apps WS – SOAP Java API Customer Voice (Internet, Unstruc)
- 4. 6 6
- 5. 7 A data fabric is an architecture pattern that informs and automates the design, integration and deployment of data objects regardless of deployment platforms and architectural approaches. It utilizes continuous analytics and AI/ML over all metadata assets to provide actionable insights and recommendations on data management and integration design and deployment patterns. This results in faster, informed and, in some cases, completely automated data access and sharing. Data Fabric Definition
- 6. 8 Pictorial View of a Data Fabric – From Gartner Data Fabric Net Compounds Customers Products Claims RDBMS/OLTP Flat Files Legacy Third Party Traditional Analytics/BI Data Warehouse ETL ETL Mart Mart Data Lakes Cloud Data Stores Apps andDocument Repositories XML • JSON • PDF DOC • WEB
- 7. 9 - Forrester Research, June 2020 Dynamically orchestrating disparate data sources intelligently and securely in a self-service manner and leveraging various data platforms to deliver integrated and trusted data to support various applications, analytics, and use cases. Data Fabric Definition
- 8. 10 Forrester: Data Fabric Architecture Data Management Metadata / Catalog Data Security Data Governance Data Processing Data Quality Data Lineage Policies Global distributed platform, in-memory, embedded, self-service, and APIs AI/ML Global data access Data modeling, preparation, curation, and graph engine AI/ML Data discovery Transformation, integration, and cleansing AI/ML Data orchestration Ingestion, streaming, and data movement AI/ML Data processing/ persistence Data ingestion / streaming AI/ML Data Platform — Processing Hadoop NoSQL Spark Data Lake EDW/BDW AI/ML On-premises Cloud Data sources AI/ML
- 9. 11 Logical Data Fabric Architecture Data Management Metadata / Catalog Data Security Data Governance Data Processing Data Quality Data Lineage Policies Global distributed platform, in-memory, embedded, self-service, and APIs AI/ML Global data access Data modeling, preparation, curation, and graph engine AI/ML Data discovery Transformation, integration, and cleansing AI/ML Data orchestration Ingestion, streaming, and data movement AI/ML Data processing/ persistence Data ingestion / streaming AI/ML Data Platform — Processing Hadoop NoSQL Spark Data Lake EDW/BDW AI/ML On-premises Cloud Data sources AI/ML
- 10. Underlying Technology of a Data Fabric
- 11. 13 Logical Data Fabric: Technical Architecture
- 12. 14 Logical Data Fabric Decouples Consumers from Data Sources IT: Flexible Source Architecture Business: Flexible Tool Choice IT can now move at slower speed without affecting the business Business can now make faster and more sophisticated decisions as all data accessible by any tool of choice
- 13. 15 A logical data layer – a “data fabric” – that provides high-performant, real-time, and secure access to integrated business views of disparate data across the enterprise. Data Virtualization: Logical Data Fabric • Data Abstraction: decoupling applications/data usage from data sources • Data Integration without replication or relocation of physical data • Easy Access to Any Data, high performant and real-time/ right- time • Data Catalog for self-service data services and easy discovery • Unified metadata, security & governance across all data assets • Data Delivery in any format with intelligent query optimization that leverages new and existing physical data platforms
- 14. Customer Story
- 15. 17 Logitech’s Data Driven Journey Logitech is a Swiss global provider of personal computer and tablet accessories with EMEA headquarters in Lausanne, Switzerland and American headquarters in Newark, California. In 2019, the company reported around $2.8 billion in revenue with its 9,000 employees. Business Need • The company’s product line was evolving and includes sophisticated devices such as smart home automation, VR gaming, video collaboration, home security, etc. • Business users needed to find answers to problems relating to price violations on retail sites and perform text mining and sentiment analysis of Logitech’s products on social media and gaming websites, and other domains. • They were also challenged by fragmented analytics caused by data being trapped across multiple on- premises systems such as ERP, POS, DRM and MDM.
- 17. 19 Data Virtualization Benefits Experienced by Logitech • The Denodo Platform made Logitech’s cloud journey not only possible but possible as a live migration, with minimal impact on business operations. • Before the Denodo Platform was implemented, the weekly demand forecast program (run on a combination of R and Oracle Exadata) ran for three days; now it takes a few hours. • Logitech leveraged the Denodo Platform to create a more sophisticated data service, one that is powered by natural language processing (NLP) and fuzzy logic to enable nontechnical users to ask stats-based questions to a chatbot in English and receive a more humanized, English answer in response.
- 18. 20 - Avinash Deshpande, Principal, Big Data and Analytics, Logitech We deployed the Denodo Platform to aid us in moving our data analytics platforms to the cloud, and Denodo data virtualization played a critical role in that journey. The Denodo platform, already excellent, is constantly evolving, and getting better every day.
- 19. Key Takeaways § Analytics in a Modern Data Architecture means more than just the Enterprise Data Warehouse – there will be other data repositories and analytical engines. § A Logical Data Fabric allows users to access any of the data in your analytics environment – seemingly from a single (virtual) repository. § A Logical Data Fabric provides a flexible and extensible modern data architecture which can adapt to new data technologies and tools without unduly impacting the users.
- 22. Thanks! www.denodo.com [email protected] © Copyright Denodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.