Implementar una estrategia eficiente de gobierno y seguridad del dato con la virtualización (LATAM)
0 likes110 views
The document discusses the challenges in data management, emphasizing the need for solutions like data virtualization to enhance agility, governance, and cost-effectiveness in handling data. It highlights the benefits of adopting a logical data warehouse architecture, including improved integration and security, while also addressing the pitfalls of traditional data governance tools. Case studies, particularly from organizations like Spectrum Health and BHP, illustrate the practical implementation and advantages of these solutions in real-world scenarios.
Implementar una estrategia eficiente de gobierno y seguridad del dato con la virtualización (LATAM)
- 1. 1
- 2. Speakers Alberto Pan Chief Technology Officer Juan González Consultor Senior Líder Técnico Marcelo Méndez Gerente General
- 3. Enabling Agile Analytics and Data Governance with Data Virtualization Free your Data Alberto Pan CTO March 2021
- 4. Agenda 1. Current Challenges in Data Management 2. Data Virtualization and the Logical Data Warehouse 3. Data Virtualization: What Analysts Say 4. Case Studies 5. Q&A
- 5. Current Challenges in Data Management 1. Faster & more complex demands for decision making ▪ Provide useful information for decision making at all organization levels ▪ New users with advanced analytical skills and needs: e.g. data scientists ▪ Solution? Self Service Initiatives lead by business users, etc. → Either too complex (direct access) or too costly (specific data marts) , Governance and consistency problems 2. Regulations, enterprise-wide governance & data security ▪ Tens of new regulations worldwide: tax, finance, privacy, HR, environmental, etc. ▪ Ensure consistency in semantics of delivered data and data quality ▪ Enforce security policies ▪ Solution? Data Governance tools. Separate, static system for documentation→ get out of sync easily, don’t enforce policies & don’t deliver data to users 3. Complexity of DM infrastructure: IT cost reduction ▪ Huge data growth, operation costs → IT is looking for cheaper and more flexible solutions ▪ Solution? Cloud, Data Lakes → Increase integration complexity in the short term. E.g. Gartner says “83% of Data Lakes projects have failed”
- 6. 6 What is the Problem ? Lack of Agility: • No unified infrastructure (multiple data sources and analysis / visualization tools) • Integrating, transforming and combining data is slow with traditional methods Agility vs Governance: • Inconsistent reports / Single Source of Truth • Compliance with company glossaries and policies • How to enforce consistent security, data quality and governance policies across multiple systems • Too much replicated data
- 7. 7 Do Data Governance Tools Solve the Problem ? DG Tools allow: • Informing about data assets and their level of quality • Defining unified glossaries and terminology • Defining data quality and data governance policies, and managing/tracking changes Disconnected from the data delivery process • Do not ensure delivered data conforms to glossaries • Do not enforce security, data quality and governance policies in the data delivery process • The problem of how to enforce these policies across multiple data sources and consumption tools remain
- 8. 8 Gartner – The Evolution of Analytical Environments This is a Second Major Cycle of Analytical Consolidation Operational Application Operational Application Operational Application IoT Data Other NewData Operational Application Operational Application Cube Operational Application Cube ? Operational Application Operational Application Operational Application IoT Data Other NewData 1980s Pre EDW 1990s EDW 2010s 2000s Post EDW Time LDW Operational Application Operational Application Operational Application Data Warehouse Data Warehouse Data Lake ? Logical Data Warehouse Data Warehouse Data Lake Marts ODS Staging/Ingest Unified analysis › Consolidated data › "Collect the data" › Single server, multiple nodes › More analysis than any one server can provide ©2018 Gartner, Inc. Unified analysis › Logically consolidated view of all data › "Connect and collect" › Multiple servers, of multiple nodes › More analysis than any one system can provide ID: 342254 Fragmented/ nonexistent analysis › Multiple sources › Multiple structured sources Fragmented analysis › "Collect the data" (Into › different repositories) › New data types, › processing, requirements › Uncoordinated views “Adopt the Logical Data Warehouse Architecture to Meet Your Modern Analytical Needs”. Henry Cook, Gartner April 2018
- 9. 9 Denodo proprietary and confidential. DO NOT DISTRIBUTE Gartner: Unified Data Integration, Delivery and Governance Denodo
- 10. 10 Denodo’s Logical Data Fabric Links: Business Interface to Data 1. Single Access Point to all Data at any location 2. Semantic Layer – Expose Data in Business-Friendly form, adapted to the needs of each consumer 3. Up to 80% reduction in integration costs, in terms of resources and technology data 4. Consume data with any tool and access technology (SQL, REST, GraphQL, OData,…) 5. Single entry point to apply security and governance policies 6. Abstraction: change vendor / location / processing engine without affecting data consumers
- 11. 11 Data Virtualization: Logical Data Delivery for the Business Development Lifecycle Monitoring & Audit Governance Security Development Tools / SDK Scheduler Cache Optimiser JDBC/ODBC/ADO.Net REST / GraphQL / OData U LoB View Mart View J Application Layer Business Layer Unified View Unified View Unified View Unified View A J J Derived View Derived View J J S Transformation & Cleansing Data Source Layer Base View Base View Base View Base View Base View Base View Base View Abstraction
- 12. 12 Data Virtualization for Data Governance Single Entry Point for Enforcing Security and Governance Policies Data on-premises and off, combined through the same governed virtual layer Single Source of Truth / Canonical Views Who is Doing / Accessing What, When and How Fewer copies of personal data. Lineage of copies is available.
- 13. Query Execution and Performance Source Abstraction Virtual Modelling Business Delivery Query Optimizer Security & Governance Query Engine Delegate processing to data sources ▪ Transparently switch workloads according to cost or performance Most advanced execution engine for distributed scenarios ▪ Unique techniques automatically rewrite user queries to maximize pushdown ▪ Leverage MPP capabilities to deal with large data volumes Advancing Caching / Acceleration Mechanisms ▪ Selectively materialize subsets of the data for protecting data sources and query acceleration ▪ Machine Learning for automatically proposing selective data materializations for query acceleration
- 14. 14 Source: Gartner 2018 Data Virtualization Market Guide In 2020, organizations utilizing data virtualization will spend 45% less on building and managing data integration processes.” Through 2022, 60% of enterprises will implement some form of data virtualization as one enterprise production option for data integration. Source: Gartner 2018 Data Virtualization Market Guide
- 15. 15
- 16. 16 Gartner Gives DV its Highest Maturity Rating “Data Virtualization can be deployed with low risk and effort to achieve maximum value.”
- 17. 17 Source: Gartner Magic Quadrant for Data Integration, August 2018 Denodo continues to expand its leadership and mind share in data virtualization, reaching almost 95% of Gartner client inquiries on the subject.” Denodo grew at an impressive rate in 2018 and 2019... its leadership in the Data Virtualization market is enabling its growth Source: Gartner Market Share Analysis: Data Integration Worldwide, 2018 (published August 2019) and 2019 (published April 2020)
- 18. 18 Customer Satisfaction Why Customers Choose Denodo ▪ Gartner Peer Insights Customer’s Choice Award (January 2021) ▪ Both in 2019 and 2020, the only vendor where 100% of reviewers would recommend Denodo ▪ 125+ verified reviews with overall score of 4.7 out of 5
- 19. 19 Spectrum Health (Michigan) Regional Healthcare System (Hospitals, Physicians and Plans) • 170 service sites, including hospitals, urgent care centers, primary care physician offices, community clinics, rehabilitation, outpatient facilities and elderly care. • Revenue $6.9 billion with 39,000 employees and volunteers • Health plan with 1 million members Primary Challenges • Integrating multiple analytical data sources quickly • Reconciling provider data from multiple sources accurately (business impact)
- 20. 20 Spectrum Health 1st Project – COVID-19 Dashboard COMPONENTS: Tableau, Denodo, Oracle and SQL Server, 10+ other sources TEAM: 1 Tableau developer, 2 Denodo developers, 1 Denodo admin DEVELOPMENT TIME: • 2 days - Prototype • 2 weeks – Production*server available CHALLENGES: • Very short timeframe • No formal Data Fabric training • Understanding performance optimization (queries from hours to less than a minute) “Overall, I felt the team did an amazing job and the platform did help us deliver value much quicker than we would have been able to going the traditional ETL route. It would have take us at least 6 weeks.” - Senior Information Architect
- 21. 21 Data Platform and Regulatory Compliance
- 22. 22 Speeding Up M&A Integration
- 23. 23 Speeding Up M&A Integration
- 24. About BHP We are a leading global resources company ▪ Our purpose is to bring people and resources together to build a better world. ▪ Our strategy is to have the best capabilities, best commodities and best assets, to create long-term value and high returns. ▪ At BHP, we have a unique perspective on the extraordinary potential of natural resources to provide the essential building blocks of progress. ▪ We are among the world’s top producers of major commodities, including iron ore, metallurgical coal and copper. We also have substantial interests in oil and gas. ▪ We have a global presence with operations and offices across Australia, Asia, the UK, Canada, the USA and Central and South America. 24 Data Virtualization Platform - September 2020
- 25. Multi-Location Hybrid Data Fabric 25 Problems: • Repeated engineering effort • Long lead-times • Project-centric repositories: duplicate data everywhere Brisbane Perth Santiago Houston Cloud Tenancy Data Lake Data Mart Data Mart Analytics Analytics Analytics Data Virtualization Platform - September 2020
- 26. Reference architecture 26 Data Source ✓ Application data stores ✓ SaaS / Cloud Applications ✓ Application interfaces ✓ Manual data sources Data Fabric Consumers ✓ Enterprise & Regional Data Stores Self Service Data Catalogue Query Optimisation Query Development Data Federation Data Discovery Abstraction / Semantic Layer Security Layer Kerberos Delegation + Encryption in Transit + Extensive Auditing Secure Faster Connect to data stores or direct to source Get access to the right data, fast. Self service Flexible protocols ✓ Analytics ✓ Self Service ✓ Business Intelligence ✓ Transactional Applications ✓ Bring your own tool BHP Data Fabric - September 2020
- 27. Multi-Location Hybrid Data Fabric 27 Problems: • Repeated engineering effort • Long lead-times • Project-centric repositories: duplicate data everywhere Brisbane Perth Santiago Houston Cloud Tenancy Data Lake Data Mart Data Mart Analytics Analytics Analytics Data Virtualization Platform - September 2020
- 28. Enabling Agile Analytics and Data Governance with Data Virtualization Demostración de producto Juan González Líder Técnico March 2021
- 29. Revisión del modelo conceptual 29 Data Virtualization Platform - September 2020
- 30. Revisión del modelo conceptual 30 Data Virtualization Platform - September 2020
- 31. 31 Data Virtualization Platform - September 2020
- 32. 32 Data Virtualization Platform - September 2020
- 33. 33 Data Virtualization Platform - September 2020
- 34. 34 Data Virtualization Platform - September 2020
- 35. 35 Data Virtualization Platform - September 2020
- 36. 36 Data Virtualization Platform - September 2020
- 37. Revisión del modelo conceptual 37 Data Virtualization Platform - September 2020
- 38. 38 Data Virtualization Platform - September 2020
- 39. 39 Data Virtualization Platform - September 2020
- 40. 40 Data Virtualization Platform - September 2020
- 41. 41 Data Virtualization Platform - September 2020
- 42. 42 Data Virtualization Platform - September 2020
- 43. 43 Data Virtualization Platform - September 2020
- 44. 44 Data Virtualization Platform - September 2020
- 45. 45 Data Virtualization Platform - September 2020
- 46. 46 Data Virtualization Platform - September 2020
- 47. 47 Data Virtualization Platform - September 2020
- 48. 48 Data Virtualization Platform - September 2020
- 49. 49 Data Virtualization Platform - September 2020
- 50. 50 Data Virtualization Platform - September 2020
- 51. 51 Data Virtualization Platform - September 2020
- 52. 52 Data Virtualization Platform - September 2020
- 53. Revisión del modelo conceptual 53 Data Virtualization Platform - September 2020
- 54. 54 Data Virtualization Platform - September 2020
- 55. 55 Data Virtualization Platform - September 2020
- 56. 56 Data Virtualization Platform - September 2020
- 57. 57 Data Virtualization Platform - September 2020
- 58. 58 Data Virtualization Platform - September 2020
- 59. 59 Data Virtualization Platform - September 2020
- 60. 60 Data Virtualization Platform - September 2020
- 89. Revisión del modelo conceptual 89 Data Virtualization Platform - September 2020
- 102. Q&A
- 103. Q&A ¡Gracias! www.denodo.com [email protected] (+34) 912 77 58 55 www.auctus.cl [email protected] (+56 2) 32 13 99 53 (+56 2) 22 45 72 84