SAP HANA Architecture Overview | SAP HANA Tutorial
1 like640 views
The document discusses challenges in data integration across heterogeneous information sources and the complexities of data management in large enterprises. It evaluates two main approaches to data integration: query-driven methods and data warehousing, highlighting the benefits and drawbacks of each. The concept of a data warehouse is defined, emphasizing its role as a centralized repository for analytical purposes, supporting decision-making with consistent and historical data.
SAP HANA Architecture Overview | SAP HANA Tutorial
- 2. 2 Problem: Heterogeneous Information Sources “Heterogeneities are everywhere” Different interfaces Different data representations Duplicate and inconsistent information Personal Databases Digital Libraries Scientific Databases World Wide Web
- 3. 3 Problem: Data Management in Large Enterprises • Vertical fragmentation of informational systems (vertical stove pipes) • Result of application (user)-driven development of operational systems Sales Administration Finance Manufacturing ... Sales Planning Stock Mngmt ... Suppliers ... Debt Mngmt Num. Control ... Inventory
- 4. 4 Goal: Unified Access to Data Integration System Collects and combines information Provides integrated view, uniform user interface Supports sharing World Wide Web Digital Libraries Scientific Databases Personal Databases
- 5. 5 Two Approaches: Query-Driven (Lazy) Warehouse (Eager) Source Source ? Why a Warehouse?
- 6. 6 The Traditional Research Approach Source SourceSource . . . Integration System . . . Metadata Clients Wrapper WrapperWrapper Query-driven (lazy, on-demand)
- 7. 7 Disadvantages of Query-Driven Approach Delay in query processing Slow or unavailable information sources Complex filtering and integration Inefficient and potentially expensive for frequent queries Competes with local processing at sources
- 8. 8 The Warehousing Approach Data Warehouse Clients Source SourceSource . . . Extractor/ Monitor Integration System . . . Metadata Extractor/ Monitor Extractor/ Monitor Information integrated in advance Stored in wh for direct querying and analysis
- 9. CS 336 9 Advantages of Warehousing Approach • High query performance – But not necessarily most current information • Doesn’t interfere with local processing at sources – Complex queries at warehouse – OLTP at information sources • Information copied at warehouse – Can modify, annotate, summarize, restructure, etc. – Can store historical information – Security, no auditing
- 10. 10 Not Either-Or Decision • Query-driven approach still better for – Rapidly changing information – Rapidly changing information sources – Truly vast amounts of data from large numbers of sources – Clients with unpredictable needs
- 11. 11 What is a Data Warehouse? A Practitioners Viewpoint “A data warehouse is simply a single, complete, and consistent store of data obtained from a variety of sources and made available to end users in a way they can understand and use it in a business context.” -- Barry Devlin, IBM Consultant
- 12. 12 What is a Data Warehouse? An Alternative Viewpoint “A DW is a – subject-oriented, – integrated, – time-varying, – non-volatile collection of data that is used primarily in organizational decision making.” -- W.H. Inmon, Building the Data Warehouse, 1992
- 13. 13 A Data Warehouse is... • Stored collection of diverse data – A solution to data integration problem – Single repository of information • Subject-oriented – Organized by subject, not by application – Used for analysis, data mining, etc. • Optimized differently from transaction- oriented db • User interface aimed at executive
- 14. 14 … Cont’d • Large volume of data (Gb, Tb) • Non-volatile – Historical – Time attributes are important • Updates infrequent • May be append-only • Examples – All transactions ever at Sainsbury’s – Complete client histories at insurance firm – LSE financial information and portfolios
- 15. 15 Generic Warehouse Architecture Extractor/ Monitor Extractor/ Monitor Extractor/ Monitor Integrator Warehouse Client Client Design Phase Maintenance Loading ... Metadata Optimization Query & Analysis
- 16. 16
- 17. 17
- 18. 18 Data Warehouse Architectures: Conceptual View • Single-layer – Every data element is stored once only – Virtual warehouse • Two-layer – Real-time + derived data – Most commonly used approach in industry today “Real-time data” Operational systems Informational systems Derived Data Real-time data Operational systems Informational systems
- 19. 19 Three-layer Architecture: Conceptual View • Transformation of real-time data to derived data really requires two steps Derived Data Real-time data Operational systems Informational systems Reconciled Data Physical Implementation of the Data Warehouse View level “Particular informational needs”
- 20. 20 Data Warehousing: Two Distinct Issues (1) How to get information into warehouse “Data warehousing” (2) What to do with data once it’s in warehouse “Warehouse DBMS” • Both rich research areas • Industry has focused on (2)
- 21. 21 Issues in Data Warehousing • Warehouse Design • Extraction – Wrappers, monitors (change detectors) • Integration – Cleansing & merging • Warehousing specification & Maintenance • Optimizations • Miscellaneous (e.g., evolution)
- 22. 22 OLTP: On Line Transaction Processing Describes processing at operational sites OLAP: On Line Analytical Processing Describes processing at warehouse OLTP vs. OLAP
- 23. 23 Warehouse is a Specialized DB Standard DB (OLTP) • Mostly updates • Many small transactions • Mb - Gb of data • Current snapshot • Index/hash on p.k. • Raw data • Thousands of users (e.g., clerical users) Warehouse (OLAP) Mostly reads Queries are long and complex Gb - Tb of data History Lots of scans Summarized, reconciled data Hundreds of users (e.g., decision-makers, analysts)
- 24. 24
- 25. 25
- 26. 26
- 27. 27
- 28. 28
- 29. 29
- 30. 30
- 31. 31
- 32. 32
- 33. 33
- 34. 34
- 35. 35
- 36. 36
- 37. 37
- 38. 38
- 39. 39
- 40. 40
- 41. 41
- 42. 42
- 43. 43
- 44. 44
- 45. 45
- 46. 46
- 47. 47
- 48. 48
- 49. 49
- 50. 50
- 51. 51
- 52. 52
- 53. 53
- 54. 54
- 55. 55
- 56. 56
- 57. 57
- 58. 58