Cs437 lecture 13
- 1. Lecture 13 Datawarehouse SchemaDatawarehouse SchemaDatawarehouse SchemaDatawarehouse Schema
- 2. If one is not careful, with the increase in number of dimensions, the number of summary tables gets very large Consider the example discussed earlier with the following two dimensions on the fact table... Time: Day,Week, Month, Quarter,Year,All Days Product: Item, Sub-Category, Category,All Products ROLAP and Space Requirement
- 3. EXAMPLE: ROLAP and Space Requirement A naïve implementation will require all combinations of summary tables at each and every aggregation level. …
- 4. Maintenance. Non standard hierarchy of dimensions. Non standard conventions. Explosion of storage space requirement. ROLAP Issues
- 5. Summary tables are mostly a maintenance issue (similar to MOLAP) than a storage issue. Notice that summary tables get much smaller as dimensions get less detailed (e.g., year vs. day). Should plan for twice the size of the unsummarized data for ROLAP summaries in most environments. Assuming "to-date" summaries, every detail record that is received into warehouse must aggregate into EVERY summary table. ROLAP Issues: Maintenance
- 6. Dimensions are NOT always simple hierarchies Dimensions can be more than simple hierarchies i.e. item, subcategory, category, etc. The product dimension might also branch off by trade style that cross simple hierarchy boundaries such as: Looking at sales of air conditioners that cross manufacturer boundaries, such as COY1, COY2, COY3 etc. Looking at sales of all “green colored” items that even cross product categories (washing machine, refrigerator, split-AC, etc.). Looking at a combination of both. ROLAP Issues: Hierarchies
- 7. Conventions are NOT absolute Example:What is calendar year?What is a week? Calendar: 01 Jan. to 31 Dec or 01 Jul. to 30 Jun. or 01 Sep to 30 Aug. Week: Mon. to Sat. orThu. toWed. ROLAP Issues: Convention
- 8. Coarser granularity correspondingly decreases potential cardinality. Aggregating whatever that can be aggregated. Throwing away the detail data after aggregation. ROLAP Issues: Aggregation Pitfalls
- 9. Many ROLAP products have developed means to reduce the number of summary tables by: Building summaries on-the-fly as required by end-user applications. Enhancing performance on common queries at coarser granularities. Providing smart tools to assist DBAs in selecting the "best” aggregations to build i.e. trade-off between speed and space. How to Reduce Summary Tables
- 10. Performance vs Space Trade-Off 20 40 60 80 100 2 4 6 8 ×××× ×××× MB %Gain Aggregation answers most queries Aggregation answers few queries
- 11. Is a relational database schema for representing multidimensional data Is the simplest form of a datawarehouse schema that contains one or more dimensions and fact tables. People usually want to see some form of aggregated data Called Measures Usually numeric and additive Example: Sales $, Number of customers Just tracking measures, however, is not enough People want to see data using a “by” condition Called dimensions Example:Time, product, Geography, etc Star Schema
- 12. Steps Involved Identify business processes for analysis (eg. Sales) Identify the measures Identify the dimensions for facts List the columns for each dimension Identify the lowest level of granularity in a fact table Aspects of Star Schema Every dimension will have a primary key (usually surrogate) Dimensions do not have parent tables Hierarchies for the dimensions are stored in the same table Star Schema
- 13. Simplified 3NF ZONE REGION zip _x_SMSA 1 ZIP ZONE ZIP SMSA ZIP ADI QTR YR STORE # ADDRESS ZIP ... WEEK QTR DATE WEEK RECEIPT # STORE # DATE ... DATE WEATHER RECEIPT #ITEM # ... $ ITEM # CATEGORY ITEM # MFCTR DEPTCATEGORY zip _x_adi year quarter week date_x_store_x_weather sale_detail item_x_category item_x_mfctr category_x_dept M 1 M 1 1 1M 1 M11 M M 1 M 1 M M M 1 1 M 1 STORE #1 M M M ADI:Area of Dominance Influence SMSA: Standard Metropolitan StatisticalArea
- 14. Simplified Star Schema ITEM# CATEGORY DEPT MFCTR ... ITEM# STORE# DATERECEIPT# ... M 1 Fact Table Product Dimension Table STORE# ADDRESS ZIP ADI SMSA ZONE 1 M Geography Dimension Table REGION $ DATE WEEK QUARTER YEAR ... Calendar Dimension Table 1 M A vastly simplified model ... may even summarize out receipt # ..... STORE# DATE WEATHER Store x Date Dimensional Table 1 M 1 M ADI:Area of Dominance Influence SMSA: Standard Metropolitan StatisticalArea
- 16. A vastly simplified physical data model! Collapse dimensional hierarchies into a single table for each dimension and create a single fact table from the header and detail records: Fewer tables. Fewer joins to get results. Merely a methodology for deploying the pre-join denormalization discussed earlier A Simplified Star Schema
- 17. Target is to get the best of both worlds. HOLAP (Hybrid OLAP) allow co-existence of pre-built MOLAP cubes alongside relational OLAP or ROLAP structures. How much to pre-build? HOLAP
- 18. DOLAP Cube on the remote server Local Machine/Server Subset of the cube is transferred to the local machine