Master Data Management's Place in the Data Governance Landscape
2 likes•1,410 views
This document provides an overview of master data management and how it relates to data governance. It defines key concepts like master data, reference data, and different master data management architectural models. It discusses how master data management aligns with and supports data governance objectives. Specifically, it notes that MDM should not be implemented without formal data quality and governance programs already in place. It also explains how various data governance functions like ownership, policies and standards apply to master data.
![An enterprise’s current state data architecture and data movement/integration patterns inform MDM, and provide requirements for
determining the MDM architecture patterns needed to support the implementation of MDM
– Examples: MDM publish/subscribe, MDM message-based integration, MDM transaction interception, MDM BI/Analytics, MDM data warehouse
patterns
MDM Bi/Analytics Pattern and Insights
– In a BI/Analytics pattern, there is an opportunity to determine whether insights gained in BI and analytics systems have relevance to the MDM system
– Can lead to a two-way integration, where results of insights can be used to update attributes in the MDM hub [e.g., customer attributes]
MDM Data Warehouse Patterns and the EDW
– This pattern describes the integration between MDM solution and data warehouses
– The DWs (typically) are treated as downstream systems that do not provide master data back to the MDM platform
– We often find that MDM data can source conformed slowly changing types of dimensions within the warehouses
– Impacts to existing integration / ETL can often be made much simpler, especially in the case of conformed dimensions with multiple sources
• DWs often serve as a merge/consolidation mechanism in the absence of MDM
– No feedback from DWs to the MDM hub
– Extracts/Rest APIs from the MDM hub require data to be in a consistent and well-formed state to support bulk extraction methods typically used in
DW ETL
Architecturally, Where does MDM Fit In?](https://image.slidesharecdn.com/datagovernanceandmdm-atlanta-190228000530/85/Master-Data-Management-s-Place-in-the-Data-Governance-Landscape-31-320.jpg)
Master Data Management's Place in the Data Governance Landscape
- 1. Master Data Management’s Place in the Data Governance Landscape SQL Server Master Data Services in Action February 21, 2019
- 2. Introductions CCG Data Governance Master Data Management Defined Why Master Data Management? MDM and Data Governance Master Data and Reference Data Master Data Architectures/Models Demo
- 3. Presenters David Langlois Data Governance Lead Tony Williams Data Management Strategist Cherian Mathew BI Architect
- 4. CCG Analytics Solutions & Services DATA MANAGEMENT Data & analytics consultants with a passion for helping clients overcome business challenges & increase performance by leveraging modern analytic solutions. BUSINESS ANALYTICS DATA STRATEGY
- 5. CCG Analytics We bring great People together to do extraordinary Things DATA ANALYTICS STRATEGY Working with CCG is like working with extended team members. Consultants become an integral part of the work bringing expertise for cutting edge design and development. - CIO, HCPS
- 6. CCG Clients
- 7. Key Drivers and Benefits of Data Governance Information Fit for Purpose
- 8. 1 2 3 Key Drivers for Data Governance: Inactive There are some aspects of DG employed within the organization, but there are no enterprise standards in place(e.g. the IS team has developed a data dictionary). Reactive The enterprise is responding to a specific issue or problem (e.g. data breach or audit). The enterprise is facing a major change or there is a potential regulatory threat to the organization (e.g. GDPR, acquisitions, or preparing for a pubic offering) Proactive The enterprise recognizes the value of data and has decided to treat data as a corporate asset (e.g. recruitment of a CDO, budgeted DG program, etc.).
- 9. 1 2 3 Benefits of DG Increase Revenue Improve opportunity to rapidly exploit information for business insights and competitive advantage Improve profitability with better analytics for improved decision making Reduce Cost through Operational Efficiencies Smoother business processes Standardized and high quality information Reduced IT costs Minimize Risk Reduce regulatory compliance risk and improve confidence in operational and management decisions Provide better insights into fraud with improved analytics; Improve reporting to regulators and authorities through defined data processes and data management
- 10. Respondents to CIO Watercooler’s Data Governance Survey 2017 - How long has your organization been implementing data governance?
- 11. “Hiding within those mounds of data is knowledge that could change the life of a patient, or change the world.” – Atul Butte, Stanford “Information is the oil of the 21st century, and analytics is the combustion engine.” – Peter Sondergaard “$3.1 trillion, IBM’s estimate of the yearly cost of poor quality data, in the US alone, in 2016” – Harvard Business Review Data Informed, “Without accurate data on customers, an organization can’t achieve revenue goals. Poor data quality… could lead to missed sales opportunities.” Forrester reports that “Nearly one third of analysts spend more than 40 percent of their time vetting and validating their analytics data before it can be used for strategic decision-making” Ponemon Institute and GlobalScape report the annual cost of non-compliance to businesses now runs an average of $14.8 million, a 45% increase since 2011 GDPR: There will be two levels of fines based on the GDPR. The first is up to €10 million or 2% of the company’s global annual turnover of the previous financial year, whichever is higher. The second is up to €20 million or 4% of the company’s global annual turnover of the previous financial year, whichever is higher. According to the Forbes Insights and KPMG “2016 Global CEO Outlook,” 84% of CEOs are concerned about the quality of the data they’re basing their decisions on.
- 12. Master Data Management Alignment with Data Governance
- 13. What Is Master Data? Definitions Master Data… Data that provides the context for business activity data in the form of common and abstract concepts that relate to the activity. It includes the details (definitions and identifiers) of internal and external objects involved in business transactions, such as customers, products, employees, vendors, and controlled domains (code values). Master Data Management… Processes that control management of master data values to enable consistent, shared, contextual use across systems, of the most accurate, timely, and relevant version of truth about essential business entities. (DAMA- DMBOK Guide, 1st edition, pg. 171.) Master Data Management Architecture Pattern… Architecture patterns, when applied to MDM, address the scope and architectural aspects of MDM. Consideration of the various Enterprise Master Data business and technical strategies, master data implementation approaches, and MDM methods of use are also included in the MDM architecture pattern. (IBM, Information service patterns, part 4)
- 14. Connecting Master Data Management And Data Governance Master Data Management… Should not be undertaken without a formal and functioning Data Quality Program – it is difficult to develop a DQ program at the same time as an MDM program A Data Quality Program will require an information ownership model to be established (stewardship) – implementing MDM will use this model And we should all already know this: Data Governance and Data Quality have a symbiotic, mutually interdependent relationship. You shouldn't want one without the other. Therefore, to ensure take up, adoption, and sustainability of an MDM program, we need Data Governance! Like Data Governance and Data Quality, MDM is a business program, not a technology solution implementation! Therefore, it must be sponsored by the business. If implementing MDM is under the CIO, it is not receiving the enterprise attention is must have to become a sustaining program Master Data Management Program Sponsorship…
- 15. Master Data Management Some Key Implementation Factors MDM needs to align to data governance and data management strategy and objectives – which are in turn aligned to business strategy and objectives. The business case used to justify a Data Governance (or Data Quality) program can be reused and adapted for a MDM program. When determining data and information (management) functions as part of Data Governance Functional Design, we map them to business objectives. An MDM program needs to align to the same business objectives and to the designed data functions specified in Data Governance. Requires an information/data ownership model, and that data stewardship be in place or implemented simultaneously Like Data Governance, an MDM program should have an appropriate scope: Geographic, lines of business, enterprise- wide. In fact it will often align to DG’s boundaries and federation. Policies and standards will need to shift, be added, changed, etc. when implementing MDM. Data Governance provides the policy and standards framework and the processes and procedures to accomplish this Processes and procedures around our data will change because we are implementing MDM. Data Governance to the rescue again – processes and procedures should already be defined for all data management functions Some Key Factors in Implementing MDM… Other Factors?
- 16. Data Governance and Data Management MDM is a Core Data Management Function Data Governance Data Quality Program Master Data Management Data and Information Life Cycles Data Management ENSURING DATA IS WELL-GOVERNED ENSURING THE RIGHT DATA IS WELL-MANAGED Making sure data is managed properly Managing data to achieve goals Data Governance Council • Planning Functions • Design Functions • Managing Functions • Operate Functions • Planning Functions • Design Functions • Managing Functions • Operate Functions • Planning Functions • Design Functions • Managing Functions • Operate Functions DataOwnershipandStewardship
- 17. Improve data quality – although a MDM program only improves data quality to an extent – we need quality to be governed over the long term and the Data Quality program to frame and guide our efforts as we clean (and keep clean) data for use in MDM Streamline data architecture Reduce data integration complexity – regardless of the architecture approach chosen for MDM there is a huge opportunity to investigate current state data integration, movement, and interoperability Fully enable ownership and stewardship – MDM can further maturity around data ownership and stewardship. Data steward processes and procedures to maintain reference data in its various locations should become much easier as this data BECOMES master data. Tools integrate with metadata management needs – from a tool and technology perspective, MDM provides another source and target of enterprise metadata. Natural overlap in capturing business rules to enable process changes Facilitate process and culture change – especially towards a culture of data Why are we talking about MDM? Some compelling outcomes we may be seeking… Other Potential Outcomes?
- 18. Master Data Management Master Data Management’s Mission Apply a combination of data governance practices and technologies intended to confirm integrity and accuracy of data to help organizations frame the “single version of the truth” needed to answer business questions and drive insights. What business questions and insights? KPIs related to revenue growth, for example: – Revenue per customer – Inventory levels – Customer profitability – New customer sales due to new products/services launched
- 19. Master Data vs. Reference Data What is Reference Data? Reference Data Definition Is data that is used to structure and constrain other data. Typically, it is very stable with a known set of values that are rarely changed It classifies and categorizes other data, and it can be hierarchical Examples of Reference Data Geographical Locations – localities, states, regions, countries, zip/postal codes, census blocks Financial Markets – valid stock tickers, for example Industry Classifications – NAIC, SIC, etc. Science – known elements in the periodic table Calendars – lists of days of the week, months, ISO weeks, reporting weeks, accounting months Product Hierarchy – a company’s product listing used as reference data (for purchase and customer orders, inventory, etc.)** Reference data is any data we agree can be used as reference data across our enterprise and systems
- 20. Master Data vs. Reference Data The Differences, the Similarities Reference Data Master Data Provides structure to other data ✓ ✓ Constrains other data ✓ ✓ Provides context to other data ✓ ✓ Analytical “Slicers and Dicers” ✓ ✓ Limited to pre-defined values ✓ Can be defined/controlled by business rules ✓ Represent business entities (the “nouns” of a business) ✓ Data quality formality and requirements Data governance control and authority Data ownership and stewardship Managed via standard technology/tools/procedures (MDM!)
- 21. External Reference Data Management Internal and External Reference Data Internal Internal reference data applies to business concepts that are specific to the organization. Managing internal reference data requires a federated approach, since it is created and managed by many different business data stewards. Requires alignment to the data ownership model External reference data is maintained by authorities outside the enterprise (e.g., ISO, government agencies, etc.). External reference data must be discovered, selected, understood, and captured appropriately. May align to the ownership model Common Mapping reference data often requires human judgment, so the need for intervention by business data stewards in the reference data management process should not be overlooked. Requires standard processes and procedures to maintain Apply MDM governance, processes, procedures, technology
- 22. Master Data Management Architectural Models Application Application DB Master Data Hub or Repository Registry Master Data Repository Master Data Hybrid Master Data
- 23. Registry (or Federated) MDM Architecture Application #1 Application DB contains: Business key + some attributes Application #2 Application DB contains: Business key + some attributes Master Data Hub System of Record: MDM Hub System of Entry: Application Timing: Batch Hub stores • Business keys • Limited attributes + - Minimal application changes Queries hit against all systems containing elements involved Minimal impact to end users of applications Master data is focused on referential use Quicker to deploy Lower implementation cost The most common approach pulls limited master data from application systems and stores in a hub.
- 24. Data Governance with Registry (or Federated) MDM Architecture Application #1 Application DB contains: Business key + some attributes Application #2 Application DB contains: Business key + some attributes Master Data Hub System of Record: MDM Hub System of Entry: Application Timing: Batch MDM Hub stores • Business keys • Limited attributes • Stewards at the application level to manage data quality • Potential for differentiated stewardship roles (application vs. data vs. business) • Clear communication on batch timing • Reporting may source from application or Hub database depending on timing requirements
- 25. Repository MDM Architecture Application #1 Application DB contains: Only business key Application #2 Application DB contains: Only business key Master Data Repository System of Record: Repository System of Entry: Repository Timing: Realtime Repository stores • All business keys • All attributes + - No versioning across applications May be hard to modify OTS applications. Less duplication of data Difficult to model completeness and usability Cost All master data is in the repository, with other systems modified to access repository
- 26. Repository MDM Architecture Application #1 Application DB contains: Only business key Application #2 Application DB contains: Only business key Master Data Repository System of Record: MDM Repository System of Entry: MDM Repository Timing: Realtime MDM Repository stores • All business keys • All attributes • Technical Stewards at the integration points, responsible for ensuring that input and output match • Implementing data quality rules • Monitoring quality and performance • Issue management processes to support integration issues • Repository is primary reporting data source
- 27. Coexistence MDM Architecture Application #1 Application DB contains: Master data records Application #2 Application DB contains: Master data records Master Data Hub System of Record: MDM Hub System of Entry: Application Timing: Realtime, potential latency Hub • Enhances data • Publishes data back to apps + - Consolidated master data Master data is focused on referential use Documents lineage Cost of integration Easier to monitor processes A coexistence architecture will ingest data from source systems, then publishes it back to other source systems.
- 28. Coexistence MDM Architecture Application #1 Application DB contains: Master data records Application #2 Application DB contains: Master data records Master Data Hub System of Record: MDM Hub System of Entry: Application Timing: Realtime, potential latency Hub • Enhances data • Publishes data back to apps • Federated stewardship • Quality controls in place at MDM hub • Reporting source is MDM Hub primary, other attributes from applications as needed • Technical stewardship of integrations required
- 29. Transaction MDM Architecture Application #1 Application DB contains: Calls to MDM Hub Application #2 Application DB contains: Calls to MDM Hub Master Data Hub System of Record: MDM Hub System of Entry: MDM Hub Timing: Realtime, potential latency Hub publishes data to apps + - Single point of entry Cost of integration Centralized control Simplified lineage Easier to monitor processes In a transactional style, the MDM hub becomes the sole entry system, then publishes this data to applications.
- 30. Transaction MDM Architecture Application #1 Application DB contains: Calls to MDM Hub Application #2 Application DB contains: Calls to MDM Hub Master Data Hub System of Record: MDM Hub System of Entry: MDM Hub Timing: Realtime, potential latency Hub publishes data to apps • A centralized team of stewards can support master data • MDM main reporting source • Strong support of quality • Technical stewardship of integrations required
- 31. An enterprise’s current state data architecture and data movement/integration patterns inform MDM, and provide requirements for determining the MDM architecture patterns needed to support the implementation of MDM – Examples: MDM publish/subscribe, MDM message-based integration, MDM transaction interception, MDM BI/Analytics, MDM data warehouse patterns MDM Bi/Analytics Pattern and Insights – In a BI/Analytics pattern, there is an opportunity to determine whether insights gained in BI and analytics systems have relevance to the MDM system – Can lead to a two-way integration, where results of insights can be used to update attributes in the MDM hub [e.g., customer attributes] MDM Data Warehouse Patterns and the EDW – This pattern describes the integration between MDM solution and data warehouses – The DWs (typically) are treated as downstream systems that do not provide master data back to the MDM platform – We often find that MDM data can source conformed slowly changing types of dimensions within the warehouses – Impacts to existing integration / ETL can often be made much simpler, especially in the case of conformed dimensions with multiple sources • DWs often serve as a merge/consolidation mechanism in the absence of MDM – No feedback from DWs to the MDM hub – Extracts/Rest APIs from the MDM hub require data to be in a consistent and well-formed state to support bulk extraction methods typically used in DW ETL Architecturally, Where does MDM Fit In?
- 32. DATA MGMT SOURCES STOREINGEST PREP & TRAIN MODEL & SERVE Modern Cloud Analytics Data Architecture: Governed Delivery with Quality Big Data Storage Core Deposits, Member Information Analytical Services Enterprise Data WarehouseInternet/Mobile Banking and Bill Pay Rel Mgmt, Contact Tracking, Referral Tracking Imaged Statements Structured Data Stage and Archive Loaders Unstructured Data Unstructured Data Loaders ODS / Archive Transform Transform Check Images Debit Card Processing Analytic Enablement Streaming Analytics Machine Learning Cognitive Analytics Data Exploration Business Intelligence Platform Dimensional Data Storage G/L, Fixed Assets, Accts Payable, Acct Receivable Universal Semantic Layer Data Quality Metadata Management Data Architecture Data Security & Privacy Data Lineage DQ Corrective Actions DATA GOVERANCE PROGRAM MANAGEMENT MDM Hub MDM Solution Patterns
- 33. RapidDG 5-6 week engagement Business case development Charter development Competency level analysis 9-12 month roadmap Data Governance Service Offerings CCGDG (includes RapidDG) 9-12 month engagement Alignment to organizations strategic plan Competency/marker level analysis Marker level strategic roadmap DG Consultative Services Variable
- 35. Demo Microsoft Master Data Services
- 36. SQL Server MDS 2017 Overview https://docs.microsoft.com/en-us/sql/master-data- services/master-data-services-overview-mds?view=sql-server- 2017 Server requirements: Pre-Installation and Post Installation Tasks – IIS, SQL Server 2017 • Other years supported, but 2017 cleanest https://docs.microsoft.com/en-us/sql/master-data- services/install-windows/install-master-data-services?view=sql- server-2017 Licensing / cost: – Enterprise (license fees) and Developer Edition (free) Client operating requirements (IE, Silverlight Plugin) MDS: Setup
- 37. MDS: Functional Overview High level overview of the MDS functions – MDS Service – User access to master data • Data Explorer in the Web UI • Data Explorer Add-In in Excel – MDS Database – SSIS connections via: • the subscriptions views • Direct table connections
- 38. Demo MDS
- 39. Start with Data Governance Ensure Data Quality Make sure you’ve defined data ownership Master your first Master Data Entity – Define key attributes – Identify business & data quality rules – Begin modeling out an architecture that fits your organization • Determine the needed architecture patterns – Choose appropriate systems based on scale and environment • Excel • Master Data Services • Larger enterprise systems Next Steps: Evolution of an MDM Program
- 40. Networking, Q&A, and Discussion
- 41. Thank You!