SlideShare a Scribd company logo
Data Contracts Management
Schema Registry and Beyond
About me
Pietro La Torre
Strategy & Business Development
@ Quantyca SpA
Quantyca is a privately owned technological consulting firm
specialized in data and metadata management based in Italy
quantyca.it
Always a good idea
Data contracts
Data as a Product
Data Centricity Data Fabric
Data Contract
Needed toformalize applications’
accountability on generated data
Foundational to proactively provide
metadatathat enables automation
Essential to clarify meaning, shape, content
and how to access data
images: Flaticon.com
Formal agreement between data producer and consumers
on the structure and semantics of data exchanged
Key characteristics
Good data contracts
Addressable
Expressive
Scoped
Stable
Reliable
Computable
Data Contract
images: Flaticon.com
From an architectural perspective
An interesting paradox
images: Flaticon.com
Slow and ruled context
with vulnerable flows
Fluid landscape resilient
to changes
Schema is not enough
Data Contracts
How do I use this data?
Where can I consume it? How?
How often it is updated?
Who is allowed to use it?
What is its quality?
What does this value mean?
API
How-to
for data interactions
• Service location
• Communication protocols
• Authentication methods
• Endpoints
• Schema
• SLO
• SLA
• Terms and conditions
• Deprecation policy
• Billing policy
• Expected usage patterns
• Schema tagging
• Schema linking
• Semantic linking
Constraints
Rules
to properly use data
Semantic
Context
and meaning
A socio-technical puzzle
Adoption challenges
images: Flaticon.com
Define and enforce
Specifications and tools still immature
Require evolution of operating model,
strong sponsorship and engagement
Technical
perspective
Organizational
perspective
Consumer
Producer
Negotiate and own
Team A Team B
Data Contract
Standardization through a specification
Addressing the technical challenge
dpds.opendatamesh.org
Automation
Interoperability
Scalability
images: Flaticon.com
Validation and Monitoring
Enforcing Data Contracts
CEP
Contract Enforcing Point
Central governance of
data contracts’ lifecycle
CDP
Contract Decision Point
Integration with
underlying tech stack
CIP
Contract Information Point
Census for
archival and retrieval
CEP
Data Contract
CDP
CDP
CDP
1
CIP
4
2
3
5
Validation
CEP CDP
CDP
CDP
CIP
1
3
4
Monitoring
2
A unified approach across service and platforms
Mastering Data Contracts
• Data contracts span across numerous services and platforms
• Schema Registry serves as a cornerstone
• Applying a holistic strategy ensures end-to-end effectiveness
Data
product
Data
contract
CDP
CDP
CIP
Confluent Platform
Schema
Registry
CEP
Data Platform
Data Contracts
Control Center
Thank you. @pietrolatorre
pietro.latorre@quantyca.it
Pietro La Torre
Ad

More Related Content

Similar to Data Contracts Management: Schema Registry and Beyond (20)

A Successful Data Strategy for Insurers in Volatile Times (EMEA)
A Successful Data Strategy for Insurers in Volatile Times (EMEA)A Successful Data Strategy for Insurers in Volatile Times (EMEA)
A Successful Data Strategy for Insurers in Volatile Times (EMEA)
Denodo
 
Artificial intelligence capabilities overview yashowardhan sowale cwin18-india
Artificial intelligence capabilities overview yashowardhan sowale cwin18-indiaArtificial intelligence capabilities overview yashowardhan sowale cwin18-india
Artificial intelligence capabilities overview yashowardhan sowale cwin18-india
Capgemini
 
Automation Trends What to Expect in Automated Data Collection in 2024
Automation Trends What to Expect in Automated Data Collection in 2024Automation Trends What to Expect in Automated Data Collection in 2024
Automation Trends What to Expect in Automated Data Collection in 2024
Andrew Leo
 
The internet of value: What CFOs need to know about Blockchain
The internet of value: What CFOs need to know about BlockchainThe internet of value: What CFOs need to know about Blockchain
The internet of value: What CFOs need to know about Blockchain
Deloitte United States
 
Unlocking the Full Potential of Your Telecom Data with Data Integrity
Unlocking the Full Potential of Your Telecom Data with Data IntegrityUnlocking the Full Potential of Your Telecom Data with Data Integrity
Unlocking the Full Potential of Your Telecom Data with Data Integrity
Precisely
 
Case study of Katpro on SharePoint, Office 365, ASP.Net
Case study of Katpro on SharePoint, Office 365, ASP.NetCase study of Katpro on SharePoint, Office 365, ASP.Net
Case study of Katpro on SharePoint, Office 365, ASP.Net
Katpro Technologies Pvt Ltd
 
Workable Enteprise Data Governance
Workable Enteprise Data GovernanceWorkable Enteprise Data Governance
Workable Enteprise Data Governance
Bhavendra Chavan
 
Democratized Data & Analytics for the Cloud​
Democratized Data & Analytics for the Cloud​Democratized Data & Analytics for the Cloud​
Democratized Data & Analytics for the Cloud​
Precisely
 
Data and Application Modernization in the Age of the Cloud
Data and Application Modernization in the Age of the CloudData and Application Modernization in the Age of the Cloud
Data and Application Modernization in the Age of the Cloud
redmondpulver
 
Unleashing the value of metadata with Talend
Unleashing the value of metadata with Talend Unleashing the value of metadata with Talend
Unleashing the value of metadata with Talend
Jean-Michel Franco
 
Designing Products for the Cloud
Designing Products for the CloudDesigning Products for the Cloud
Designing Products for the Cloud
Steve Keifer
 
Denodo DataFest 2017: The Need for Speed and Agility in Business
Denodo DataFest 2017: The Need for Speed and Agility in BusinessDenodo DataFest 2017: The Need for Speed and Agility in Business
Denodo DataFest 2017: The Need for Speed and Agility in Business
Denodo
 
Vertica Analytics Database general overview
Vertica Analytics Database general overviewVertica Analytics Database general overview
Vertica Analytics Database general overview
Stratebi
 
Transforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming DataTransforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming Data
confluent
 
Maximo User Group Presentation extract - BIRT Reporting options
Maximo User Group Presentation extract - BIRT Reporting optionsMaximo User Group Presentation extract - BIRT Reporting options
Maximo User Group Presentation extract - BIRT Reporting options
Sai Paravastu
 
Mergers Acquisitions and Tech Due Diligence
Mergers Acquisitions and Tech Due DiligenceMergers Acquisitions and Tech Due Diligence
Mergers Acquisitions and Tech Due Diligence
Sharanjeet Kaur
 
Data Contracts Course - Data Management & Data Quality
Data Contracts Course -  Data Management & Data QualityData Contracts Course -  Data Management & Data Quality
Data Contracts Course - Data Management & Data Quality
Data & Analytics Magazin
 
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
confluent
 
Bab 8 (managing it services delivery)
Bab 8 (managing it services delivery)Bab 8 (managing it services delivery)
Bab 8 (managing it services delivery)
Siti Mustiani
 
Loving_HowToDrive-ValuA7A3B4
Loving_HowToDrive-ValuA7A3B4Loving_HowToDrive-ValuA7A3B4
Loving_HowToDrive-ValuA7A3B4
Steven Loving
 
A Successful Data Strategy for Insurers in Volatile Times (EMEA)
A Successful Data Strategy for Insurers in Volatile Times (EMEA)A Successful Data Strategy for Insurers in Volatile Times (EMEA)
A Successful Data Strategy for Insurers in Volatile Times (EMEA)
Denodo
 
Artificial intelligence capabilities overview yashowardhan sowale cwin18-india
Artificial intelligence capabilities overview yashowardhan sowale cwin18-indiaArtificial intelligence capabilities overview yashowardhan sowale cwin18-india
Artificial intelligence capabilities overview yashowardhan sowale cwin18-india
Capgemini
 
Automation Trends What to Expect in Automated Data Collection in 2024
Automation Trends What to Expect in Automated Data Collection in 2024Automation Trends What to Expect in Automated Data Collection in 2024
Automation Trends What to Expect in Automated Data Collection in 2024
Andrew Leo
 
The internet of value: What CFOs need to know about Blockchain
The internet of value: What CFOs need to know about BlockchainThe internet of value: What CFOs need to know about Blockchain
The internet of value: What CFOs need to know about Blockchain
Deloitte United States
 
Unlocking the Full Potential of Your Telecom Data with Data Integrity
Unlocking the Full Potential of Your Telecom Data with Data IntegrityUnlocking the Full Potential of Your Telecom Data with Data Integrity
Unlocking the Full Potential of Your Telecom Data with Data Integrity
Precisely
 
Case study of Katpro on SharePoint, Office 365, ASP.Net
Case study of Katpro on SharePoint, Office 365, ASP.NetCase study of Katpro on SharePoint, Office 365, ASP.Net
Case study of Katpro on SharePoint, Office 365, ASP.Net
Katpro Technologies Pvt Ltd
 
Workable Enteprise Data Governance
Workable Enteprise Data GovernanceWorkable Enteprise Data Governance
Workable Enteprise Data Governance
Bhavendra Chavan
 
Democratized Data & Analytics for the Cloud​
Democratized Data & Analytics for the Cloud​Democratized Data & Analytics for the Cloud​
Democratized Data & Analytics for the Cloud​
Precisely
 
Data and Application Modernization in the Age of the Cloud
Data and Application Modernization in the Age of the CloudData and Application Modernization in the Age of the Cloud
Data and Application Modernization in the Age of the Cloud
redmondpulver
 
Unleashing the value of metadata with Talend
Unleashing the value of metadata with Talend Unleashing the value of metadata with Talend
Unleashing the value of metadata with Talend
Jean-Michel Franco
 
Designing Products for the Cloud
Designing Products for the CloudDesigning Products for the Cloud
Designing Products for the Cloud
Steve Keifer
 
Denodo DataFest 2017: The Need for Speed and Agility in Business
Denodo DataFest 2017: The Need for Speed and Agility in BusinessDenodo DataFest 2017: The Need for Speed and Agility in Business
Denodo DataFest 2017: The Need for Speed and Agility in Business
Denodo
 
Vertica Analytics Database general overview
Vertica Analytics Database general overviewVertica Analytics Database general overview
Vertica Analytics Database general overview
Stratebi
 
Transforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming DataTransforming Financial Services with Event Streaming Data
Transforming Financial Services with Event Streaming Data
confluent
 
Maximo User Group Presentation extract - BIRT Reporting options
Maximo User Group Presentation extract - BIRT Reporting optionsMaximo User Group Presentation extract - BIRT Reporting options
Maximo User Group Presentation extract - BIRT Reporting options
Sai Paravastu
 
Mergers Acquisitions and Tech Due Diligence
Mergers Acquisitions and Tech Due DiligenceMergers Acquisitions and Tech Due Diligence
Mergers Acquisitions and Tech Due Diligence
Sharanjeet Kaur
 
Data Contracts Course - Data Management & Data Quality
Data Contracts Course -  Data Management & Data QualityData Contracts Course -  Data Management & Data Quality
Data Contracts Course - Data Management & Data Quality
Data & Analytics Magazin
 
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
Using Kafka in Your Organization with Real-Time User Insights for a Customer ...
confluent
 
Bab 8 (managing it services delivery)
Bab 8 (managing it services delivery)Bab 8 (managing it services delivery)
Bab 8 (managing it services delivery)
Siti Mustiani
 
Loving_HowToDrive-ValuA7A3B4
Loving_HowToDrive-ValuA7A3B4Loving_HowToDrive-ValuA7A3B4
Loving_HowToDrive-ValuA7A3B4
Steven Loving
 

More from HostedbyConfluent (20)

Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
HostedbyConfluent
 
Renaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit LondonRenaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit London
HostedbyConfluent
 
Evolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at TrendyolEvolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at Trendyol
HostedbyConfluent
 
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking TechniquesEnsuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
HostedbyConfluent
 
Exactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and KafkaExactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and Kafka
HostedbyConfluent
 
Fish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit LondonFish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit London
HostedbyConfluent
 
Tiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit LondonTiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit London
HostedbyConfluent
 
Building a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And WhyBuilding a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And Why
HostedbyConfluent
 
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
HostedbyConfluent
 
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
HostedbyConfluent
 
Navigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka ClustersNavigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka Clusters
HostedbyConfluent
 
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data PlatformApache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
HostedbyConfluent
 
Explaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy PubExplaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy Pub
HostedbyConfluent
 
TL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit LondonTL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit London
HostedbyConfluent
 
A Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSLA Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSL
HostedbyConfluent
 
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing PerformanceMastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
HostedbyConfluent
 
Code-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink AppsCode-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink Apps
HostedbyConfluent
 
Debezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC EcosystemDebezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC Ecosystem
HostedbyConfluent
 
Beyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local DisksBeyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local Disks
HostedbyConfluent
 
Automating Speed: A Proven Approach to Preventing Performance Regressions in ...
Automating Speed: A Proven Approach to Preventing Performance Regressions in ...Automating Speed: A Proven Approach to Preventing Performance Regressions in ...
Automating Speed: A Proven Approach to Preventing Performance Regressions in ...
HostedbyConfluent
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
HostedbyConfluent
 
Renaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit LondonRenaming a Kafka Topic | Kafka Summit London
Renaming a Kafka Topic | Kafka Summit London
HostedbyConfluent
 
Evolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at TrendyolEvolution of NRT Data Ingestion Pipeline at Trendyol
Evolution of NRT Data Ingestion Pipeline at Trendyol
HostedbyConfluent
 
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking TechniquesEnsuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
HostedbyConfluent
 
Exactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and KafkaExactly-once Stream Processing with Arroyo and Kafka
Exactly-once Stream Processing with Arroyo and Kafka
HostedbyConfluent
 
Fish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit LondonFish Plays Pokemon | Kafka Summit London
Fish Plays Pokemon | Kafka Summit London
HostedbyConfluent
 
Tiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit LondonTiered Storage 101 | Kafla Summit London
Tiered Storage 101 | Kafla Summit London
HostedbyConfluent
 
Building a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And WhyBuilding a Self-Service Stream Processing Portal: How And Why
Building a Self-Service Stream Processing Portal: How And Why
HostedbyConfluent
 
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
HostedbyConfluent
 
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
HostedbyConfluent
 
Navigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka ClustersNavigating Private Network Connectivity Options for Kafka Clusters
Navigating Private Network Connectivity Options for Kafka Clusters
HostedbyConfluent
 
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data PlatformApache Flink: Building a Company-wide Self-service Streaming Data Platform
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
HostedbyConfluent
 
Explaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy PubExplaining How Real-Time GenAI Works in a Noisy Pub
Explaining How Real-Time GenAI Works in a Noisy Pub
HostedbyConfluent
 
TL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit LondonTL;DR Kafka Metrics | Kafka Summit London
TL;DR Kafka Metrics | Kafka Summit London
HostedbyConfluent
 
A Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSLA Window Into Your Kafka Streams Tasks | KSL
A Window Into Your Kafka Streams Tasks | KSL
HostedbyConfluent
 
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing PerformanceMastering Kafka Producer Configs: A Guide to Optimizing Performance
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
HostedbyConfluent
 
Code-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink AppsCode-First Approach: Crafting Efficient Flink Apps
Code-First Approach: Crafting Efficient Flink Apps
HostedbyConfluent
 
Debezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC EcosystemDebezium vs. the World: An Overview of the CDC Ecosystem
Debezium vs. the World: An Overview of the CDC Ecosystem
HostedbyConfluent
 
Beyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local DisksBeyond Tiered Storage: Serverless Kafka with No Local Disks
Beyond Tiered Storage: Serverless Kafka with No Local Disks
HostedbyConfluent
 
Automating Speed: A Proven Approach to Preventing Performance Regressions in ...
Automating Speed: A Proven Approach to Preventing Performance Regressions in ...Automating Speed: A Proven Approach to Preventing Performance Regressions in ...
Automating Speed: A Proven Approach to Preventing Performance Regressions in ...
HostedbyConfluent
 
Ad

Recently uploaded (20)

AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
SOFTTECHHUB
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...
Vishnu Singh Chundawat
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
organizerofv
 
Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.
hpbmnnxrvb
 
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond SparkRusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Spark
carlyakerly1
 
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsAI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global Trends
InData Labs
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes
 
Electronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploitElectronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploit
niftliyevhuseyn
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Impelsys Inc.
 
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBuild Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For Devs
Brian McKeiver
 
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
SOFTTECHHUB
 
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath MaestroDev Dives: Automate and orchestrate your processes with UiPath Maestro
Dev Dives: Automate and orchestrate your processes with UiPath Maestro
UiPathCommunity
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...What is Model Context Protocol(MCP) - The new technology for communication bw...
What is Model Context Protocol(MCP) - The new technology for communication bw...
Vishnu Singh Chundawat
 
Cybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure ADCybersecurity Identity and Access Solutions using Azure AD
Cybersecurity Identity and Access Solutions using Azure AD
VICTOR MAESTRE RAMIREZ
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
Semantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AISemantic Cultivators : The Critical Future Role to Enable AI
Semantic Cultivators : The Critical Future Role to Enable AI
artmondano
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
IEDM 2024 Tutorial2_Advances in CMOS Technologies and Future Directions for C...
organizerofv
 
Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.
hpbmnnxrvb
 
Rusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond SparkRusty Waters: Elevating Lakehouses Beyond Spark
Rusty Waters: Elevating Lakehouses Beyond Spark
carlyakerly1
 
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsAI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global Trends
InData Labs
 
Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025Splunk Security Update | Public Sector Summit Germany 2025
Splunk Security Update | Public Sector Summit Germany 2025
Splunk
 
Technology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data AnalyticsTechnology Trends in 2025: AI and Big Data Analytics
Technology Trends in 2025: AI and Big Data Analytics
InData Labs
 
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-UmgebungenHCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
HCL Nomad Web – Best Practices und Verwaltung von Multiuser-Umgebungen
panagenda
 
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes Partner Innovation Updates for May 2025
ThousandEyes
 
Electronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploitElectronic_Mail_Attacks-1-35.pdf by xploit
Electronic_Mail_Attacks-1-35.pdf by xploit
niftliyevhuseyn
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Enhancing ICU Intelligence: How Our Functional Testing Enabled a Healthcare I...
Impelsys Inc.
 
Build Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For DevsBuild Your Own Copilot & Agents For Devs
Build Your Own Copilot & Agents For Devs
Brian McKeiver
 
Ad

Data Contracts Management: Schema Registry and Beyond

  • 1. Data Contracts Management Schema Registry and Beyond
  • 2. About me Pietro La Torre Strategy & Business Development @ Quantyca SpA Quantyca is a privately owned technological consulting firm specialized in data and metadata management based in Italy quantyca.it
  • 3. Always a good idea Data contracts Data as a Product Data Centricity Data Fabric Data Contract Needed toformalize applications’ accountability on generated data Foundational to proactively provide metadatathat enables automation Essential to clarify meaning, shape, content and how to access data images: Flaticon.com Formal agreement between data producer and consumers on the structure and semantics of data exchanged
  • 4. Key characteristics Good data contracts Addressable Expressive Scoped Stable Reliable Computable Data Contract images: Flaticon.com
  • 5. From an architectural perspective An interesting paradox images: Flaticon.com Slow and ruled context with vulnerable flows Fluid landscape resilient to changes
  • 6. Schema is not enough Data Contracts How do I use this data? Where can I consume it? How? How often it is updated? Who is allowed to use it? What is its quality? What does this value mean? API How-to for data interactions • Service location • Communication protocols • Authentication methods • Endpoints • Schema • SLO • SLA • Terms and conditions • Deprecation policy • Billing policy • Expected usage patterns • Schema tagging • Schema linking • Semantic linking Constraints Rules to properly use data Semantic Context and meaning
  • 7. A socio-technical puzzle Adoption challenges images: Flaticon.com Define and enforce Specifications and tools still immature Require evolution of operating model, strong sponsorship and engagement Technical perspective Organizational perspective Consumer Producer Negotiate and own Team A Team B Data Contract
  • 8. Standardization through a specification Addressing the technical challenge dpds.opendatamesh.org Automation Interoperability Scalability images: Flaticon.com
  • 9. Validation and Monitoring Enforcing Data Contracts CEP Contract Enforcing Point Central governance of data contracts’ lifecycle CDP Contract Decision Point Integration with underlying tech stack CIP Contract Information Point Census for archival and retrieval CEP Data Contract CDP CDP CDP 1 CIP 4 2 3 5 Validation CEP CDP CDP CDP CIP 1 3 4 Monitoring 2
  • 10. A unified approach across service and platforms Mastering Data Contracts • Data contracts span across numerous services and platforms • Schema Registry serves as a cornerstone • Applying a holistic strategy ensures end-to-end effectiveness Data product Data contract CDP CDP CIP Confluent Platform Schema Registry CEP Data Platform Data Contracts Control Center