Tutorial Getting Started part 2 - Polyglot Data Modeling

0 likes67 views

The document provides an overview of polyglot data modeling, highlighting its advantages in creating a flexible and consistent data architecture that can adapt to evolving schemas. It emphasizes the concept of a polyglot data model that serves multiple technologies and allows for complex data representations. Additionally, it offers resources for further reading and encourages the use of Hackolade's tools and documentation.

Hackolade Tutorial
Tutorial – Vision & Getting Started
Copyright © 2016-2023 Hackolade 1

Polyglot Data Modeling
See article
Copyright © 2016-2023 Hackolade 2

Logical or Polyglot?
Technology agnostic?
Copyright © 2016-2023 Hackolade 3

Data modeling for polyglot persistence
One single application in the 21st century
Copyright © 2016-2023 Hackolade 4

Every organization has polyglot data pipelines
Build pipelines so they don’t break when the schema evolves
Copyright © 2016-2023 Hackolade 5

Polyglot data models
• Sits over the previous boundary between logical and
physical data models.
• Polyglot data mode is a “Logical model on steroids”:
• allows denormalization, if desired, given access patterns;
• allows complex data types;
• Represents a common physical model that generates
schemas for a variety of technologies, with automatic
mapping to the specific data types of the respective
target technologies.
• built so you could create a library of canonical objects for
your domains, and use them consistently across physical
data models for different target technologies.
Copyright © 2016-2023 Hackolade 6

Polyglot data model
Copyright © 2016-2023 Hackolade 7

Additional benefit of Polyglot data models
A modular, interdependent, consistent data model that allows your
enterprise to leverage the same building blocks across all of their
different models
Copyright © 2016-2023 Hackolade 8

Graph view: “Business user”-friendly representation of concepts
A higher level representation of the entities and relationships in a
model, by levering a Graph Diagram representation of the model
Copyright © 2016-2023 Hackolade 9

Reading material
• See Hackolade online documentation
• The Hackolade Blog
• This excellent new book:
MongoDB Data Modeling & Schema Design
• Many of the principles in the book are related to query
driven modeling based on access patterns
• Hackolade’s on social media: LinkedIn page, Twitter page
• Download Hackolade Studio for free
Copyright © 2016-2023 Hackolade 10

Order on Amazon
Copyright © 2016-2023 Hackolade 11
Pre-order

Upcoming
Copyright © 2016-2023 Hackolade 12

Questions?
Answers!
Copyright © 2016-2023 Hackolade 13

The document outlines the significance of data modeling as an abstract representation of data organization and relationships, which is essential for IT systems' development to ensure data consistency and quality. It describes different types of data models (conceptual, logical, and physical), the importance of normalization and polyglot persistence, and offers guidance on starting data modeling. Additionally, it emphasizes domain-driven data modeling to enhance communication between stakeholders and improve problem-solving.

Hackolade Tutorial - part 13 - Leverage a Polyglot data modelPascalDesmarets1

A polyglot data model is a common physical data model that can generate schemas for different technologies while remaining technology agnostic. It allows for denormalization and complex data types. The polyglot model acts as a master model that physical target models can be derived from. Changes are made at the polyglot level and then target models are refreshed. The document discusses how to create, promote, and derive target models from a polyglot data model as well as handling changes between the models.

Tutorial Getting Started part 1 - OverviewPascalDesmarets1

The document describes Hackolade's data modeling solutions, emphasizing the importance of aligning business and IT through visual modeling and detailed documentation. It discusses the management of complex data backends in an agile manner, highlighting features like metadata-as-code, polyglot data modeling, and the integration of data models with DevOps processes. Hackolade aims to provide a shared understanding of data context to enhance software quality and ensure compliance with regulations.

Hackolade Tutorial - part 4 - Create your first data modelPascalDesmarets1

Tutorial Expert How-To - Add reusable DefinitionsPascalDesmarets1

This document provides an overview of reusable definitions in Hackolade: - Definitions can be created at the entity, model, and external levels to increase productivity, consistency, and data quality - Definitions can be converted from objects and referenced in multiple places - The Model Definitions tab allows maintaining and viewing where definitions are used - JSON Schema previews show how definitions are resolved and converted to internal structures - Definitions can be extended or replaced as needed, and referenced from external files

Tutorial Getting Started part 4 - Domain-Driven Data ModelingPascalDesmarets1

The document is a tutorial on domain-driven data modeling, emphasizing the importance of creating smaller, manageable data models to enhance application complexity. It provides resources for further learning, including links to Hackolade’s documentation, blog, and a recommended book on MongoDB data modeling. Additionally, it encourages readers to engage with Hackolade on social media and download their software.

Hackolade Tutorial - part 9 - Export or forward-engineer.pdfPascalDesmarets1

The Hackolade tutorial discusses the importance of using ER diagrams to facilitate understanding of data structures for application design and schema generation. It highlights the forward-engineering capabilities of Hackolade, detailing various export options such as JSON, YAML, Excel, and integration with CI/CD workflows. Additionally, it provides resources for further reading and learning, including online documentation and social media links.

Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...PascalDesmarets1

This document discusses query-driven data modeling for NoSQL databases. It explains that NoSQL databases have different data models, capabilities, and transactional properties than SQL databases. Data modeling for NoSQL requires unlearning normalization rules and embedding related data together to serve queries from a single document. Important considerations for query-driven modeling include document size, relationship cardinality, indexing impacts, schema versioning strategies, choice of sharding keys, and facilitating communication.

Tutorial Expert How-To - Command Line Interface (CLI)PascalDesmarets1

The document provides a tutorial on using Hackolade's command line interface (CLI) for managing data structures, emphasizing the importance of syncing technical data models with business data dictionaries. It outlines the automation of repetitive tasks, the co-location of data models with application code, and various use cases for governance and code-first approaches. Additionally, it suggests starting with the GUI for visualization and offers resources for further learning.

Tutorial Getting Started part 3 - Metadata-as-CodePascalDesmarets1

The document outlines Hackolade's approach to metadata-as-code, which integrates data models with application code in a single repository, ensuring consistent governance and understanding among stakeholders. It emphasizes the role of Git for version control and collaboration in managing data models and metadata alongside CI/CD workflows. Additionally, it addresses the challenges of maintaining up-to-date metadata as applications evolve and provides resources for further learning.

Tutorial Workgroup - Model versioning and collaborationPascalDesmarets1

The document discusses the integration of data modeling within an agile and DevOps context using Hackolade Studio, emphasizing version control through Git and a collaborative approach to data models. It describes various scenarios for managing changes in data models and highlights the importance of metadata management and orchestration. Additionally, it points to resources for further learning and encourages collaboration between business and IT teams.

Hackolade Tutorial - part 8 - Import or reverse-engineer.pdfPascalDesmarets1

This document discusses importing or reverse-engineering structures into Hackolade data models from various sources. It describes reverse-engineering existing database instances or file-based sources like JSON or YAML to derive a data model. Different techniques are used depending on the source, such as inferring the schema, using connection parameters, or normalizing nested structures. The document provides resources for further reading on reverse-engineering and data modeling.

Tutorial Expert How-To - Compare and MergePascalDesmarets1

Tutorial Expert How-To - Create a model for Avro schemasPascalDesmarets1

The document provides an overview of Apache Avro, highlighting its role in data serialization, schema evolution, and interoperability in various data systems. It introduces Hackolade Studio as a visual design tool for Avro schemas, which supports schema modeling and integration with schema registries. Additionally, it emphasizes the benefits of using Avro schema design, including message validation, optimization, and compatibility features.

Hackolade Tutorial - part 2 - Overview of JSON and JSON schemaPascalDesmarets1

The document provides an overview of JSON (JavaScript Object Notation) as a lightweight data-interchange format, highlighting its structures, including objects and arrays. It discusses the advantages of using JSON in document databases and introduces JSON Schema for annotating and validating JSON data. The tutorial emphasizes the importance of understanding JSON's flexibility while also cautioning about the responsibilities that come with it.

Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdfPascalDesmarets1

This document provides a tutorial on advanced JSON schema attributes including choice, conditionals, and pattern fields, emphasizing their functionality in schema validation and polymorphism. It details how to implement these features with examples, outlining the constraints and conditions under which they operate. Additionally, it offers resources for further reading and utilization of Hackolade tools.

Introduction to Polyglot Persistence Antonios Giannopoulos

This document discusses polyglot persistence, which refers to applications that use multiple database technologies suited to different data and access needs. It provides examples of using a relational database for transactions alongside a NoSQL key-value store for session data. The challenges of such architectures include defining the data model across databases and maintaining consistency. The document advocates choosing databases based on data structure, access patterns, and other organizational needs rather than conforming to stereotypes.

Tutorial Advanced How-To - Oracle 23c Duality viewsPascalDesmarets1

The document introduces Oracle Duality Views, which integrate JSON document advantages with relational model benefits while mitigating their limitations. It explains how these views function in Hackolade Studio, detailing their creation, nested objects, and how to forward and reverse-engineer DDL scripts. Additional resources and reading materials related to data modeling and schema design are also provided.

01_Chapter_Introducing Data Modeling.pdfhieuminhpham1001

The document serves as an introduction to MongoDB data modeling, outlining the importance and types of data models, including conceptual, logical, and physical models. It explains the components of data models, the flexible schema of MongoDB, and highlights specific features such as schema flexibility, horizontal scaling, and the handling of relationships within documents. Furthermore, the document emphasizes the methodology for data modeling and various constraints that may impact design decisions.

01_Chapter_Introducing Data Modeling.pdfhieuminhpham1001

The document provides an overview of data modeling in MongoDB, highlighting its importance for structuring data effectively and ensuring compliance with business rules. It discusses three types of data models: conceptual, logical, and physical, explaining their roles in representing data relationships and structures. Additionally, it covers specific MongoDB features such as schema flexibility, horizontal scaling, and document structures, outlining methodologies for modeling data efficiently.

Chapter – 2 Data Models.pdfTamiratDejene1

The document discusses key concepts in relational data models including entities, attributes, relationships, and constraints. It provides examples of each concept and explains how they are the basic building blocks used to structure data in a relational database. Specific types of entities, attributes, relationships and their properties are defined, such as one-to-one, one-to-many, and many-to-many relationships. Overall, the document serves as an introduction to fundamental concepts in relational data modeling.

Processing large-scale graphs with Google PregelMax Neunhöffer

The document discusses the implementation of Pregel, a framework for processing large-scale graphs, utilizing a map-reduce approach tailored for graph data structures. It details the different types of graph algorithms, the multi-model database capabilities of ArangoDB, and features of the Foxx microservice framework for extending database operations. The emphasis is on Pregel's efficient operation for distributed graphs, allowing for custom queries and real-time data processing.

Data modelingAbdoulaye M Yansane

This document provides an overview of fundamental data modeling skills that all developers should have. It discusses what data modeling is, how data models are used in practice, common data modeling notations, and how to model data through tasks like identifying entity types and attributes, applying naming conventions, identifying relationships, and normalizing data. The primary goal is to help application developers gain an understanding of the critical activities performed by Agile DBAs to improve communication between these roles.

Dry-ing MagritteESUG

This document discusses extending the Magritte framework to allow for more customization options when displaying domain objects. It proposes using ports and adapters, custom components, descriptions, renderers, and mementos to add features like custom formatting, multi-language support, dynamic hiding of fields, AJAX interactions, and user-defined reports and settings without duplicating the framework's core functionality.

Polyglot PersistenceScott Leberknight

This document provides a high-level overview of polyglot persistence and different database technologies. It begins by discussing the benefits of polyglot persistence in allowing the use of multiple data storage technologies based on application needs. It then summarizes several common database types including relational, document-oriented, key-value, and column-oriented databases. It also discusses database properties like ACID compliance, scaling, and data consistency models. The document concludes with examples of Amazon's SimpleDB and DynamoDB key-value stores.

Data modelsSamuel Igbanogu

1-SDLC - Development Models – Waterfall, Rapid Application Development, Agile...JOHNLEAK1

This document provides information about different types of data models: 1. Conceptual data models define entities, attributes, and relationships at a high level without technical details. 2. Logical data models build on conceptual models by adding more detail like data types but remain independent of specific databases. 3. Physical data models describe how the database will be implemented for a specific database system, including keys, constraints and other features.

Lecture 2-DATABASE MODELS lecture 2.pptxelvis24mutura

Tutorial Workgroup - Working with ForksPascalDesmarets1

The document outlines strategies for using Git in development, focusing on the 'shared repository' and 'fork & pull' methods. It explains how the fork & pull strategy allows contributors to work independently without needing direct access to the original repository, making it ideal for open-source projects. Additionally, it discusses how to manage forks, collaborate effectively, and keep repositories updated within environments like GitHub.

Tutorial Expert How-To - Docker-based automationPascalDesmarets1

The document discusses Hackolade's tutorial on utilizing Docker for data modeling automation, emphasizing the integration of data models within a DevOps lifecycle. It covers concepts such as domain-driven design, governance, and various use cases for automating schema generation and documentation processes. Additionally, it provides resources for further learning and collaboration through Git and Docker functionalities.

More Related Content

More from PascalDesmarets1 (7)

Tutorial Workgroup - Working with ForksPascalDesmarets1

Tutorial Expert How-To - Docker-based automationPascalDesmarets1

Tutorial Expert How-To - Verify Data ModelPascalDesmarets1

The document outlines the process of data model verification using Hackolade, emphasizing the importance of ensuring consistency and quality of data models through glossary and attribute verifications. It details the definitions and significance of class and prime terms, along with guidelines on implementing verifications and interpreting exception reports. Specific verification rules for business names and attributes are also provided, highlighting potential issues and the structure of the verification reports.

Tutorial Expert How-To - Naming ConventionsPascalDesmarets1

The document provides a tutorial on naming conventions for Hackolade Studio, highlighting the importance of maintaining separate business and technical names for modeling objects. It outlines the options for synchronizing these names, the use of a conversion file for handling abbreviations, and the specific parameters that vary based on the target environment. Additionally, it addresses how to make changes in configurations, including coupling relationships between names during reverse-engineering.

Tutorial Expert How-To - Export-Import with Excel templatePascalDesmarets1

The document provides a tutorial on using an Excel template for exporting and importing data with Hackolade Studio. It outlines capabilities such as bulk editing, maintaining properties, and reverse engineering models, while emphasizing the importance of matching versions and ensuring valid references. Best practices for successful integration include conducting small tests and leveraging Excel's features for efficient modifications.

Tutorial Expert How-To - Custom propertiesPascalDesmarets1

This document provides an overview of custom properties in Hackolade Studio. It explains that custom properties allow extending standard properties to track additional metadata. They are configured through JSON files and can be applied at different levels like the model, entity, or field. Various control types are supported for custom properties. Examples demonstrate how to define custom properties for entity details, attribute details for different data types, and a custom properties tab. Advanced syntax and sharing custom properties across targets are also discussed.

Hackolade Tutorial - part 12 - Create a REST API modelPascalDesmarets1

This tutorial explains how to create a REST API model visually using Swagger or OpenAPI without needing to know the syntax, appealing to API-first and design-first methodologies. It provides a step-by-step approach to design, from creating reusable components to generating OpenAPI documentation, illustrated through an example of a URL shortener service. The tutorial emphasizes the benefits of design-first processes, including reduced total cost of ownership and easier evolution of the API.

Tutorial Workgroup - Working with ForksPascalDesmarets1

Tutorial Expert How-To - Docker-based automationPascalDesmarets1

Tutorial Expert How-To - Verify Data ModelPascalDesmarets1

Tutorial Expert How-To - Naming ConventionsPascalDesmarets1

Tutorial Expert How-To - Export-Import with Excel templatePascalDesmarets1

Tutorial Expert How-To - Custom propertiesPascalDesmarets1

Hackolade Tutorial - part 12 - Create a REST API modelPascalDesmarets1

Recently uploaded (20)

Raman Bhaumik - Passionate Tech EnthusiastRaman Bhaumik

FME for Distribution & Transmission Integrity Management Program (DIMP & TIMP)Safe Software

Peoples Gas in Chicago, IL has changed to a new Distribution & Transmission Integrity Management Program (DIMP & TIMP) software provider in recent years. In order to successfully deploy the new software we have created a series of ETL processes using FME Form to transform our gas facility data to meet the required DIMP & TIMP data specifications. This presentation will provide an overview of how we used FME to transform data from ESRI’s Utility Network and several other internal and external sources to meet the strict data specifications for the DIMP and TIMP software solutions.

Edge-banding-machines-edgeteq-s-200-en-.pdfAmirStern2

OpenACC and Open Hackathons Monthly Highlights June 2025OpenACC

The OpenACC organization focuses on enhancing parallel computing skills and advancing interoperability in scientific applications through hackathons and training. The upcoming 2025 Open Accelerated Computing Summit (OACS) aims to explore the convergence of AI and HPC in scientific computing and foster knowledge sharing. This year's OACS welcomes talk submissions from a variety of topics, from Using Standard Language Parallelism to Computer Vision Applications. The document also highlights several open hackathons, a call to apply for NVIDIA Academic Grant Program and resources for optimizing scientific applications using OpenACC directives.

OWASP Barcelona 2025 Threat Model LibraryPetraVukmirovic

Wenn alles versagt - IBM Tape schützt, was zählt! Und besonders mit dem neust...Josef Weingand

Smarter Aviation Data Management: Lessons from Swedavia Airports and SwecoSafe Software

Managing airport and airspace data is no small task, especially when you’re expected to deliver it in AIXM format without spending a fortune on specialized tools. But what if there was a smarter, more affordable way? Join us for a behind-the-scenes look at how Sweco partnered with Swedavia, the Swedish airport operator, to solve this challenge using FME and Esri. Learn how they built automated workflows to manage periodic updates, merge airspace data, and support data extracts – all while meeting strict government reporting requirements to the Civil Aviation Administration of Sweden. Even better? Swedavia built custom services and applications that use the FME Flow REST API to trigger jobs and retrieve results – streamlining tasks like securing the quality of new surveyor data, creating permdelta and baseline representations in the AIS schema, and generating AIXM extracts from their AIS data. To conclude, FME expert Dean Hintz will walk through a GeoBorders reading workflow and highlight recent enhancements to FME’s AIXM (Aeronautical Information Exchange Model) processing and interpretation capabilities. Discover how airports like Swedavia are harnessing the power of FME to simplify aviation data management, and how you can too.

Creating Inclusive Digital Learning with AI: A Smarter, Fairer FutureImpelsys Inc.

Have you ever struggled to read a tiny label on a medicine box or tried to navigate a confusing website? Now imagine if every learning experience felt that way—every single day. For millions of people living with disabilities, poorly designed content isn’t just frustrating. It’s a barrier to growth. Inclusive learning is about fixing that. And today, AI is helping us build digital learning that’s smarter, kinder, and accessible to everyone. Accessible learning increases engagement, retention, performance, and inclusivity for everyone. Inclusive design is simply better design.

You are not excused! How to avoid security blind spots on the way to productionMichele Leroux Bustamante

We live in an ever evolving landscape for cyber threats creating security risk for your production systems. Mitigating these risks requires participation throughout all stages from development through production delivery - and by every role including architects, developers QA and DevOps engineers, product owners and leadership. No one is excused! This session will cover examples of common mistakes or missed opportunities that can lead to vulnerabilities in production - and ways to do better throughout the development lifecycle.

Powering Multi-Page Web Applications Using Flow Apps and FME Data StreamingSafe Software

Unleash the potential of FME Flow to build and deploy advanced multi-page web applications with ease. Discover how Flow Apps and FME’s data streaming capabilities empower you to create interactive web experiences directly within FME Platform. Without the need for dedicated web-hosting infrastructure, FME enhances both data accessibility and user experience. Join us to explore how to unlock the full potential of FME for your web projects and seamlessly integrate data-driven applications into your workflows.

ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdfMuhammad Rizwan Akram

DC Inverter Air Conditioners are revolutionizing the cooling industry by delivering affordable, energy-efficient, and environmentally sustainable climate control solutions. Unlike conventional fixed-speed air conditioners, DC inverter systems operate with variable-speed compressors that modulate cooling output based on demand, significantly reducing energy consumption and extending the lifespan of the appliance. These systems are critical in reducing electricity usage, lowering greenhouse gas emissions, and promoting eco-friendly technologies in residential and commercial sectors. With advancements in compressor control, refrigerant efficiency, and smart energy management, DC inverter air conditioners have become a benchmark in sustainable climate control solutions

Turning the Page – How AI is Exponentially Increasing Speed, Accuracy, and Ef...Impelsys Inc.

Artificial Intelligence (AI) has become a game-changer in content creation, automating tasks that were once very time-consuming and labor-intensive. AI-powered tools are now capable of generating high-quality articles, blog posts, and even poetry by analyzing large datasets of text and producing human-like writing. However, AI’s influence on content generation is not limited to text; it has also made advancements in multimedia content, such as image, video, and audio generation. AI-powered tools can now transform raw images and footage into visually stunning outputs, and are all set to have a profound impact on the publishing industry.

The Future of Data, AI, and AR: Innovation Inspired by You.pdfSafe Software

Artificial Intelligence in the Nonprofit Boardroom.pdfOnBoard

cnc-processing-centers-centateq-p-110-en.pdfAmirStern2

מרכז עיבודים תעשייתי בעל 3/4/5 צירים, עד 22 החלפות כלים עם כל אפשרויות העיבוד הדרושות. בעל שטח עבודה גדול ומחשב נוח וקל להפעלה בשפה העברית/רוסית/אנגלית/ספרדית/ערבית ועוד.. מסוגל לבצע פעולות עיבוד שונות המתאימות לענפים שונים: קידוח אנכי, אופקי, ניסור, וכרסום אנכי.

Enabling BIM / GIS integrations with Other Systems with FMESafe Software

Jacobs has successfully utilized FME to tackle the complexities of integrating diverse data sources in a confidential $1 billion campus improvement project. The project aimed to create a comprehensive digital twin by merging Building Information Modeling (BIM) data, Construction Operations Building Information Exchange (COBie) data, and various other data sources into a unified Geographic Information System (GIS) platform. The challenge lay in the disparate nature of these data sources, which were siloed and incompatible with each other, hindering efficient data management and decision-making processes. To address this, Jacobs leveraged FME to automate the extraction, transformation, and loading (ETL) of data between ArcGIS Indoors and IBM Maximo. This process ensured accurate transfer of maintainable asset and work order data, creating a comprehensive 2D and 3D representation of the campus for Facility Management. FME's server capabilities enabled real-time updates and synchronization between ArcGIS Indoors and Maximo, facilitating automatic updates of asset information and work orders. Additionally, Survey123 forms allowed field personnel to capture and submit data directly from their mobile devices, triggering FME workflows via webhooks for real-time data updates. This seamless integration has significantly enhanced data management, improved decision-making processes, and ensured data consistency across the project lifecycle.

“Key Requirements to Successfully Implement Generative AI in Edge Devices—Opt...Edge AI and Vision Alliance

For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2025/06/key-requirements-to-successfully-implement-generative-ai-in-edge-devices-optimized-mapping-to-the-enhanced-npx6-neural-processing-unit-ip-a-presentation-from-synopsys/ Gordon Cooper, Principal Product Manager at Synopsys, presents the “Key Requirements to Successfully Implement Generative AI in Edge Devices—Optimized Mapping to the Enhanced NPX6 Neural Processing Unit IP” tutorial at the May 2025 Embedded Vision Summit. In this talk, Cooper discusses emerging trends in generative AI for edge devices and the key role of transformer-based neural networks. He reviews the distinct attributes of transformers, their advantages over conventional convolutional neural networks and how they enable generative AI. Cooper then covers key requirements that must be met for neural processing units (NPU) to support transformers and generative AI in edge device applications. He uses transformer-based generative AI examples to illustrate the efficient mapping of these workloads onto the enhanced Synopsys ARC NPX NPU IP family.

Tech-ASan: Two-stage check for Address Sanitizer - Yixuan Cao.pdfcaoyixuan2019

FIDO Seminar: Authentication for a Billion Consumers - Amazon.pptxFIDO Alliance

MuleSoft for AgentForce : Topic Center and API Catalogshyamraj55

This presentation dives into how MuleSoft empowers AgentForce with organized API discovery and streamlined integration using Topic Center and the API Catalog. Learn how these tools help structure APIs around business needs, improve reusability, and simplify collaboration across teams. Ideal for developers, architects, and business stakeholders looking to build a connected and scalable API ecosystem within AgentForce.