SlideShare a Scribd company logo
Hackolade Tutorial
Tutorial – Vision & Getting Started
Copyright © 2016-2023 Hackolade 1
Hackolade Vision
Copyright © 2016-2023 Hackolade 2
At the highest level
Hackolade wants to
Reconcile
Business and IT
through a
shared understanding of the
context and meaning of data
Copyright © 2016-2023 Hackolade 3
Copyright © 2016-2023 Hackolade 4
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries , single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Copyright © 2016-2023 Hackolade 5
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries , single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Copyright © 2016-2023 Hackolade 6
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries, single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Copyright © 2016-2023 Hackolade 7
Managing data
complexity: different
technologies,
transformations,
constant evolution
(mis)Alignment of
Business and IT:
(mis)understanding of
requirements,
(mis)interpretation of
data, (un)informed
business decisions
Design, document and
maintain complex data
backends in an Agile way
Align business and IT
through visual modeling
and detailed
documentation
Data governance to
increase quality and
facilitate regulatory
compliance
Visual data modeling for all
21st century data formats and
backend systems: SQL and
NoSQL databases, Storage
formats & IDLs,
(REST)APIs, JSON
Polyglot Data
Modeling: enterprise-wide
data model covering a variety
of different targets
Metadata-as-Code:
Collaboration, versioning,
branching, change tracking,
traceability, peer reviews,
conflict resolution, publication
to business-facing data catalogs
and dictionaries , single source-
of-truth for business and
technical stakeholders
Better Software quality,
faster delivery, lower Total
Cost of Ownership
Shared understanding of
context and meaning of
data, allowing you to make
business sense of your
data
Compliance with rules and
regulations
Why? How? What? Results!
Hackolade Overview
Copyright © 2016-2023 Hackolade 8
1. Data modeling for the 21st century
• Hackolade Studio provides
developers, architects and
data modelers with the core
functionality that they need –
without trying to offer all
solutions to everyone
• Hackolade Studio has been
built from the ground up for
the polyglot, agile world
• Simplicity and productivity
are core design principles that
have been adopted from the
start
2. Polyglot data modeling
• One, comprehensive data model that covers both SQL/NoSQL
backends, data pipelines, as well as streaming services and APIs
• Full forward- and reverse-engineering capabilities, with automation
and customizability built in
Tutorial Getting Started part 1 - Overview
3. Enabling Agile with Metadata-as-Code
• Providing the same level of governance over our schemas, data
models and metadata as we do for our code, will require us to
manage these artefacts in a similar way
• Git (or its platform providers) are excellent repositories to provide
control, versioning, collaboration, and automation
• Using this, Hackolade orchestrates Metadata pipelines to keep in sync
the technical data structures with business-facing data dictionaries
• Provide a Single Source-of-Truth for both business and technical
stakeholders
Design
Data Model
Generate
Schema
Catalog
Publish
Use
Evaluate
Data
Modeler
Data
Architect
Subject
Matter
Expert
Data
Engineer
Data
Analyst
Business
Analyst
Metadata lifecycle unified with DevOps applications
Copyright © 2016-2023 Hackolade
Application
Metadata Dev Ops
What sets Hackolade Studio apart
• Patented technology: unique way of representing database schema
models for database systems and REST APIs
• Runs not only on Windows, but also Mac and Linux
• Simple to install and get started
• Unique collection of target-specific plugins to enable reverse and
forward engineering of data models for relational and non-relational
data stores and storage formats
• True technology-agnostic polyglot data modeling
• Unique integration with Git repositories, enabling co-location of data
models with other business and technical artefacts
• Unique integration with data dictionaries (eg. Collibra)
Copyright © 2016-2023 Hackolade 14
Polyglot Data Modeling
See article
Copyright © 2016-2023 Hackolade 15
Logical or Polyglot?
Technology agnostic?
Copyright © 2016-2023 Hackolade 16
Data modeling for polyglot persistence
One single application in the 21st century
Copyright © 2016-2023 Hackolade 17
Every organization has polyglot data pipelines
Build pipelines so they don’t break when the schema evolves!
Copyright © 2016-2023 Hackolade 18
Polyglot data models
• Sits over the previous boundary between logical and
physical data models.
• Polyglot data mode is a “Logical model on steroids”:
• allows denormalization, if desired, given access patterns;
• allows complex data types;
• Represents a common physical model that generates
schemas for a variety of technologies, with automatic
mapping to the specific data types of the respective
target technologies.
• built so you could create a library of canonical objects for
your domains, and use them consistently across physical
data models for different target technologies.
Copyright © 2016-2023 Hackolade 19
Additional benefit of Polyglot Data Models
A modular, interdependent, consistent data model that allows your
enterprise to leverage the same building blocks across all of their
different models
Copyright © 2016-2023 Hackolade 20
“Business user”-friendly representation of concepts
A higher level representation of the entities and relationships in a
model, by levering a Graph Diagram representation of the model
Copyright © 2016-2023 Hackolade 21
Metadata-as-code
See article
Copyright © 2016-2023 Hackolade 22
Metadata-as-Code: WHAT is it?
• data models are co-located with application code thanks to a tight
integration with Git repositories and DevOps CI/CD workflows.
• This repository becomes the Single Source-of-Truth for all stakeholders
• data models and their schema artefacts closely follow the lifecycle of
application development and deployment to operations.
• technical structures are published in data catalogs and kept in-sync to
ensure a shared understanding of meaning and context with business
users.
Copyright © 2016-2023 Hackolade 23
Metadata-as-Code answers key questions
• Does everyone interpret a column name in a report the same
way?
• Does the column name represent exactly the nuances of what
has been measured?
• Does the tech side of the organization share the same
understanding as the business side?
• How about when applications evolve quickly and new columns
are added at a rapid pace?
• Is the metadata in the catalogs of the business kept up-to-date
with the changes in data structures?
Copyright © 2016-2023 Hackolade 24
Data models
Schema contracts
Application code
Git repository DevOps CI/CD
pipeline
Application
Database
Copyright © 2016-2023 Hackolade 25
Hackolade + Git: Co-locate metadata with code
Environments:
• Dev
• Test
• Integration
• Production
ALTER scripts
Command line interface
To automate conversions and operations
Copyright © 2016-2023 Hackolade 26
Publishing to business community of data citizens
Orchestrate metadata pipelines to keep in sync technical data
structures with business facing data dictionaries
Copyright © 2016-2023 Hackolade 27
Domain-Driven data
modeling
See article
Copyright © 2016-2023 Hackolade 28
Delivering Domain-Driven Data Modeling
Copyright © 2016-2023 Hackolade 29
Getting Started with
Hackolade Studio
Copyright © 2016-2023 Hackolade 30
Three easy steps!
Copyright © 2016-2023 Hackolade 31
Design high-performing data structures
Design your tables, collections, and graphs, attributes and their data types, descriptions and constraints,
using a visual representation of nested structures.
Polyglot data modeling
Optimize with dynamic schema evolution during agile cycles
Link entities logically. Keep track of implicit relationships and denormalization. For each database
or protocol technology, add the required metadata. Or reverse-engineer from dev or prod instances.
Schema design features
Publish your schemas and documentation
Leverage GitOps to achieve metadata-as-code and synchronize technical schemas with business data
dictionaries. Generate MongoDB validator, CQL, HQL, DDLs for relational, REST APIs, JSON Schema, Avro,
Parquet, etc… plus human-readable documentation in HTML, Markdown, or PDF.
Metadata-as-Code
Hackolade Editions
• Differences outlined on this page.
• Professional Edition
• no limit on the size of models,
• produces documentation in different flexible formats.
• Key features include
• forward- and reverse-engineering functions tightly integrated with all different target technologies
• a powerful Command-Line Interface
• ability to compare different versions of your model, and optionally merge selected parts
• naming conventions, library of reusable definitions, domain/subject area ERD views,
inference of PK & FK relationships, generation of mock data for testing, model-driven API
generation, model verification, denormalization of SQL schemas, …
• Workgroup Edition
• includes all of the features of the Professional edition, plus
• a native integration with Git repositories for your data models: this enables versioning, branching,
change tracking, collaboration, peer review, and other related capabilities.
• more information in this section of our online documentation.
Copyright © 2016-2023 Hackolade 32
Installing Hackolade
• Download the latest version of the software from our website
• Support for Windows, Mac and Linux
• Check the installation instructions:
• For Windows
• For Mac
• For Linux
• For Docker containers
• Check the notes on Software registration
• Different behaviour for Individual or Concurrent licenses: see article
• Specific instructions for Network proxies
• Specific instructions for Virtual Machines / VDI environments
Copyright © 2016-2023 Hackolade 33
Getting help
• eLearning platform with self-paced online tutorials
• Read the online manual
• Watch our videos
• Send an email (support@hackolade.com)
Copyright © 2016-2023 Hackolade 34
Reading material
• See Hackolade online documentation
• The Hackolade Blog
• This excellent new book:
MongoDB Data Modeling & Schema Design
• Many of the principles in the book are related to query
driven modeling based on access patterns
• Hackolade’s on social media: LinkedIn page, Twitter page
• Download Hackolade Studio for free
Copyright © 2016-2023 Hackolade 35
Order on Amazon
Copyright © 2016-2023 Hackolade 36
Pre-order
Upcoming
Copyright © 2016-2023 Hackolade 37
Questions?
Answers!
Copyright © 2016-2023 Hackolade 38
Ad

Recommended

Hackolade Tutorial - part 1 - What is a data model
Hackolade Tutorial - part 1 - What is a data model
PascalDesmarets1
 
Data Modeling in Looker
Data Modeling in Looker
Looker
 
FIWARE Global Summit - Big Data and Machine Learning with FIWARE
FIWARE Global Summit - Big Data and Machine Learning with FIWARE
FIWARE
 
data mining and data warehousing
data mining and data warehousing
Sunny Gandhi
 
NoSQL
NoSQL
Radu Potop
 
Data lake
Data lake
GHAZOUANI WAEL
 
Hackolade Tutorial - part 13 - Leverage a Polyglot data model
Hackolade Tutorial - part 13 - Leverage a Polyglot data model
PascalDesmarets1
 
Building Data Quality pipelines with Apache Spark and Delta Lake
Building Data Quality pipelines with Apache Spark and Delta Lake
Databricks
 
Integrate with coupa
Integrate with coupa
Coupa Software
 
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
Simplilearn
 
Data Product Architectures
Data Product Architectures
Benjamin Bengfort
 
Scaling Data Analytics Workloads on Databricks
Scaling Data Analytics Workloads on Databricks
Databricks
 
OLAP & Data Warehouse
OLAP & Data Warehouse
Zalpa Rathod
 
Integrating Coupa with Your Enterprise
Integrating Coupa with Your Enterprise
Coupa Software
 
Schema Evolution for Resilient Data microservices
Schema Evolution for Resilient Data microservices
Vinícius Carvalho
 
Spark rdd vs data frame vs dataset
Spark rdd vs data frame vs dataset
Ankit Beohar
 
Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...
Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...
Denodo
 
Big Data Modeling
Big Data Modeling
Hans Hultgren
 
Spark Performance Tuning .pdf
Spark Performance Tuning .pdf
Amit Raj
 
Normal Formlar
Normal Formlar
Sibel Kuzgun AKIN
 
Big Data: Big SQL and HBase
Big Data: Big SQL and HBase
Cynthia Saracco
 
Machine Learning and AI at Oracle
Machine Learning and AI at Oracle
Sandesh Rao
 
Introduction to Oracle Database
Introduction to Oracle Database
puja_dhar
 
Modern Data architecture Design
Modern Data architecture Design
Kujambu Murugesan
 
Chapitre 4 no sql
Chapitre 4 no sql
Mouna Torjmen
 
Observability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineage
Databricks
 
FAIR principles and metrics for evaluation
FAIR principles and metrics for evaluation
Michel Dumontier
 
Distributed Models Over Distributed Data with MLflow, Pyspark, and Pandas
Distributed Models Over Distributed Data with MLflow, Pyspark, and Pandas
Databricks
 
Tutorial Getting Started part 3 - Metadata-as-Code
Tutorial Getting Started part 3 - Metadata-as-Code
PascalDesmarets1
 
Tutorial Getting Started part 2 - Polyglot Data Modeling
Tutorial Getting Started part 2 - Polyglot Data Modeling
PascalDesmarets1
 

More Related Content

What's hot (20)

Integrate with coupa
Integrate with coupa
Coupa Software
 
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
Simplilearn
 
Data Product Architectures
Data Product Architectures
Benjamin Bengfort
 
Scaling Data Analytics Workloads on Databricks
Scaling Data Analytics Workloads on Databricks
Databricks
 
OLAP & Data Warehouse
OLAP & Data Warehouse
Zalpa Rathod
 
Integrating Coupa with Your Enterprise
Integrating Coupa with Your Enterprise
Coupa Software
 
Schema Evolution for Resilient Data microservices
Schema Evolution for Resilient Data microservices
Vinícius Carvalho
 
Spark rdd vs data frame vs dataset
Spark rdd vs data frame vs dataset
Ankit Beohar
 
Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...
Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...
Denodo
 
Big Data Modeling
Big Data Modeling
Hans Hultgren
 
Spark Performance Tuning .pdf
Spark Performance Tuning .pdf
Amit Raj
 
Normal Formlar
Normal Formlar
Sibel Kuzgun AKIN
 
Big Data: Big SQL and HBase
Big Data: Big SQL and HBase
Cynthia Saracco
 
Machine Learning and AI at Oracle
Machine Learning and AI at Oracle
Sandesh Rao
 
Introduction to Oracle Database
Introduction to Oracle Database
puja_dhar
 
Modern Data architecture Design
Modern Data architecture Design
Kujambu Murugesan
 
Chapitre 4 no sql
Chapitre 4 no sql
Mouna Torjmen
 
Observability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineage
Databricks
 
FAIR principles and metrics for evaluation
FAIR principles and metrics for evaluation
Michel Dumontier
 
Distributed Models Over Distributed Data with MLflow, Pyspark, and Pandas
Distributed Models Over Distributed Data with MLflow, Pyspark, and Pandas
Databricks
 
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
What Is Apache Spark? | Introduction To Apache Spark | Apache Spark Tutorial ...
Simplilearn
 
Scaling Data Analytics Workloads on Databricks
Scaling Data Analytics Workloads on Databricks
Databricks
 
OLAP & Data Warehouse
OLAP & Data Warehouse
Zalpa Rathod
 
Integrating Coupa with Your Enterprise
Integrating Coupa with Your Enterprise
Coupa Software
 
Schema Evolution for Resilient Data microservices
Schema Evolution for Resilient Data microservices
Vinícius Carvalho
 
Spark rdd vs data frame vs dataset
Spark rdd vs data frame vs dataset
Ankit Beohar
 
Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...
Denodo Data Virtualization Platform: Overview (session 1 from Architect to Ar...
Denodo
 
Spark Performance Tuning .pdf
Spark Performance Tuning .pdf
Amit Raj
 
Big Data: Big SQL and HBase
Big Data: Big SQL and HBase
Cynthia Saracco
 
Machine Learning and AI at Oracle
Machine Learning and AI at Oracle
Sandesh Rao
 
Introduction to Oracle Database
Introduction to Oracle Database
puja_dhar
 
Modern Data architecture Design
Modern Data architecture Design
Kujambu Murugesan
 
Observability for Data Pipelines With OpenLineage
Observability for Data Pipelines With OpenLineage
Databricks
 
FAIR principles and metrics for evaluation
FAIR principles and metrics for evaluation
Michel Dumontier
 
Distributed Models Over Distributed Data with MLflow, Pyspark, and Pandas
Distributed Models Over Distributed Data with MLflow, Pyspark, and Pandas
Databricks
 

Similar to Tutorial Getting Started part 1 - Overview (20)

Tutorial Getting Started part 3 - Metadata-as-Code
Tutorial Getting Started part 3 - Metadata-as-Code
PascalDesmarets1
 
Tutorial Getting Started part 2 - Polyglot Data Modeling
Tutorial Getting Started part 2 - Polyglot Data Modeling
PascalDesmarets1
 
Tutorial Workgroup - Model versioning and collaboration
Tutorial Workgroup - Model versioning and collaboration
PascalDesmarets1
 
Mark Simpson - UKOUG23 - Refactoring Monolithic Oracle Database Applications ...
Mark Simpson - UKOUG23 - Refactoring Monolithic Oracle Database Applications ...
marksimpsongw
 
Data Architecture, Solution Architecture, Platform Architecture — What’s the ...
Data Architecture, Solution Architecture, Platform Architecture — What’s the ...
DATAVERSITY
 
Data and Application Modernization in the Age of the Cloud
Data and Application Modernization in the Age of the Cloud
redmondpulver
 
Data Virtualization: Introduction and Business Value (UK)
Data Virtualization: Introduction and Business Value (UK)
Denodo
 
Tutorial Expert How-To - Command Line Interface (CLI)
Tutorial Expert How-To - Command Line Interface (CLI)
PascalDesmarets1
 
Hackolade Tutorial - part 9 - Export or forward-engineer.pdf
Hackolade Tutorial - part 9 - Export or forward-engineer.pdf
PascalDesmarets1
 
ER/Studio Data Architect Datasheet
ER/Studio Data Architect Datasheet
Embarcadero Technologies
 
Unlock Your Data for ML & AI using Data Virtualization
Unlock Your Data for ML & AI using Data Virtualization
Denodo
 
SegmentOfOne
SegmentOfOne
Dave Callaghan
 
Operational Analytics Using Spark and NoSQL Data Stores
Operational Analytics Using Spark and NoSQL Data Stores
DATAVERSITY
 
locotalk-whitepaper-2016
locotalk-whitepaper-2016
Anthony Wijnen
 
Virtualisation de données : Enjeux, Usages & Bénéfices
Virtualisation de données : Enjeux, Usages & Bénéfices
Denodo
 
Resume
Resume
Vasudeva Bhaskara
 
Embedded-ml(ai)applications - Bjoern Staender
Embedded-ml(ai)applications - Bjoern Staender
Dataconomy Media
 
MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands
MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands
MongoDB
 
Tutorial Expert How-To - Docker-based automation
Tutorial Expert How-To - Docker-based automation
PascalDesmarets1
 
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo
 
Tutorial Getting Started part 3 - Metadata-as-Code
Tutorial Getting Started part 3 - Metadata-as-Code
PascalDesmarets1
 
Tutorial Getting Started part 2 - Polyglot Data Modeling
Tutorial Getting Started part 2 - Polyglot Data Modeling
PascalDesmarets1
 
Tutorial Workgroup - Model versioning and collaboration
Tutorial Workgroup - Model versioning and collaboration
PascalDesmarets1
 
Mark Simpson - UKOUG23 - Refactoring Monolithic Oracle Database Applications ...
Mark Simpson - UKOUG23 - Refactoring Monolithic Oracle Database Applications ...
marksimpsongw
 
Data Architecture, Solution Architecture, Platform Architecture — What’s the ...
Data Architecture, Solution Architecture, Platform Architecture — What’s the ...
DATAVERSITY
 
Data and Application Modernization in the Age of the Cloud
Data and Application Modernization in the Age of the Cloud
redmondpulver
 
Data Virtualization: Introduction and Business Value (UK)
Data Virtualization: Introduction and Business Value (UK)
Denodo
 
Tutorial Expert How-To - Command Line Interface (CLI)
Tutorial Expert How-To - Command Line Interface (CLI)
PascalDesmarets1
 
Hackolade Tutorial - part 9 - Export or forward-engineer.pdf
Hackolade Tutorial - part 9 - Export or forward-engineer.pdf
PascalDesmarets1
 
Unlock Your Data for ML & AI using Data Virtualization
Unlock Your Data for ML & AI using Data Virtualization
Denodo
 
Operational Analytics Using Spark and NoSQL Data Stores
Operational Analytics Using Spark and NoSQL Data Stores
DATAVERSITY
 
locotalk-whitepaper-2016
locotalk-whitepaper-2016
Anthony Wijnen
 
Virtualisation de données : Enjeux, Usages & Bénéfices
Virtualisation de données : Enjeux, Usages & Bénéfices
Denodo
 
Embedded-ml(ai)applications - Bjoern Staender
Embedded-ml(ai)applications - Bjoern Staender
Dataconomy Media
 
MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands
MongoDB .local Toronto 2019: MongoDB – Powering the new age data demands
MongoDB
 
Tutorial Expert How-To - Docker-based automation
Tutorial Expert How-To - Docker-based automation
PascalDesmarets1
 
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo Partner Connect: A Review of the Top 5 Differentiated Use Cases for th...
Denodo
 
Ad

More from PascalDesmarets1 (16)

Tutorial Workgroup - Working with Forks
Tutorial Workgroup - Working with Forks
PascalDesmarets1
 
Tutorial Advanced How-To - Oracle 23c Duality views
Tutorial Advanced How-To - Oracle 23c Duality views
PascalDesmarets1
 
Tutorial Getting Started part 4 - Domain-Driven Data Modeling
Tutorial Getting Started part 4 - Domain-Driven Data Modeling
PascalDesmarets1
 
Tutorial Expert How-To - Create a model for Avro schemas
Tutorial Expert How-To - Create a model for Avro schemas
PascalDesmarets1
 
Tutorial Expert How-To - Verify Data Model
Tutorial Expert How-To - Verify Data Model
PascalDesmarets1
 
Tutorial Expert How-To - Naming Conventions
Tutorial Expert How-To - Naming Conventions
PascalDesmarets1
 
Tutorial Expert How-To - Export-Import with Excel template
Tutorial Expert How-To - Export-Import with Excel template
PascalDesmarets1
 
Tutorial Expert How-To - Compare and Merge
Tutorial Expert How-To - Compare and Merge
PascalDesmarets1
 
Tutorial Expert How-To - Custom properties
Tutorial Expert How-To - Custom properties
PascalDesmarets1
 
Tutorial Expert How-To - Add reusable Definitions
Tutorial Expert How-To - Add reusable Definitions
PascalDesmarets1
 
Hackolade Tutorial - part 12 - Create a REST API model
Hackolade Tutorial - part 12 - Create a REST API model
PascalDesmarets1
 
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf
PascalDesmarets1
 
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
PascalDesmarets1
 
Hackolade Tutorial - part 4 - Create your first data model
Hackolade Tutorial - part 4 - Create your first data model
PascalDesmarets1
 
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
PascalDesmarets1
 
Hackolade Tutorial - part 2 - Overview of JSON and JSON schema
Hackolade Tutorial - part 2 - Overview of JSON and JSON schema
PascalDesmarets1
 
Tutorial Workgroup - Working with Forks
Tutorial Workgroup - Working with Forks
PascalDesmarets1
 
Tutorial Advanced How-To - Oracle 23c Duality views
Tutorial Advanced How-To - Oracle 23c Duality views
PascalDesmarets1
 
Tutorial Getting Started part 4 - Domain-Driven Data Modeling
Tutorial Getting Started part 4 - Domain-Driven Data Modeling
PascalDesmarets1
 
Tutorial Expert How-To - Create a model for Avro schemas
Tutorial Expert How-To - Create a model for Avro schemas
PascalDesmarets1
 
Tutorial Expert How-To - Verify Data Model
Tutorial Expert How-To - Verify Data Model
PascalDesmarets1
 
Tutorial Expert How-To - Naming Conventions
Tutorial Expert How-To - Naming Conventions
PascalDesmarets1
 
Tutorial Expert How-To - Export-Import with Excel template
Tutorial Expert How-To - Export-Import with Excel template
PascalDesmarets1
 
Tutorial Expert How-To - Compare and Merge
Tutorial Expert How-To - Compare and Merge
PascalDesmarets1
 
Tutorial Expert How-To - Custom properties
Tutorial Expert How-To - Custom properties
PascalDesmarets1
 
Tutorial Expert How-To - Add reusable Definitions
Tutorial Expert How-To - Add reusable Definitions
PascalDesmarets1
 
Hackolade Tutorial - part 12 - Create a REST API model
Hackolade Tutorial - part 12 - Create a REST API model
PascalDesmarets1
 
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf
PascalDesmarets1
 
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
PascalDesmarets1
 
Hackolade Tutorial - part 4 - Create your first data model
Hackolade Tutorial - part 4 - Create your first data model
PascalDesmarets1
 
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
PascalDesmarets1
 
Hackolade Tutorial - part 2 - Overview of JSON and JSON schema
Hackolade Tutorial - part 2 - Overview of JSON and JSON schema
PascalDesmarets1
 
Ad

Recently uploaded (20)

FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance
 
High Availability On-Premises FME Flow.pdf
High Availability On-Premises FME Flow.pdf
Safe Software
 
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Alliance
 
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
NTT DATA Technology & Innovation
 
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Alliance
 
Crypto Super 500 - 14th Report - June2025.pdf
Crypto Super 500 - 14th Report - June2025.pdf
Stephen Perrenod
 
FME for Good: Integrating Multiple Data Sources with APIs to Support Local Ch...
FME for Good: Integrating Multiple Data Sources with APIs to Support Local Ch...
Safe Software
 
SAP Modernization Strategies for a Successful S/4HANA Journey.pdf
SAP Modernization Strategies for a Successful S/4HANA Journey.pdf
Precisely
 
Artificial Intelligence in the Nonprofit Boardroom.pdf
Artificial Intelligence in the Nonprofit Boardroom.pdf
OnBoard
 
Down the Rabbit Hole – Solving 5 Training Roadblocks
Down the Rabbit Hole – Solving 5 Training Roadblocks
Rustici Software
 
Supporting the NextGen 911 Digital Transformation with FME
Supporting the NextGen 911 Digital Transformation with FME
Safe Software
 
Edge-banding-machines-edgeteq-s-200-en-.pdf
Edge-banding-machines-edgeteq-s-200-en-.pdf
AmirStern2
 
Murdledescargadarkweb.pdfvolumen1 100 elementary
Murdledescargadarkweb.pdfvolumen1 100 elementary
JorgeSemperteguiMont
 
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC
 
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
Muhammad Rizwan Akram
 
Viral>Wondershare Filmora 14.5.18.12900 Crack Free Download
Viral>Wondershare Filmora 14.5.18.12900 Crack Free Download
Puppy jhon
 
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
 
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
 
Providing an OGC API Processes REST Interface for FME Flow
Providing an OGC API Processes REST Interface for FME Flow
Safe Software
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 
FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance Seminar State of Passkeys.pptx
FIDO Alliance
 
High Availability On-Premises FME Flow.pdf
High Availability On-Premises FME Flow.pdf
Safe Software
 
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Seminar: Evolving Landscape of Post-Quantum Cryptography.pptx
FIDO Alliance
 
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
Can We Use Rust to Develop Extensions for PostgreSQL? (POSETTE: An Event for ...
NTT DATA Technology & Innovation
 
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Seminar: Perspectives on Passkeys & Consumer Adoption.pptx
FIDO Alliance
 
Crypto Super 500 - 14th Report - June2025.pdf
Crypto Super 500 - 14th Report - June2025.pdf
Stephen Perrenod
 
FME for Good: Integrating Multiple Data Sources with APIs to Support Local Ch...
FME for Good: Integrating Multiple Data Sources with APIs to Support Local Ch...
Safe Software
 
SAP Modernization Strategies for a Successful S/4HANA Journey.pdf
SAP Modernization Strategies for a Successful S/4HANA Journey.pdf
Precisely
 
Artificial Intelligence in the Nonprofit Boardroom.pdf
Artificial Intelligence in the Nonprofit Boardroom.pdf
OnBoard
 
Down the Rabbit Hole – Solving 5 Training Roadblocks
Down the Rabbit Hole – Solving 5 Training Roadblocks
Rustici Software
 
Supporting the NextGen 911 Digital Transformation with FME
Supporting the NextGen 911 Digital Transformation with FME
Safe Software
 
Edge-banding-machines-edgeteq-s-200-en-.pdf
Edge-banding-machines-edgeteq-s-200-en-.pdf
AmirStern2
 
Murdledescargadarkweb.pdfvolumen1 100 elementary
Murdledescargadarkweb.pdfvolumen1 100 elementary
JorgeSemperteguiMont
 
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC and Open Hackathons Monthly Highlights June 2025
OpenACC
 
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
ENERGY CONSUMPTION CALCULATION IN ENERGY-EFFICIENT AIR CONDITIONER.pdf
Muhammad Rizwan Akram
 
Viral>Wondershare Filmora 14.5.18.12900 Crack Free Download
Viral>Wondershare Filmora 14.5.18.12900 Crack Free Download
Puppy jhon
 
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik - Passionate Tech Enthusiast
Raman Bhaumik
 
AI VIDEO MAGAZINE - June 2025 - r/aivideo
AI VIDEO MAGAZINE - June 2025 - r/aivideo
1pcity Studios, Inc
 
Providing an OGC API Processes REST Interface for FME Flow
Providing an OGC API Processes REST Interface for FME Flow
Safe Software
 
MuleSoft for AgentForce : Topic Center and API Catalog
MuleSoft for AgentForce : Topic Center and API Catalog
shyamraj55
 

Tutorial Getting Started part 1 - Overview

  • 1. Hackolade Tutorial Tutorial – Vision & Getting Started Copyright © 2016-2023 Hackolade 1
  • 2. Hackolade Vision Copyright © 2016-2023 Hackolade 2
  • 3. At the highest level Hackolade wants to Reconcile Business and IT through a shared understanding of the context and meaning of data Copyright © 2016-2023 Hackolade 3
  • 4. Copyright © 2016-2023 Hackolade 4 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries , single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 5. Copyright © 2016-2023 Hackolade 5 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries , single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 6. Copyright © 2016-2023 Hackolade 6 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries, single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 7. Copyright © 2016-2023 Hackolade 7 Managing data complexity: different technologies, transformations, constant evolution (mis)Alignment of Business and IT: (mis)understanding of requirements, (mis)interpretation of data, (un)informed business decisions Design, document and maintain complex data backends in an Agile way Align business and IT through visual modeling and detailed documentation Data governance to increase quality and facilitate regulatory compliance Visual data modeling for all 21st century data formats and backend systems: SQL and NoSQL databases, Storage formats & IDLs, (REST)APIs, JSON Polyglot Data Modeling: enterprise-wide data model covering a variety of different targets Metadata-as-Code: Collaboration, versioning, branching, change tracking, traceability, peer reviews, conflict resolution, publication to business-facing data catalogs and dictionaries , single source- of-truth for business and technical stakeholders Better Software quality, faster delivery, lower Total Cost of Ownership Shared understanding of context and meaning of data, allowing you to make business sense of your data Compliance with rules and regulations Why? How? What? Results!
  • 8. Hackolade Overview Copyright © 2016-2023 Hackolade 8
  • 9. 1. Data modeling for the 21st century • Hackolade Studio provides developers, architects and data modelers with the core functionality that they need – without trying to offer all solutions to everyone • Hackolade Studio has been built from the ground up for the polyglot, agile world • Simplicity and productivity are core design principles that have been adopted from the start
  • 10. 2. Polyglot data modeling • One, comprehensive data model that covers both SQL/NoSQL backends, data pipelines, as well as streaming services and APIs • Full forward- and reverse-engineering capabilities, with automation and customizability built in
  • 12. 3. Enabling Agile with Metadata-as-Code • Providing the same level of governance over our schemas, data models and metadata as we do for our code, will require us to manage these artefacts in a similar way • Git (or its platform providers) are excellent repositories to provide control, versioning, collaboration, and automation • Using this, Hackolade orchestrates Metadata pipelines to keep in sync the technical data structures with business-facing data dictionaries • Provide a Single Source-of-Truth for both business and technical stakeholders
  • 14. What sets Hackolade Studio apart • Patented technology: unique way of representing database schema models for database systems and REST APIs • Runs not only on Windows, but also Mac and Linux • Simple to install and get started • Unique collection of target-specific plugins to enable reverse and forward engineering of data models for relational and non-relational data stores and storage formats • True technology-agnostic polyglot data modeling • Unique integration with Git repositories, enabling co-location of data models with other business and technical artefacts • Unique integration with data dictionaries (eg. Collibra) Copyright © 2016-2023 Hackolade 14
  • 15. Polyglot Data Modeling See article Copyright © 2016-2023 Hackolade 15
  • 16. Logical or Polyglot? Technology agnostic? Copyright © 2016-2023 Hackolade 16
  • 17. Data modeling for polyglot persistence One single application in the 21st century Copyright © 2016-2023 Hackolade 17
  • 18. Every organization has polyglot data pipelines Build pipelines so they don’t break when the schema evolves! Copyright © 2016-2023 Hackolade 18
  • 19. Polyglot data models • Sits over the previous boundary between logical and physical data models. • Polyglot data mode is a “Logical model on steroids”: • allows denormalization, if desired, given access patterns; • allows complex data types; • Represents a common physical model that generates schemas for a variety of technologies, with automatic mapping to the specific data types of the respective target technologies. • built so you could create a library of canonical objects for your domains, and use them consistently across physical data models for different target technologies. Copyright © 2016-2023 Hackolade 19
  • 20. Additional benefit of Polyglot Data Models A modular, interdependent, consistent data model that allows your enterprise to leverage the same building blocks across all of their different models Copyright © 2016-2023 Hackolade 20
  • 21. “Business user”-friendly representation of concepts A higher level representation of the entities and relationships in a model, by levering a Graph Diagram representation of the model Copyright © 2016-2023 Hackolade 21
  • 23. Metadata-as-Code: WHAT is it? • data models are co-located with application code thanks to a tight integration with Git repositories and DevOps CI/CD workflows. • This repository becomes the Single Source-of-Truth for all stakeholders • data models and their schema artefacts closely follow the lifecycle of application development and deployment to operations. • technical structures are published in data catalogs and kept in-sync to ensure a shared understanding of meaning and context with business users. Copyright © 2016-2023 Hackolade 23
  • 24. Metadata-as-Code answers key questions • Does everyone interpret a column name in a report the same way? • Does the column name represent exactly the nuances of what has been measured? • Does the tech side of the organization share the same understanding as the business side? • How about when applications evolve quickly and new columns are added at a rapid pace? • Is the metadata in the catalogs of the business kept up-to-date with the changes in data structures? Copyright © 2016-2023 Hackolade 24
  • 25. Data models Schema contracts Application code Git repository DevOps CI/CD pipeline Application Database Copyright © 2016-2023 Hackolade 25 Hackolade + Git: Co-locate metadata with code Environments: • Dev • Test • Integration • Production ALTER scripts
  • 26. Command line interface To automate conversions and operations Copyright © 2016-2023 Hackolade 26
  • 27. Publishing to business community of data citizens Orchestrate metadata pipelines to keep in sync technical data structures with business facing data dictionaries Copyright © 2016-2023 Hackolade 27
  • 29. Delivering Domain-Driven Data Modeling Copyright © 2016-2023 Hackolade 29
  • 30. Getting Started with Hackolade Studio Copyright © 2016-2023 Hackolade 30
  • 31. Three easy steps! Copyright © 2016-2023 Hackolade 31 Design high-performing data structures Design your tables, collections, and graphs, attributes and their data types, descriptions and constraints, using a visual representation of nested structures. Polyglot data modeling Optimize with dynamic schema evolution during agile cycles Link entities logically. Keep track of implicit relationships and denormalization. For each database or protocol technology, add the required metadata. Or reverse-engineer from dev or prod instances. Schema design features Publish your schemas and documentation Leverage GitOps to achieve metadata-as-code and synchronize technical schemas with business data dictionaries. Generate MongoDB validator, CQL, HQL, DDLs for relational, REST APIs, JSON Schema, Avro, Parquet, etc… plus human-readable documentation in HTML, Markdown, or PDF. Metadata-as-Code
  • 32. Hackolade Editions • Differences outlined on this page. • Professional Edition • no limit on the size of models, • produces documentation in different flexible formats. • Key features include • forward- and reverse-engineering functions tightly integrated with all different target technologies • a powerful Command-Line Interface • ability to compare different versions of your model, and optionally merge selected parts • naming conventions, library of reusable definitions, domain/subject area ERD views, inference of PK & FK relationships, generation of mock data for testing, model-driven API generation, model verification, denormalization of SQL schemas, … • Workgroup Edition • includes all of the features of the Professional edition, plus • a native integration with Git repositories for your data models: this enables versioning, branching, change tracking, collaboration, peer review, and other related capabilities. • more information in this section of our online documentation. Copyright © 2016-2023 Hackolade 32
  • 33. Installing Hackolade • Download the latest version of the software from our website • Support for Windows, Mac and Linux • Check the installation instructions: • For Windows • For Mac • For Linux • For Docker containers • Check the notes on Software registration • Different behaviour for Individual or Concurrent licenses: see article • Specific instructions for Network proxies • Specific instructions for Virtual Machines / VDI environments Copyright © 2016-2023 Hackolade 33
  • 34. Getting help • eLearning platform with self-paced online tutorials • Read the online manual • Watch our videos • Send an email ([email protected]) Copyright © 2016-2023 Hackolade 34
  • 35. Reading material • See Hackolade online documentation • The Hackolade Blog • This excellent new book: MongoDB Data Modeling & Schema Design • Many of the principles in the book are related to query driven modeling based on access patterns • Hackolade’s on social media: LinkedIn page, Twitter page • Download Hackolade Studio for free Copyright © 2016-2023 Hackolade 35
  • 36. Order on Amazon Copyright © 2016-2023 Hackolade 36 Pre-order