Layout Data Architecture
Layout Data Architecture
TOPICAL SURVEY
Foreword...................................................................................................................................................................................... 3
Management summary.............................................................................................................................................................. 4
Demographics........................................................................................................................................................................... 17
BARC profile.............................................................................................................................................................................. 19
Sponsor profiles...................................................................................................................................................................... 20
Authors...................................................................................................................................................................................... 24
Data architecture is a topic that is as relevant today as ever. It is widely their detail. However, is this best practice, which has basically lasted for
regarded as a matter for data engineers, not business domain experts. three decades, still suitable for meeting the challenges of the increasingly
But is it really? Statements from countless interviews with our customers distributed data landscapes of today’s digital enterprises?
reveal that the data warehouse is seen as a “black box” by many and
understood by few business users. Therefore, it is not clear why the costly Our survey results show that the opinions of different roles in the
and apparently flexibility-inhibiting data warehouse is needed at all. We organization differ considerably in many cases. It is primarily the data &
even know companies that are cutting back their investments in a “single analytics experts in the central teams who adhere to the central approach.
point of truth” for these reasons. They see it as proven and successful.
Participants around the world were surveyed, with most responses Among executives and managers, as well as process and data & analytics
coming from Europe. We examined the current architecture approaches experts in business units, the picture is often different. The question arises:
of companies of different sizes from various industries. Participants were Should future data architecture be determined more centrally from the
asked to rate the skills and competencies in the handling of data in their perspective of data engineers or more decentrally from the perspective of
company compared to their main competitors. This allowed us to gain an domain experts? Or is a federated approach the best way forward?
understanding of how “best-in-class” companies are shaping their data
architecture in comparison to “laggards” (see demographics). We hope that our survey results will provide you with valuable insights and
help you find the right solution for the future design of your company’s
In general, central data & analytics teams determine the data architecture data landscape. We welcome your feedback at any time.
for analytical data, decoupled from the landscape of operational data
sources. Our survey shows that they continue to pursue clearly centralized
architecture concepts. Indeed, this is what the data warehouse, data lake Jacqueline Bloemen and Timm Grosser
and data lakehouse have in common, regardless of the differences in Würzburg, August 2022
MANAGEMENT SUMMARY
Companies are aware that organizational measures are needed above all to Establishing a data culture is a long-term process. It requires a change
further develop their data culture. But in practice, only a minority is implementing in mindset and a willingness to look outside the box. You need to bring
such measures. Initiatives that support self-service analytics are the most data producers and consumers together to enable the exchange of
widespread. This is good, but not enough. Strengthening cross-domain data perspectives and objectives. Technology such as a data catalog can
collaboration requires both harmonized master and reference data, and an effectively support data collaboration. But it is ultimately the business
understandable and shared view of data regardless of system silos. users who, through their contributions, bring the technology to life. It
This can only be achieved if business process owners and experts, the data takes cross-domain collaboration to find the right measures to improve
producers, are actively involved. the data landscape.
Self-service analytics has been a trend for some introducing analytics tools suitable for business While these centralized data repositories seem
years, and this survey underpins its importance. users or plan to do so. Strikingly, only 24 percent to work well at first glance, almost one in three
The fact that business users cannot implement actually find that existing tools are too technical. respondents finds that data warehouses and
new requirements themselves is regarded as the Even though line-of-business data & analytics data lakes do not cover their requirements.
top challenge data users are facing (48 percent). experts in particular agree with this point (35 Dissatisfaction is even greater among business
At 53 percent, this opinion is particularly strong percent), all other persona groups concur to a managers and process experts. 43 percent of
within central data & analytics teams. In line lesser degree. So, the tool issue does not seem to these personas believe that the data warehouse/
with this finding, 74 percent are either already be all that important. data lake does not meet their requirements.
How relevant are the following business-related/organizational measures for your company? (n=253)
Which of these approaches to modernizing your data landscape are planned in your company? (n=260)
Data lakehouse 40%
The identified shortcomings in established data & Improving source data and redesigning the 28%
analytics landscapes as well as the limited ability data architecture are also popular measures. 36%
Document modeling 27%
to respond to new requirements show a need Interestingly, the majority still relies on the
for action. Companies are therefore planning to data warehouse as a design paradigm. Among 31%
Data vault modeling
modernize their landscapes. The most common laggards, the figure is as high as 64 percent. The 15%
approach is to optimize existing data models, popularity of the approaches overall indicates that
Best-in-class (BIC)
data processes and queries. Among best-in-class the redesign is largely taking place with tried-and- All
companies, as many as 60 percent have chosen tested concepts including dimensional modeling
this approach. Not surprisingly, the second most and frequently adding a data lake. Which data architecture and data modeling aproaches
will be important for your future data & analytics
popular modernization measure is migration to
landscape? (all: n=260; BIC: n=78)
the cloud.
But there are good reasons to question the Open stream processing solutions 36%
central concept. While only 31 percent of experts
in central teams are convinced that a centralized Distributed data pipelining & orchestration 29%
data architecture cannot prevent data silos, the
Cloud-provider-specific services 29%
corresponding figure is 43 percent at executive
level and 40 percent within IT. At the same time, Improved data sources and / or their interfaces 29%
data users in the business (42 percent) an are Best-in-class (BIC)
particularly aware of the fact that distributed data
is not easily linkable. Which concepts and technologies will be important for your future data & analytics landscape? By BIC (n=79)
To enable cross-divisional data collaboration, prerequisite for both, with the proportion in the
the data must be understandable. 93 percent central data teams being somewhat higher (23
92% find it relevant to provide understandable percent). This may also be the reason why only
On average documentation and navigation assistance for 28 percent plan to introduce a data catalog and/
of respondents believe business-related/ data. Especially business managers and process or metadata solution. Interestingly, in companies
organizational measures are relevant to experts (55 percent) demand making all relevant where tools are too technical, business users
improve data & analytics utilization. data sources searchable and accessible. cannot implement new requirements themselves
or IT cannot react quickly enough to new
Only a few (17 percent) are aware that an requirements, this measure is taken much more
overarching semantic data layer is actually a frequently (by 44, 53 and 58 percent respectively).
Other Other
REGION 4% 2%
Europe What is your role in the company? (n=260) Which of the following best describes your
52% organization’s industry sector? (n=260)
North America COMPANY SIZE
35%
Asia and Pacific Less than 500
8% 30%
South America 500 - 4,999
3% 39%
Africa 5,000 or more
2% 31%
In which region are you located? (n=260) How many employees does your company have?
(n=260)
BARC (Business Application Research Center) is Our independent research brings market
one of Europe’s leading analyst firms for business developments into clear focus, puts software and
software, focusing on the areas of data, business vendors through their paces and gives users a
intelligence (BI) and analytics, enterprise content place to express their opinions. BARC GmbH
Berliner Platz 7
management (ECM), customer relationship
Events D-97080 Würzburg
management (CRM) and enterprise resource
Decision-makers and IT industry leaders come Germany
planning (ERP). +49 931 880 6510
together at BARC events. BARC seminars in small
Our passion is to help organizations become digital www.barc.de
groups, online webinars and conferences with
companies of tomorrow. We do this by using
more than 1,000 participants annually all offer Hirschstettner Straße 19 / I / IS314
technology to rethink the world, trusting data-
inspiration and interactivity. Through exchange A-1200 Wien
based decisions and optimizing and digitalizing
with peers and an overview of current trends Austria
processes. It‘s about finding the right tools and +43 660 6366870
and market developments, you will receive new
using them in a way that gives your company the
impetus to drive your business forward.
best possible advantage. BARC Schweiz GmbH
This unique blend of knowledge, exchange of Consulting Täfernstraße 22a
CH-5405 Baden-Dättwil
information and independence distinguishes In confidential expert workshops, coaching and
Switzerland
our services in the areas of research, events and in-house consultations, we transform the needs +41 56 470 94 34
consulting. of your company into future-proof decisions. We
provide you with successful, holistic concepts that Rest of the World
Research
enable you to use the right information correctly. +44 1536 772 451
Our BARC studies are based on internal market Our project support covers all stages of the www.barc-research.com
research, software tests and analyst comments, successful use of software.
giving you the security to make the right decisions.
ABOUT DATABRICKS
ABOUT DENODO
Denodo is a leader in data management. The than 400% ROI and millions of dollars in benefits,
award-winning Denodo Platform is the leading data Denodo’s customers across large enterprises and
integration, management, and delivery platform mid-market companies in 30+ industries have
using a logical approach to enable self-service BI, received payback in less than 6 months. For more
Denodo Technologies GmbH
data science, hybrid/multi-cloud data integration, information, visit www.denodo.com Oberanger 28
and enterprise data services. Realizing more Munich, 80331
Germany
+49 89 599 904 50
www.denodo.com
[email protected]
UK
+44 20 3196 4710
Spain
+34 912 77 58 55
France
+33 1 42 68 51 27
Italy
+39 2 72546395
Precisely is the global leader in data integrity, The Precisely Data Integrity Suite is a set of seven
providing accuracy, consistency, and context interoperable modules that enable your business
in data for 12,000 customers in more than 100 to build trust in its data. Data with integrity has
countries, including 99 of the Fortune 100. maximum accuracy, consistency, and context – Precisely Software GmbH
Precisely’s data integration, data quality, data empowering fast, confident decisions that help Ungelsheimer Weg 14
governance, location intelligence, and data you add, grow, and retain customers, move 40472 Düsseldorf
enrichment products power better business quickly and reduce costs, and manage risk and Germany
www.precisely.com/de
decisions to create better outcomes. compliance.
• The Forrester Wave™: Data Governance
Contact
Solutions At their heart, the modules of the Data Integrity
Kerstin Krämer
Precisely (Infogix) is named a Leader in The Suite are built on proven Precisely technology that Marketing Campaigns Manager
Forrester Wave™: Data Governance Solutions, over 12,000 businesses across the globe rely on to [email protected]
Q3 2021 meet their data integrity needs. The interoperable
• IDC MarketScape: Data Catalog Software modules work together to build value at each step
2022 Vendor Assessment of your journey to data integrity, integrate with
Precisely Named a Leader for Data Catalog Precisely’s portfolio of market-leading products,
Software and work seamlessly with both traditional and
• The Forrester Wave™: Location Intelligence modern tech stacks.
Platforms
Precisely is named a Leader in The Forrester The modular, interoperable Precisely Data
Wave: Location Intelligence Platforms, Q2 Integrity Suite contains everything you need to
2020 deliver accurate, consistent, contextual data
to your business - wherever and whenever it’s
Learn more at needed.
www.precisely.com
Jacqueline Bloemen is a senior analyst with a As a senior analyst, Timm Grosser has been
major focus on data & analytics strategy and advising domestic and international companies
culture, architecture and technology, governance of various sizes and industries in the areas of
and organization. She is an author and speaker BI, data management and analytics for more
and has been advising companies of various than 10 years. During his time as a consultant,
sizes and industries for over 35 years. Currently, he has designed numerous solutions in BI/big
her research and consulting activities focus on data strategy, organization, architecture and
the transformation to becoming a data-driven tool selection with customers and in the BARC
company. test lab. He is a frequent speaker at conferences
and seminars as well as the author of numerous
industry articles and market studies.