[Rasheed, 3(3): March, 2014] ISSN: 2277-9655

Impact Factor: 1.852

IJESRT
INTERNATIONAL JOURNAL OF ENGINEERING SCIENCES & RESEARCH
TECHNOLOGY
A Review on Data Warehouse Management
Umair Rasheed*1, M.Umer Sarwar2, Ramzan Talib3
*1,2,3
College of Computer Science & Information Studies, Government College University, Faisalabad,
Pakistan
[email protected]
Abstract
Data warehouse management is a crucial part of industry and business that have been adopted and put into
practice with increased complexity of managing pools of data. The architectural layout of data warehousing is
composed of a centralized warehouse from which views are generated for needy users. The convention in this case is
that the warehouse receives an information injection from individual distributed databases which may as well be not
related to the warehouse but hold the raw data needed to justify the existence of warehouse. However, data
warehouse management is a relatively complex procedure that needs data updates from distributed databases. The
complexity of this procedure relates to anomaly-prone informational transaction between the data source end and the
materialized view end. This description supports the critical requirements of data warehouses which are accuracy
and timeliness. This review dwells on the assessment of data warehouse literature to verify the dominant strategy
employed in data warehouse management and the trade-offs involved. For this purpose various data warehouse
management related articles were selected and carefully reviewed. The findings revealed a favor towards Immediate
Incremental Management (IIM) of warehouse against Deferred Incremental Management (DIM). It is observed that
system availability which constitutes accurate views and real-time updating was the base determining factor.
Deferred Incremental Management has been the conventional way but with an increase in data volume that requires
a gigabyte transaction rate that defines contemporary business and industry this updating mechanism is quite
unsuitable and strips data warehouse management its contextual meaning.

Keywords: Immediate Incremental Management (IIM), Deferred Incremental Management (DIM), Algorithm,
Informational transaction.

Introduction
Data warehousing has been prevalent in Data warehouse is particularly a repository
contemporary business and industry and be described of integrated information that has been gathered from
as the de facto informatics of the time. It has been a distributed sources often databases [2, 3, 4, 5]. The
prerequisite for conducting information-oriented information is bundled up by dedicated hardware and
business and various industrial functions. Intelligence software systems that subject the information (data)
informatics has been applied in these two to an Extraction-Transfer-Load schema by
environments by use of data warehousing to create a implementing algorithms and definite schemas
discernible culmination of events and generate optimized for the process [6]. This enables logical
logically understandable summary information representation of juggled machine-oriented data
critical for decision-making [1]. The term “data within the warehouse system to relevant viewers.
warehouse” traces root back to the 1980s and has Since it is vital for data warehouse management to
seen adaptation and structural evolution since then consider timing and accuracy of materialized views
[2]. The terms carried a general meaning of the and it is imperative to assess the optimum mechanism
nature of basic system behind the process and the of upholding user-expected time while accessing
purpose of decision making it supported. Recent warehoused information in the user-expected
advancement however has given data warehousing a accuracy. This review assumes Immediate
comprehensive meaning solely related to time and the Incremental Management as the optimal strategy in
revolutionary changes involving information-flow achieving accuracy and timing in fast and reliable
protocol and transaction guidelines. data warehouse management. It also assumes
concurrent exchanges between the source and the

http: // www.ijesrt.com(C)International Journal of Engineering Sciences & Research Technology

[1247-1251]
[Rasheed, 3(3): March, 2014] ISSN: 2277-9655
Impact Factor: 1.852
viewers is optimally transacted on the fly rather than transactions employ a handshake protocol between
apportion a downtime against warehouse access the specified modules to realize a proper data
which locks out non-admin access. placement with the right confirmation notification. A
typified model suggested by Warehousing
Materials and Methods Information Project at Stanford (WHIPS) outlays the
This paper is based on the factual extracted modular architecture that characterizes conventional
information from randomly selected journal articles data warehousing [9]. The model composes a source
in the field of Information Systems. The articles were module (the various distributed databases from which
initially subjected to abstract analysis to ascertain information originates), monitor wrapper module
their relevance in information gathering. Twenty-two (tasked with monitoring changes at the source level
articles were positively identified fit for the study and and relaying notice to the integrator by use of trigger-
thoroughly read wholly to further ascertain their based technique by use of system query language),
credibility for the review. All articles were subjected integrator module (receives source-related updates
to a text search to verify the inclusion of keywords and forwards the to view managers), view manager
and phrases that characterize the review and module (every viewer is assigned a manager and
stipulated hypotheses. Out of the 22 three were utilizes a strobe algorithm to guarantee consistency),
particularly shallower than the rest 19 due to the a warehouse wrapper (translate information to a
presence of an abstract and an introduction without warehouse discernible signal), warehouse module
detailed figurative data. They were however included and a warehouse application module (the interfacing
for inclusion of critically needed facts in the infrastructure accessible to information seekers). At
introduction and their abstract tendency to positively users disposal are data sources, system logs and
engage the issue of data warehouse management. The warehouses involved [9]. According to [10] there are
review however took particular interest in ten of the three types of implementation architectures for a data
articles due to direct relationship of researched warehousing, which are single-layer, two-layer and
information and the review’s hypotheses. three-layer having their own strengths and
Calculations included in these journal articles were weaknesses.
carefully rerun to guarantee information accuracy and Data flow
consistency throughout the paper. Figurative data was The consistent and most defining factor of
verified to be authentic and the review delved into the data warehouse management is the organization of
articles to collect needed information to challenge the data. A simplified explanation towards the concept of
hypotheses. data warehouse management is proposed in [10]
which imply complete and consistent store of data
Result collected from a variety of sources and made
Modernized data access is run from systems available to needing users. The major consideration
that implement fast and parallel information in this fact is the neediness of accessing data. The
transactions to give user a real-time experience with infrastructural working in the warehouse ensures this
information [7]. Systems have been evolved to needed data is captured, processed and relayed to the
embrace a parallel communication system between user in a logical, understandable way free from errors
data source end and material viewer end. Immediate and at the right time. Data movement in the
Incremental Management (IMM) occasionally warehouse is an important and indispensable aspect
referred to Immediate Incremental View Maintenance in this review and understanding the flow of data
(IIVM) is the current most relevant mechanism to from the source through the infrastructure is a vital
implement in data warehouse management besides step to discerning the results and what they imply.
from scratch re-computing. IMM/IIVM is preferred Data obtainable from distributed databases (often
for its resilience and better handling of anomalies involving a particular industry) is transferable to the
related to online-analysis process (OLAP) queries. A user through three main components the source
majority of the reviewed literature which further (which contains needed data in raw form), the
suggests different algorithms to realize the full warehouse infrastructure (which processes the data)
potential of the technique has affirmed this result. and the access application (the user-warehouse
interface). This generalized method is referred to as
Discussion Extraction, Transfer and Load (ETL) according to
Architecture process [6]. Extraction is reading the data from the
Data warehouse management involves a source in its raw form and feeding it. Reading is
transactional technique to communicate between the executed at the monitor wrapper interface obtains
different infrastructural modules involved [8]. The data deemed updates (augmentation), deletion or
subtraction and the reading command picks the
http: // www.ijesrt.com(C)International Journal of Engineering Sciences & Research Technology
[1247-1251]
[Rasheed, 3(3): March, 2014] ISSN: 2277-9655
Impact Factor: 1.852
necessary query trigger to make the opted changes concurrencies gone wrong SDCC locates the query
(SQL). Transfer involves the infusion of the data gathers information about the faulty-query result and
from the source into the warehouse infrastructure the maintenance-concurrent updates. LC algorithm
where organizational commands are initiated to solves the problem through a four-leveled correction.
adjust materialized view accordingly before First procedure is creation of a temporary relational
availability to the needing user through the Load table. Second procedure is the creation of one local-
procedure. compensation query. Third procedure is ascertaining
Algorithmic implementation the delta (change) in query. Fourth procedure is
Concurrency and parallel exchange in obtaining the value transition by subtracting the
schematic flow of a data warehouse system has been query delta from the created local-compensation
an issue of concern that has played a big setback in query derived in the second procedure [11].
maximizing the potential of many data warehouses Analysis
observed in a programming model exemplified by Data execution in the warehousing
[11]. The abstracted rapid exchange of information environment is prone to errors in the contemporary
between two database sources while the warehousing business and industry sector which are prevalent
system is still making necessary changes to previous customers that use this utility. Technological
summary logs is prone to inconsistencies and errors advancement has raised the bar of data transmission
in final tables and unfortunately manifest to users. to terabytes per second. Online platforms as well
This calls for yet another wasted minute to make have been submerged into using data warehousing
necessary changes and maybe by then the management to facilitate easy access of information
inconsistencies and errors would have already taken [14]. Since the globalization and worldwide access to
effect on users thought the dynamic materialized information has been the norm of the time and data
views. This sort of system is highly unacceptable and warehouse downtime that has been characteristic of
is non-adaptive to present day systems that are the dawn-of-connectivity warehouses has ceased to
modeled to operate under an up-to-date sentiment in be efficient and have been deemed inefficient and
influencing just-in-time decisions [5]. rather pose as impediments than utilities [6].
To counter these system flaws certain Downtime is the deactivation of user-access of the
software structures are needed to implement data warehouse as a management effort to make
sustainable and dependable system behavior. necessary adjustments like cache refreshments and
Conventional procedures that help shield these re-computing tables. Downtime however as much as
transactional inconsistencies and anomalies have it is planned and executed in the best interest of the
been put across and include SWEEP for source data utility holds negative consequences of locking users
verification, View Adaptation (VA) to monitoring out of the system without much notice. Personalized
view changes and Evolvable View Environment contact with information therefore is not guaranteed
(EVE) for schema change monitoring at the on a full time basis. Even though warehouse
information source. These integrative procedures managers claim the process to execute this duty when
invoke relevant commands within the whole system user access is virtually offline but some disagree with
to take necessary action when their reading changes. this fact [15].
It is done on a change-trigger basis to outdo timing Global accessible data warehouses that are
and accuracy. These three micro-algorithms are dominant in internet applications and form
infusible into a comprehensive algorithmic structure characterized websites need cannot whatsoever use
referred to as Schema change and Data update Deferred Incremental Management data warehouses
Concurrency Control system dubbed SDCC. This all- due to constant global access. Data warehouses
inclusive algorithm is implemented with accuracy useable in this instance therefore employ an
and timing in mind factors which are stressed by the Immediate Incremental Management system that
Immediate Incremental Management of data properly suits this environment. Globally accessible
warehouses. SDCC employs Local Compensation data warehouses cannot be justified to have a
(LC) that verifies every information node before possible downtime since a part of the world is
being translated into the next to counter the problem accessing the data. These interchangeable patterns
of error spillage to other non-affected system keep revolving and therefore need an increment on
modules [11]. This improvised LC quickly corrects the fly [16].
faulty queries before they enter the mainstream of Suggested and verified in this course is an
data transfer. Immediate Incremental Management system that
The LC operates under SDCC command and makes progressive updates with dedicated
is invoked and “killed” by it. In the event of faulty algorithms. IIM/IIVM is the present day implemented
query detection normally resulting from system since most data warehouses need one way or
http: // www.ijesrt.com(C)International Journal of Engineering Sciences & Research Technology
[1247-1251]
[Rasheed, 3(3): March, 2014] ISSN: 2277-9655
Impact Factor: 1.852
another a global connection. Information definitely not need DIM and the reason behind this is
collaboration has also pushed this need for IIM/IIVM the effect of downtime [21].
to carry out long stretch multinational projects. With DIM cannot be tested against IIM when time
globalization for instance and the need to abate and efficiency is a factor. IIM related algorithms as
climate deterioration worldwide information well form the optimum procedure in countering
collection in a centralized data warehouse is much errors generated during concurrent information
needed than before to enhance the likelihood of transaction between the user side accessing the data
succeeded in such a course [17]. warehouse material views and the information
sources/databases interfacing with the system to feed
Conclusion data. Accuracy and timeliness are the discriminating
Data warehouse management is a parameters that characterize data warehouse
contemporary issue that has defined intelligence and management. However, downtime-prone
advancement in the field of business industry. warehousing management system may be used in
Technological advancement has seen more non-time conscious data management environment
researchers, analysts, business and industry personnel that does not need constant access [22].
need information to demarcate trends to plan the
future and make localized decisions. However not References
only these but the common persons have also gained [1]. Elamy, A.H.; Alhajj, R.S.; Far, B.H.,
interest in data. This has generated much traffic that "Building data warehouses with incremental
has led to infrastructural hitches. On the other hand maintenance for decision support,"
the need has arisen to increase the availability of data Electrical and Computer Engineering, 2005.
warehouses to the benefit of those that need its Canadian Conference on, pp.1809-1814,
services. IIM has been selected as the best possible 2005.
way of algorithmic implementation with little [2]. Saeki, S.; Bhalla, S.; Hasegawa, M.,
compromise to either the management team or the "Parallel generation of base relation
users. Batch processing implemented in Deferred snapshots for materialized view
Incremental Management was a likely option but maintenance in data warehouse
quite disadvantageous and irrelevant to contemporary environment," Parallel Processing
times. IIM on the other hand was observed to pose a Workshops, 2002. Proceedings.
better chance of making fast corrective measures. International Conference on, pp.383-390,
SDCC local sub-querying system for example 2002.
demonstrated this principle of accurate correction
which can run simultaneously in an (n) number of
[3]. Zhang, X.; Rundensteiner, E.A., "Data
warehouse maintenance under concurrent
Information Sources. The “n” in this case
schema and data updates," Data
representing the possible number of distributed
Engineering, 1999. Proceedings., 15th
databases that communicate information to the
International Conference on, pp.253, 1999.
centralized data warehouse [20].
Different algorithms have been suggested [4]. Lee, J.W.T.; Xiang Ye, "Materialized view
however but with no in-depth analysis. This review design and maintenance in a financial data
paper took to the bottom of Schema change and Data warehouse system," Systems, Man, and
update Concurrency Control system (SDCC) to Cybernetics, 1999. IEEE SMC '99
explain a portion of the comprehensive operation of a Conference Proceedings. 1999 IEEE
sub-level algorithm. Various algorithmic expressions International Conference on, vol.5, pp.930-
however are implementable within the same 935, 1999.
functionality of LC. DIM employs a different [5]. Ismail, R.M., "Maintenance of materialized
strategy different from IIM. Operations regarding the views over peer-to-peer data warehouse
system for instance critical augmentations, architecture," Computer Engineering &
subtractions, deletions and subsequent translation to Systems (ICCES), 2011 International
summary logs are not that possible on a real time Conference on, pp.312-318, 2011.
basis which is quite a setback to a time conscious [6]. Wei Xu; Maoqing Li; Shunxiang Wu;
business and industry environment that is dominated Shunzhi Zhu; Zhoujing Wang; Kehua Miao;
by fast updating and just-in-time decisions. Internet Ying Wang, "Incremental data feed
enabled data warehousing environments which maintenance of a data warehouse system
dominate the contemporary society either will derived from multiple autonomous data
sources," Control and Automation, 2005.

http: // www.ijesrt.com(C)International Journal of Engineering Sciences & Research Technology

[1247-1251]
[Rasheed, 3(3): March, 2014] ISSN: 2277-9655
Impact Factor: 1.852
ICCA '05. International Conference on, Warehouse Maintenance," Advanced
vol.2, pp.1108-1113, 2005. Computer Control, 2009. ICACC '09.
[7]. Yeu Shiang Huang; Do Duy; Chih Chiang International Conference on, pp.623-627,
Fang, “Efficient maintenance of basic 2009.
statistical functions in data warehouses,” [17]. Xin Zhang; Elke A. Rundensteiner,
Decision Support Systems, vol.57, pp.94- “Integrating the maintenance and
104, 2014. synchronization of data warehouses using a
[8]. Chao C, “Incremental maintenance of cooperative framework,” Information
object-oriented data warehouses,” Systems, vol. 27, no.4, pp. 219-243, 2002.
Information Sciences, vol.160, no.1-4, [18]. Yeung, G.C.H.; Gruver, W.A., "Multiagent
pp.91-110, 2004. immediate incremental view maintenance
[9]. Wiener, J.L.; Gupta, H.; Labio, W.J.; Yue for data warehouses," Systems, Man and
Zhuge; Garcia-Molina, H.; Widom, J., "The Cybernetics, Part A: Systems and Humans,
WHIPS prototype for data warehouse IEEE Transactions on, vol.35, no.2, pp.305-
creation and maintenance," Data 310, 2005
Engineering, 1997. Proceedings. 13th [19]. Yi-Yang Yang; Yain-Whar Si; Wai-Leong
International Conference on, pp.589, 1997. Leong, "Data warehousing massive real-
[10]. Chan, A.; Crane, G.; MacGregor, I.; Meyer, time elevator signals and maintenance
S., "Data warehouse on the Web for records," Industrial Technology, 2008. ICIT
accelerator fabrication and 2008. IEEE International Conference on,
maintenance," Particle Accelerator pp.1-8, 2008.
Conference, 1997. Proceedings of the 1997, [20]. Xiaogang Zhang; Luming Yang; De Wang,
vol.2, pp.2413-2415, 1997. "Incremental view maintenance based on
[11]. Zhang, X.; Rundensteiner, E.A., "The SDCC data source compensation in data
framework for integrating existing warehouses," Computer Application and
algorithms for diverse data warehouse System Modeling (ICCASM), 2010
maintenance tasks," Database Engineering International Conference on, vol.2, pp.287-
and Applications, 1999. IDEAS '99. 291, 2010.
International Symposium Proceedings, [21]. Yow, T.G.; Grubb, J.W.; Jennings, S.V.,
pp.206-214, 1999. "Managing data warehouse metadata using
[12]. de Amo, S.; Alves, M.H.F., "Efficient the Web: a Web-based DBA maintenance
maintenance of temporal data tool suite," System Sciences, 1998.,
warehouses," Database Engineering and Proceedings of the Thirty-First Hawaii
Applications Symposium, 2000 International Conference on, vol.6, pp.49-
International , pp.188-196, 2000. 54, 1998.
[13]. Buren, G.; Ruckert, C., "Architectural [22]. Lijuan Zhou; Qian Shi; Haijun Geng, "The
Maintenance Using a Data Warehouse minimum incremental maintenance of
System for Availability Analysis," Software materialized views in data
Maintenance and Reengineering, 2009. warehouse," Informatics in Control,
CSMR '09. 13th European Conference on, Automation and Robotics (CAR), 2010 2nd
pp.307-308, 2009. International Asia Conference on, vol.3,
[14]. Kulkarni, S.; Mohania, M., "Concurrent pp.220-223, 2010.
maintenance of views using multiple
versions," Database Engineering and
Applications, 1999. IDEAS '99.
International Symposium Proceedings,
pp.254-258, 1999.
[15]. Ram, P.; Do, L., "Extracting delta for
incremental data warehouse
maintenance," Data Engineering, 2000.
Proceedings. 16th International Conference
on, pp.220-229, 2000
[16]. Reddy, S.S.S.; Lavanya, A.; Khanna, V.;
Reddy, L. S S, "Research Issues on Data
http: // www.ijesrt.com(C)International Journal of Engineering Sciences & Research Technology
[1247-1251]