See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/321586026

Efficient MapReduce Matrix Multiplication with Optimized Mapper Set

Chapter  in  Advances in Intelligent Systems and Computing · April 2017

DOI: 10.1007/978-3-319-57264-2_19

CITATIONS READS

3 789

4 authors, including:

Azzam Sleit Mais Ahmad Haj Qasem

University of Jordan University of Jordan
139 PUBLICATIONS   591 CITATIONS    14 PUBLICATIONS   22 CITATIONS   

SEE PROFILE SEE PROFILE

Ahmad Abdel-Aziz Sharieh

University of Jordan
78 PUBLICATIONS   221 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The Exhaustion of Agriculture Lands by Construction in Amman View project

Data Visualization View project

All content following this page was uploaded by Azzam Sleit on 17 December 2017.

The user has requested enhancement of the downloaded file.

 Efficient MapReduce Matrix Multiplication
with Optimized Mapper Set

Methaq Kadhum, Mais Haj Qasem ✉ , Azzam Sleit, and Ahamd Sharieh
( )

King Abdullah II School for Information Technology, Computer Science Department,

University of Jordan, Amman, Jordan
[email protected], [email protected],
{azzam.sleit,sharieh}@ju.edu.jo

Abstract. The efficiency of matrix multiplication is a popular research topic

given that matrices compromise large data in computer applications and other
fields of study. The proposed schemes utilize data blocks to balance processing
overhead results from a small mapper set and I/O overhead results from a large
mapper set. Balancing between the two processing steps, however, consumes time
and resources. The proposed technique uses a single MapReduce job and pre-
processing step. The pre-processing step reads an element from the first array and
a block from the second array prior to merging both elements into one file. The
map task performs the multiplication operations, whereas the reduce task
performs the sum operations. Comparing the proposed and existing schemes
reveals that the proposed schemes more efficiently consume time and memory.

Keywords: Hadoop · MapReduce · Matrix multiplication · Optimized mapper

set

1 Introduction

Matrix multiplication is a fundamental operation in linear algebra with related real-life

applications, such as matrix factorization, chemical system formulation, and graph anal‐
ysis [14]. In addition to its naturally related applications, several problems are reducible
by matrix multiplication. Thus, these problems should be investigated thoroughly to
enhance the efficiency of implemented algorithms for matrix multiplication. Given the
inputs of two matrices A and B, where the number of columns in A equals the number
of rows in B, matrix multiplication produces matrix C with the number of rows equal
to that in A and number of columns equal to that in B. The Brute-Force matrix multi‐
plication algorithm for square matrices is given in Algorithm 1. The Brute-Force algo‐
rithm has a high processing complexity, of O(n3), but suffers from the massive memory
lookup process required to locate each array element for multiplication. Over the years,
several matrix multiplication algorithms have been proposed to reduce the cost and time
of the matrix multiplication process [2, 15].

© Springer International Publishing AG 2017

R. Silhavy et al. (eds.), Cybernetics and Mathematics Applications in Intelligent Systems,
Advances in Intelligent Systems and Computing 574, DOI 10.1007/978-3-319-57264-2_19
 Efficient MapReduce Matrix Multiplication 187

MapReduce is an algorithm design and processing paradigm that was proposed by

Dean and Ghemawat in 2004 [4]. MapReduce enables efficient parallel and distributed
computing and consists of two serial tasks, Map and Reduce. Each serial task is imple‐
mented with several parallel sub-tasks. Map task, the first task in MapReduce, accepts
input for conversion into a different form as the output. In map task, both input and
output data are formed using a series of elements with individual key-value pairs. The
reduce task takes the map output and implements an aggregation process for pairs with
identical keys. Although composing or implementing the algorithm in the map and
reduce tasks for execution in MapReduce are nothing but trivial, the gain of such a
decomposition process is massive. Thus, the program can be run over hundreds and
thousands of parallel nodes over a cluster of machines [5].
Hadoop is a Java open-source platform used for developing MapReduce applica‐
tions. This platform was developed by Google [6]. The Hadoop architecture is illustrated
in Fig. 1.

Fig. 1. Hadoop MapReduce architecture

188 M. Kadhum et al.

As shown in Fig. 1, the Hadoop framework is responsible for distributing the input
into the involved mappers. These mappers implement the map task, collect the results
for sorting during the shuffle process, and feed and collect the output of the reducers.
The MapReduce paradigm has been used to decompose enormous tasks, such as
data-mining algorithms. Specific MapReduce paradigms include: MapReduce with
expectation maximization for text filtering [11], MapReduce with K-means for remote-
sensing image clustering [12], and MapReduce with decision tree for classification [22].
Additionally, MapReduce has been used for job scheduling [23] and real-time
systems [11].
Matrix multiplication that uses MapReduce has been proposed [9]. The earlier
decomposition process of matrix multiplication involved two MapReduce tasks.
However, problems arose with these decomposition processes: the processing overhead
and file I/O overhead were obvious. Hence, it was necessary to re-decompose the matrix
multiplication process in the MapReduce paradigm to enhance and decrease the
computing overhead.
This paper proposes a technique for matrix multiplication. The technique uses a
single MapReduce task with an optimized mapper set. The optimal number of mappers
that formed the utilized mapper set is selected to balance the processing overhead results
of a small mapper set and the I/O overhead results of a large mapper set. These two
processes consume time and resources.
The rest of the paper is organized as follows: Sect. 2 reviews work that is closely
related to the implemented MapReduce matrix multiplication task. Section 3 presents
the proposed work for matrix multiplication and highlights the relation between
proposed and previous techniques. Section 4 presents the experimental results. Finally,
the conclusion is given in Sect. 5.

2 Related Work

The traditional sequential algorithms for matrix multiplication consume considerable

space and time. To enhance the efficiency of matrix multiplication, the Fox [1], Cannon
[14], and DNS [7] algorithms have been proposed to parallelize the matrix multiplication
process. To maximize efficiency, these approaches balance inter-process communica‐
tion, dependencies, and parallelism level. Parallel matrix multiplication depends on the
independence of the multiplication process, which includes multiple independent
element-to-element multiplications and multiple aggregations of independent multipli‐
cation results, as illustrated in Fig. 2.
Traditional parallel-based matrix multiplication was recently replaced with MapRe‐
duce, a parallel and distributed framework for large-scale data [3]. Typical MapReduce-
based matrix multiplication requires two MapReduce jobs. The first job creates a pair
of elements for multiplication by combining input arrays together during the map task.
The reduce task of this job is inactive at this point. In the second job, the map task
independently implements the multiplication operations on each pair of elements. The
reduce job aggregates the results that correspond to each output element. The overall
scheme of this technique is element-to-element, because each mapper implements
 Efficient MapReduce Matrix Multiplication 189

Fig. 2. Matrix multiplication independency process

Table 1. Element- by- element operations

Scheme Input Output
Element-by-element Map Files < aij , bkj >
Reduce –
Map < aij , bkj > < key, aij ∗ bkj >
[ ]
Reduce < key, aij ∗ bkj … … aij ∗ bkj > < key, cij >

multiplication element by element, as illustrated in Fig. 3. The operations implemented

by the involved MapReduce jobs are presented in Table 1. Overall, one job is responsible
for obtaining input elements from input arrays and the other is responsible for the actual
matrix multiplication process. This approach is problematic given its requirement for
high sorting and numerous map tasks. Note that the sorting process is implemented by
the shuffle task in the MapReduce platform.

Fig. 3. Element- by- element matrix multiplication

A blocking scheme was proposed to overcome the disadvantages of the element-by-

element scheme and to reduce overall computational cost. Sun et al. [24] proposed a
MapReduce matrix factorization approach. Matrix multiplication, a significant part of
190 M. Kadhum et al.

factorization, was carefully investigated to achieve efficiency. Two matrix multiplica‐

tion jobs were used to accomplish the multiplication process. The process depended on
the decomposition of the first matrix into row vectors and the second matrix into column
vectors. Multiplication of the elements of these blocks was implemented on a single
mapper. Thus, communication overhead and intermediate memory utilization were
minimized. However, the number of computational process per-mappers increased.
Jianhua et al. [8] argued that such processes require time as the mapper computation
costs are high and input writing consumes memory. Thus, two matrix multiplication
jobs decomposed the first matrix into elements or columns vectors and the second matrix
into rows instead of columns vectors. A single mapper multiplied the elements of these
blocks. Then, aggregation was implemented over the producers. Therefore, the results
of each mapper corresponded to multiple elements in the output array.

Fig. 4. Deng and Wu-schemes for matrix multiplication

Table 2. Proposed scheme operations

Scheme Input Output
Element-by-row-block Pre- Files < aij , bkj , bkj … … .. >
Process
[ ] [ ]
Map < aij , bkj , bkj … … .. > < key, aij ∗ bkj … … … aij ∗ bkj >
Reduce < key, cij , cij … … > < key, cij + cij … … >
Row-block-by-column- Pre- Files < aij , aij … … bkj , bkj >
block Process
Map < aij , aij … bkj , bkj > < key, [aij ∗ bkj ] … … … [aij ∗ bkj ] >
Reduce < key, cij , cij … … > < key, cij + cij … … >

Deng and Wu [16] presented the experimental results of the element-to-element

scheme. Moreover, they presented block-based, element-to-column and row-to-column
matrix multiplication, as illustrated in Fig. 4. The operations implemented by the
involved MapReduce jobs are presented in Table 2. Their experiments showed that the
element-to-row scheme ran faster than row-to-column. In turn, row-to-column was
 Efficient MapReduce Matrix Multiplication 191

faster than the element-to-element scheme. Moreover, the best scheme had medium
input sizes and involved a medium number of mappers. Thus, their results suggested the
need to balance input size with mapper number.
In addition to the blocking scheme, reducing the number of MapReduce jobs from
two to one also reduced the overall computational cost for matrix multiplication. There‐
fore, inputs in the MapReduce Job should be as blocks. Each block should contain
elements from both matrices to be multiplied. To reduce computational cost and memory
consumption, Deng and Wu [4, 21] modified the way Hadoop read I/O files. In the
HAMA project [21], a pre-processing stage was implemented for the same purpose.

3 Proposed Work

The goal of the proposed work is to enhance the efficiency of the matrix multiplication
in MapReduce framework. This is implemented by balancing between the processing
overhead results from using a small mapper set and the I/O overhead results from using
large mapper set, which both leads to consume time and resources, based on our previous
arguments in Sect. 2.
In the proposed technique, matrix multiplication is implemented as an element-to-
block scheme, as illustrated in Fig. 5. In the first schema; first array is decomposed into
individual elements, whereas the second array is decomposed into sub-row-based
blocks, while the second schema; first array is decomposed into sub-row-based blocks,
and the second array is decomposed into sub-column-based blocks. The number of
mappers is determined by the size of the block that is generated for the second array and
selected on the basis of the capability of the underlying mapper. Subsequently, a smaller
block size increases the number of blocks, thus requiring more mappers and vice versa.

Fig. 5. Proposed schemes for matrix multiplication

192 M. Kadhum et al.

This work uses a single MapReduce job. The map task, as listed in Table 2, is
responsible for the multiplication operations, whereas the reduce task is responsible for
the sum operations. The pre-processing step reads an element from the first array and a
block from the second array, and then merges them into one file. Note that in matrix
multiplication, the whole row in the first array has to be multiplied with the whole column
in the second array to calculate the results of an element in the output. Thus, the results
of each mapper in the proposed schemes are aggregated with other multiplication results
in the reduce task.
Compared with existing schemes, the proposed work utilizes one MapReduce job
instead of two. The number of multiplications handled by a mapper is dependent on the
capability of the mapper, which is determined by block size. Previous work has inves‐
tigated element-by-element (from the first and second arrays), element-by-column, and
row-by-column multiplications. Varying the number of elements in rows and columns
in different inputs revealed that the best result involves medium size input because the
processing overhead at each mapper is ignored. Hence, to match the capabilities of that
mapper, we proposed to vary the number of elements given to the mapper.
Unlike previous techniques, this work proposes to multiply an element by a block
of elements. The block varies from a single element into a complete row. If the block
size is equal to one, then the proposed work will be identical to an element-to-element
scheme. However, if the block size is equal to the dimension of the input array, then the
proposed work will be identical to the element-to-row/column scheme. Subsequently,
the previous work is considered as a special case of our general proposed work.
Table 3 compares the proposed and existing schemes.

Table 3. Proposed schemes vs. existing schemes

Existing schemes Element- by-element Element- by-row Row- by-column
Process dependency n3 → n n2 → n n2 → n
and synchronization
Number of mappers n3 n2 n2
Number of replicated Minimum of: n Minimum of: n Minimum of: n
elements among
mappers
Shuffle traffic n3 n2 n2
Proposed Schemes Element- By- Column-Block By-Row-Block
Column-Block
Process dependency n2 ∗ #q → n n2 ∗ #q → n
and synchronization where q = size block where q = size block
Number of mappers n2 ∗ #q n2 ∗ #q
where q = size block where q = size block
Number of replicated Minimum of: n Minimum of: n
elements among
mappers
 Efficient MapReduce Matrix Multiplication 193

4 Experiments and Result

The results of matrix multiplication using Hadoop for inputs with various size is
presented. Sparse matrices of size n*n are randomly generated with numbers from 1–10.
The experiments are conducted for various block size varied in the range [1−n]. In this
work, we run a simple matrix multiplication process with size 100*100 on the platform
with various block size varied in the range [1,10,15,20,25,30] in-order to determine the
optimal length to be given to the mapper before running the actual job. The pre-experi‐
ments are shown in Table 4.

Table 4. Proposed scheme run time result

Block size Run time (MS)
1 751235
10 543686
15 1688
20 53798
25 36314
30 237079

Based on experiments results with various block in the range [1,10,15,20,25,30], we

determined that the optimal length of block size with a minimum run time is 15, So after
that, we fixed the block sizes and run matrix multiplication process with various matrix
sizes in existing schema and our proposed schema.
The results are reported as given in Table 5. As noted, the running time is cut down in
the proposed scheme, especially, for element-by-column block scheme, in which the sorting

Table 5. Proposed scheme vs. existing schema run time result

Existing schema
Matrix Element-by-element Element-by-row Row-by-column
500 * 100 751235 105145 237079
500 * 500 1715043 120562 238688
1000 * 1000 2500256 195855 500256
2000 * 2000 2621523 543686 621523
4000 * 4000 2721523 534124 621523
Proposed schema
Matrix Element-by- ColumnBlock by-rowblock
columnblock
500 * 100 1680 36314
500 * 500 17716 36256
1000 * 1000 42063 121542
2000 * 2000 76685 121555
4000 * 4000 78325 325478
194 M. Kadhum et al.

Table 6. Proposed scheme vs. existing schema memory consumption result

Existing schema
Matrix Element-by-element Element- by-row Row-by-column
500 * 100 36000 37004 36314
500 * 500 56310 55457 56214
1000 * 1000 110241 112400 114000
2000 * 2000 189254 189475 189124
4000 * 4000 212524 212142 212471
Proposed schema
Matrix Element-by- Column-block by-roblock
columnblock
500 * 100 36766 36314
500 * 500 55000 56310
1000 * 1000 111454 111245
2000 * 2000 190254 189456
4000 * 4000 212441 212111

process in the shuffle is reduced. As the matrix size growth, the stability of the proposed
scheme in better compared to the existing schemes, which, seems to be almost linear.
The results for space consumption for the proposed and existing schemes are reported
as given in Table 6. As noted, proposed and existing schemes almost identical but the
proposed work takes slightly more spaces compared to others. Therefore, if the user
cares about time our proposed schema is the best choice, but if he cares about the memory
capacity he can choose from another algorithm.
Our algorithm is written in java and the experimental results are calculated for our Proposed
Schemes and Existing Schemes on HP® core™ i7-5500U CPU @ 2.40 GHz /8 GB RAM.

5 Conclusion

A block-based matrix multiplication schemes were proposed in this paper. The proposed
schemes balance between the processing overhead results from using a small mapper set
and the I/O overhead results from using large mapper set, which both leads to consume time
and resources. This balancing is optimizing by determining the optimal block size and
number of involved mappers. The results show that the proposed schemes reduce both time
and memory utilization.

6 Future Work

Our proposed schema is implemented on sparse algorithm, our future work will be on dense
algorithm, in other hand, we can optimized reduce set.
 Efficient MapReduce Matrix Multiplication 195

References

1. Cannon, L.E.: A Cellular Computer to Implement the Kalman Filter Algorithm. No. 603-
Tl-0769. Montana State Univ Bozeman Engineering Research Labs (1969)
2. Coppersmith, D., Winograd, S.: Matrix multiplication via arithmetic progressions. In:
Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing, pp. 1–6.
ACM (1987)
3. Catalyurek, U.V., Aykanat, C.: Hypergraph-partitioning-based decomposition for parallel
sparse-matrix vector multiplication. IEEE Trans. Parallel Distrib. Syst. 10(7), 673–693 (1999)
4. Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. In: OSDI,
p. 10. USENIX (2004)
5. Dean, J., Ghemawat, S.: MapReduce: a flexible data processing tool. Commun. ACM 53(1),
72–77 (2010)
6. Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. Commun.
ACM 51(1), 107–113 (2008)
7. Dekel, E., Nassimi, D., Sahni, S.: Parallel matrix and graph algorithms. SIAM J. Comput.
10(4), 657–675 (1981)
8. Deng, S., Wenhua, W.: Efficient matrix multiplication in hadoop. Int. J. Comput. Sci. Appl.
13(1), 93–104 (2016)
9. Fox, G.C., Otto, S.W., Hey, A.J.G.: Matrix algorithms on a hypercube I: Matrix multiplication.
Parallel Comput. 4(1), 17–31 (1987)
10. Lin, J., Dyer, C.: Data-intensive text processing with MapReduce. Synth. Lect. Hum. Lang.
Technol. 3(1), 1–177 (2010)
11. Liu, X., Iftikhar, N., Xie, X.: Survey of real-time processing systems for big data. In:
Proceedings of the 18th International Database Engineering & Applications Symposium.
ACM (2014)
12. Lv, Z., Hu, Y., Zhong, H., Wu, J., Li, B., Zhao, H.: Parallel K-means clustering of remote
sensing images based on MapReduce. In: Wang, F.L., Gong, Z., Luo, X., Lei, J. (eds.) WISM
2010. LNCS, vol. 6318, pp. 162–170. Springer, Heidelberg (2010). doi:
10.1007/978-3-642-16515-3_21
13. Mahafzah, B.A., Sleit, A., Hamad, N.A., Ahmad, E.F., Abu-Kabeer, T.M.: The OTIS hyper
hexa-cell optoelectronic architecture. Computing 94(5), 411–432 (2012)
14. Norstad, J.: A mapreduce algorithm for matrix multiplication (2009). http://www.norstad.org/
matrix-multiply/index.html. Accessed 19 Feb 2013
15. Thabet, K., Al-Ghuribi, S.: Matrix multiplication algorithms. Int. J. Comput. Sci. Netw. Secur.
(IJCSNS) 12(2), 74 (2012)
16. Seo, S., Yoon, E.J., Kim, J., Jin, S., Kim, J.S., Maeng, S.: Hama: An efficient matrix
computation with the mapreduce framework. In: 2010 IEEE Second International Conference
on Cloud Computing Technology and Science (CloudCom), pp. 721–726. IEEE, November
2010
17. Sleit, A., Al-Akhras, M., Juma, I., Alian, M.: Applying ordinal association rules for cleansing
data with missing values. J. Am. Sci. 5(3), 52–62 (2009)
18. Sleit, A., Dalhoum, A.L.A., Al-Dhamari, I., Awwad, A.: Efficient enhancement on cellular
automata for data mining. In: Proceedings of the 13th WSEAS International Conference on
Systems, pp. 616–620. World Scientific and Engineering Academy and Society (WSEAS),
July 2009
19. Sleit, A., AlMobaideen, W., Baarah, A.H., Abusitta, A.H.: An efficient pattern matching
algorithm. J. Appl. Sci. 7(18), 269–2695 (2007)
196 M. Kadhum et al.

20. Sleit, A., Saadeh, H., Al-Dhamari, I., Tareef, A.: An enhanced sub image matching algorithm
for binary images. In: American Conference on Applied Mathematics, pp. 565–569, January
2010
21. Sun, Z., Li, T., Rishe, N.: Large-scale matrix factorization using mapreduce. In: 2010 IEEE
International Conference on Data Mining Workshops. IEEE (2010)
22. Wu, G., et al.: MReC4.5: C4.5 ensemble classification with MapReduce. In: 2009 Fourth
ChinaGrid Annual Conference. IEEE (2009)
23. Zaharia, M., et al.: Job scheduling for multi-user mapreduce clusters. EECS Department,
University of California, Berkeley, Technical Report UCB/EECS-2009-55 (2009)
24. Zheng, J., Zhu, R., Shen, Y.: Sparse matrix multiplication algorithm based on MapReduce. J.
Zhongkai Univ. Agric. Eng. 26(3), 1–6 (2013)

View publication stats