Available online at www.sciencedirect.

com
ScienceDirect
ScienceDirect
Procedia Computer Science 00 (2019) 000–000
Available online at www.sciencedirect.com
Procedia Computer Science 00 (2019) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia
ScienceDirect
Procedia Computer Science 163 (2019) 472–481

16th International Learning & Technology Conference 2019

16th International Learning & Technology Conference 2019
Using Data Mining Techniques to Guide Academic Programs
Using Data Mining Techniques to Guide Academic Programs
Design and Assessment
Design and Assessment
Anwar Ali Yahyaa,b * and Addin Osmana
Anwar Ali Yahyaa,b* and Addin Osmana
a
College of Computer Science and Information Systems, Najran University, Saudi Arabia
Faculty
b a
CollegeofofComputer
ComputerScience
Scienceand
andInformation
InformationSystems,
Systems,Thamar
NajranUniversity,
University,Thamar, Yemen
Saudi Arabia
b
Faculty of Computer Science and Information Systems, Thamar University, Thamar, Yemen

Abstract
Abstract
This paper presents a novel application of data mining techniques to guide academic programs design and assessment. More
This paper presents
specifically, it propose a novel application rule
using association of data mining
mining techniques
techniques to guide
to discover academic
a set programs
of rules that governdesign and assessment.
the relationship betweenMore
two
specifically, it propose
core components of anusing association
academic program, ruleprogram
mining educational
techniques toobjectives
discover (PEOs)
a set of rules that govern
and students the relationship
outcomes(SOs). As abetween two
case study,
this paper
core demonstrates
components how association
of an academic program,rule mining
program techniquesobjectives
educational are applied to mine
(PEOs) mappingoutcomes(SOs).
and students rules between theAs aPEOs and a
case study,
this paper demonstrates
predefined set of SOs adoptedhow association rule mining
by the American Board techniques
for Engineering are applied to mine mapping rulesAccreditation
and Technology-Engineering between the Commission
PEOs and a
predefined
(ABET-EAC) set for
of SOs adoptedprograms.
engineering by the American Board
To this end, forofEngineering
a set 152 ABET-EAC and Technology-Engineering Accreditation
accredited engineering programs’ Commission
self-study reports
have been collected
(ABET-EAC) and the mapping
for engineering programs. data
Tobetween
this end,their
a setPEOs
of 152and ABET-EAC
ABET-EAC SOs have
accredited been extracted.
engineering Thisself-study
programs’ dataset has been
reports
have been collected
pre-processed and the mapping
and transformed into adata between their
representation PEOs for
suitable and applying
ABET-EAC SOs haverule
association been extracted.
mining This dataset
techniques. This has been
involves
pre-processed
identifying a setand
of transformed
PEOs labels, into a representation
annotating suitable
data instances with forPEOsapplying
labels, association
and projectingruleeach
mining techniques.
multi-label This involves
data instance into a
set of single-label
identifying a set ofinstances. Apriopi
PEOs labels, algorithm
annotating dataisinstances
then appliedwithto discover
PEOs theand
labels, rules that govern
projecting eachthe mapping data
multi-label between PEOs
instance anda
into
set of single-label
ABET-EAC SOs. Theinstances. Apriopi
discovered rulesalgorithm is then importance
are of particular applied to discover the the
for guiding rules that and
design govern the mapping
assessment between PEOs
of engineering and
academic
ABET-EAC
programs. SOs. The
Besides that,discovered rules are
the discovered of particular
rules unveil of aimportance
number offor guiding the
interesting design andbetween
correlations assessment
PEOsof engineering
and ABET-EAC academic
SOs
that need further
programs. Besides investigation by pedagogists.
that, the discovered rules unveil of a number of interesting correlations between PEOs and ABET-EAC SOs
that need further investigation by pedagogists.
© 2019 The Authors. Published by Elsevier B.V.
© 2019 The Authors. Published by Elsevier B.V.
This
© 2019is an open
The accessPublished
Authors. article under the CC BY-NC-ND
by Elsevier B.V. license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
This is an
Peer-review open access
under article under
responsibility of the CC BY-NC-ND
scientific
Peer-review under responsibility of the scientific committeecommitteelicense
ofofthe(https://creativecommons.org/licenses/by-nc-nd/4.0/)
the 16thInternational
16th International Learning
Learning && Technology
Technology Conference
Conference 2019.
2019.
Peer-review under responsibility of the scientific committee of the 16th International Learning & Technology Conference 2019.
Keywords: Association rule mining; educational data mining; academic program management; program educational objectives; student
outcomes; ABET
Keywords: accreditation.
Association rule mining; educational data mining; academic program management; program educational objectives; student
outcomes; ABET accreditation.

* Corresponding author. Tel.: +966-055-417-4156; fax: +966-017-542-8617.

* E-mail [email protected]
address:author.
Corresponding Tel.: +966-055-417-4156; fax: +966-017-542-8617.
E-mail address: [email protected]
1877-0509 © 2019 The Authors. Published by Elsevier B.V.
This is an open
1877-0509 access
© 2019 The article under
Authors. the CC by
Published BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Elsevier B.V.
Peer-review under
This is an open responsibility
access of the
article under thescientific committee
CC BY-NC-ND of the(https://creativecommons.org/licenses/by-nc-nd/4.0/)
license 16th International Learning & Technology Conference 2019.
Peer-review under responsibility of the scientific committee of the 16th International Learning & Technology Conference 2019.

1877-0509 © 2019 The Authors. Published by Elsevier B.V.

This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review under responsibility of the scientific committee of the 16th International Learning & Technology Conference 2019.
10.1016/j.procs.2019.12.130
 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481 473
2 Author name / Procedia Computer Science 00 (2019) 000–000

1. Introduction

In academia, an academic Program is a unique course of study that culminates in the awarding of a specific degree
(or certificate) in combination with a specific major. As is the case of student lifecycle, each academic program has a
lifecycle; typically includes: conceptualization, introduction, growth, maturity, and decline [1] . During the program
lifecycle, the availability of information to its decisions makers is crucial for its success. This paper show how data
mining techniques can be exploited to elicit a valuable information on the relationship between two core components
of academic program, that are PEOs and SOs. The elicited information are of particular importance for program
management in different stages of its lifecycle. For example, to optimize its design, evaluate its quality, and develop
a robust assessment plan. However, prior to this, it would be useful to shed light on the structure of the Outcome Based
Education (OBE) [2] academic program and the relationship between its components.
OBE is a new education paradigm focuses on graduate attributes that allow students to accept the challenges, adopt
to technological changes, and translate their knowledge to new contexts for the benefit of the society. That is to say,
the OBE-based academic programs are designed to develop various abilities as per the requirements of graduate
attributes. An essential step in designing OBE-based academic program is the identification of long-term PEOs and
SOs [3]. While the SOs represent the knowledge, skills, and capabilities that should students possess by the time of
graduation, PEOs represent the achievements graduates should attain few years (3 to 5 years and more) after graduation
[4]. Having specified the PEOs and SOs of the OBE-based academic program and established the mapping between
them, the curriculum is designed in such a way that the student ultimately gain knowledge and develop skills stated in
the SOs. Accordingly, the curriculum is viewed as number of courses aim to attain certain Course Outcomes (COs)
that map to SOs which themselves map to PEOs which in turn map to the mission of the institution in a hierarchical
structure shown in Fig. 1 [2].

Fig. 1. Structure of OBE Academic Program [2]

It goes without saying that a robust mapping between the PEOs and SOs of an OBE-based academic program plays
a crucial role in its design and assessment. It is a connection point between a program and its context and plays a key
role in constructing program curriculum and assessment of educational objectives. For example, the questions of
alumni questionnaire, a direct program assessment method used to measure the attainment of the PEOs, are excerpted
from SOs and take into consideration their mapping to PEOs. Based on the average rating by the alumni the attainment
of PEO is obtained and thus the program is assessed [3].
Despite the key role of PEOs, SOs, and their mapping in OBE-based academic program, there is a consensus on
the lingering confusion, among practitioners, related to them. The drastic consequences of such confusion is a poor
design of curriculum and teaching strategies and a misleading assessment of PEOs and ultimately inaccurate program
corrective plan. In part, this confusion is attributed to the inherent difficulties of understanding the meaning of these
terms and their relationship to each other [4]. In case of ABET accreditation, this terminologies confusion has created
an insistent need for progressive clarification changes, over the years, in ABET wording of criterion 2 (Program
Educational Objectives) and criterion 3 (Student Outcomes), the accreditation policy, and procedure manual. This
concern has been raised in several events such as the fall 2005 ABET Summit [5], where questions regarding the
difference between PEOs, SOs, and their mapping have been asked again.
474 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481
Author name / Procedia Computer Science 00 (2019) 000–000 3

Based on the foregoing premises, it ca be concluded that the discovery of a set of well-founded and accurate
mapping rules between PEOs and SOs would be of significant benefit for the design and assessment of OBE-based
academic programs. This paper proposes an effective way of doing so by inductively interrogate a set of PEOs-SOs
mapping data of academic programs using Association Rule Mining (ARM) techniques [6] [7]. In doing so a suitable
source of data has to be specified. For this purpose, program’s self-study reports (SSRs), documents formally submitted
to the academic accreditation agency to obtain academic accreditation, is specified as a source of the required data.
The remainder of this paper reviews the related literature, presents the collected dataset, describes how ARM
technique can be applied to this dataset to discover a set of PEOs and SOs mapping rules, and evaluates these rules.

2. Related Works

The huge amount of digital content available in different fields has motivated research in, and the development of,
different techniques that make it easier to search, organize and analyze this content. Data mining is a discipline that
have emerged to analyze data in an automated manner by finding patterns and relationships in raw data [6]. Data
mining has already achieved significant success in many areas including medicine, business, robotics and computer
vision, to name just a few. By the same token, the continuous growth of data in educational institutions leads to the
emergence of Educational Data Mining (EDM) with a concern of developing, researching, and applying computerized
methods to detect patterns in large collections of educational data that would otherwise be hard or impossible to
analyze [8]. In an educational context, there are many interesting and difficult problems that may arise from four
aspects: administrative problems and problems associated with school, academic staff, and students. The application
of EDM to solve these problems have been reviewed in several surveys [8] [9] [10], however for the sake of this
research, this survey focuses on the efforts made to study various aspect of academic program at any stage of its
lifecycle. In this regards, a combination of neutral network and experts’ prior knowledge are applied in [11] to predict
and evaluate student-learning outcomes of an academic program and ultimately enhance teaching quality. K−means
clustering algorithm is applied in [12] to investigate the relationship between skills taught in business programs and
the title of the program using a data set extracted from program catalogue. The analysis shows that, with very limited
exceptions, the labels of programs match the skills one would expect to learn. Data mining methods are used in [13]
to identify the similarities between course content at a learning object, module and program level. The hierarchy and
structures of the learning environment created by educators are extracted and then the similarity between documents
are calculated and both are used for narrowing and selecting applicable learning objects with similar content.
Regression and classification techniques are used in [14] to investigate the effect of academic program type, among
others like years of study and gender, on the mental health of students. The dataset, survey responses of undergraduate
students engineering programs in a large candian university, is analyzed and interestingly the results show that due to
the competitive nature of the program, students of electrical engineering had lower mental health, though high
self-actualization, whereas due to strong classmate relationships and a flexible curriculum, students of systems design
had mental health higher scores.
The applications of EDM to academic program assessment can be found in many works. In the first work [15],
dated back to 2006, a data-driven course assessment and program assessment is applied to quantify the level to which
program’s curriculum meet the program outcomes. This involves the use of clustering and prediction techniques for
question evaluation, topic clustering, predicting missing scores, and clustering with partial topic information to
construct score matrix and relevancy matrix. Using these matrices with course matrix the numeric value for each
program outcome can be determined by simply find the average score for each topic and multiply that vector by the
course matrix. More on using data mining in programs assessment is found in [16], where different data mining
techniques are applied to online professional development program that contains workshops for pre K-12 teachers. In
this work data mining techniques is applied to study the shared learning characteristics: - frequent learning paths
(association rules),engagement prediction (clustering), expectation prediction (decision tree), workshop satisfaction
prediction (decision tree), and instructor quality Prediction (decision tree).
 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481 475
4 Author name / Procedia Computer Science 00 (2019) 000–000

3. Methodology

This section describes the detailed methodology of mining the relationship between PEOs and SOs of academic
programs. It is based on the general methodology of the knowledge discovery process, depicted in Fig. 2, which
involves raw data collection, selection, preprocessing, transformation, mining, and evaluation [17].

Fig. 2. Knowledge Discovery Steps [17]

In the data collection step, a raw data is collected and used to create a target dataset, or focusing on a subset of
variables or data samples on which discovery is to be performed. The target data is cleaned and preprocessed in order
to obtain consistent data in the pre-processing step. The transformation step transforms the data using dimensionality
reduction or other transformation methods. The step of data mining applies procedures to search for patterns of interest
in a particular representational form, depending on the data mining objective. Finally, in the interpretation/evaluation
step, the mined patterns are interpreted and evaluated.
One of the data mining techniques, which is applied to discover all hidden associations that satisfy some user-
predefined criteria is association rules algorithms [6] [18]. They work by dividing the problem into two parts: mining
for frequent itemsets and rules discovery from the frequent itemsets. A Frequent itemset is a set of items, which are
more than a threshold. The procedure of finding frequent itemsets is simple but very time consuming, because of the
large number of the possible combinations. Once they have been discovered, the rules production is a simple process.
A widely used algorithm for the association rules mining is the Apripri algorithm [18]. It is based on the following
rule: All sub-itemsets of a frequent itemset must also be frequent. By using this rule, Apriori is able to prune a huge
amount of itemsets examinations since it is certain that they are not frequent. Frequent sub-itemsets are extended one
item at a time (candidate generation), and groups of candidates are examined. It terminates when no further extensions
are found. In other words, Apriori algorithm generates candidate itemsets of length l from itemsets of length l−1 and
then it prunes the candidates which have a non frequent sub-itemset. Thus, it keeps only the frequent item sets among
the candidates.
476 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481
Author name / Procedia Computer Science 00 (2019) 000–000 5

4. Mining PEOs-SOs Mapping Rules in Engineering Programs

The following subsections give a detailed description of how the above methodology of knowledge discovery has
been particularized to discover the PEOs-SOs mapping rules in engineering programs.

4.1. Raw Data Collection

As mentioned above, this research is concerned with the discovery of the association rules between the PEOs and
SOs, therefore the SSRs of academic programs represent a suitable source of the raw data. In accreditation, SSR is the
primary document used by a program to demonstrate its compliance with all applicable criteria and policies adopted
by accreditation agency such as ABET. According to ABET, the SSR addresses all paths to completion of the degree,
all methods of instructional delivery used for the program, and all remote location offerings. In practice ABET requests
each program to develop its PEOs and link them to a set of SOs it adopts. Since ABET adopts different set of SOs for
each discipline, only engineering programs previously accredited by ABET-EAC between 2000 and 2017 has been
considered in this collection [19]. The set of SOs adopted by ABET-EAC is listed below.
(a) an ability to apply knowledge of mathematics, science, and engineering
an ability to design and conduct experiments, as well as to analyze and
(b)
interpret data
an ability to design a system, component, or process to meet desired needs
(c) within realistic constraints such as economic, environmental, social,
political, ethical, health and safety, manufacturability, and sustainability
(d) an ability to function on multidisciplinary teams
(e) an ability to identify, formulate, and solve engineering problems
(f) an understanding of professional and ethical responsibility
(g) an ability to communicate effectively
the broad education necessary to understand the impact of engineering
(h)
solutions in a global, economic, environmental, and societal context
(i) a recognition of the need for, and an ability to engage in life-long learning
(j) a knowledge of contemporary issues
an ability to use the techniques, skills, and modern engineering tools
(k)
necessary for engineering practice

The result of data collection is a set of 152 engineering programs SSRs distributed over years 2002-2017 as in Fig. 3.

Fig. 3. SSRs distribution over years

 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481 477
6 Author name / Procedia Computer Science 00 (2019) 000–000

4.2. Dataset Selection

In this step, the data of the PEOs-SOs mapping has been drawn from the collected programs’ SSRs. These data
have been extracted from sub section B of the third criteria (Student Outcomes) of each SSR and consolidated in a
table form as shown in Table 1, where ✓ denotes the presence of the SO in the PEOs-SOs mapping and × denotes its
absence
Table 1. Example of PEOs-SOs mapping data.
PEOs a b c d e f g h i j k
Practice the disciplines of
transportation,
environmental, structural, ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
water resources, and
geotechnical engineering,
and/or related
Engage infields.advanced
education, research, and ✓ ✓ ✓ × ✓ × × × ✓ × ✓
development
Pursue continuing education
× × × × × × × × ✓ × ✓
and professional licensure
Act in a responsible,
professional and ethical × × × × × × × ✓ × ✓ ×
manner

4.3. Dataset Preprocessing

The preprocessing of the dataset involves substantial verification and validation of the content, attempts to remove
spurious or duplicated objectives, fulfilling the objectives and outcomes format, etc. Figure 4 shows the distribution
of SOs over the dataset.

Fig. 4. SOs distribution over the dataset

Moreover, Table 2 shows the main statistical aspects of the collected dataset.

Table 2. Statistical aspects of PEOs-SOs dataset

Statistical Aspect Value
No. of PEOs 667
No. of SOs 11
Max. No. of PEOs in a program 17
Min. No. of PEOs in a program 2
Avg. No. of PEOs in a program 4.39
478 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481
Author name / Procedia Computer Science 00 (2019) 000–000 7

4.4. Dataset Transformation

Since the PEOs in the collected dataset are in a textual form, they are unsuitable for association mining algorithms.
Therefore, a transformation procedure that transform each PEO text into a single or a combination of labels
representing the graduates attributes expressed in its text is proposed. Prior to this, a set of PEOs labels is identified,
which is used to annotate data instances with a single or multiple PEOs labels, and then each multi-label data instance
is projected into single-label data instance. The details of the transformation procedure is given in the following
subsections.

4.4.1. PEOs Label Set Identification

In OBE model, PEOs describe both technical and professional aspects of the expected achievements of the graduate
after graduation. On this basis, they can be grouped into a finite set of common attributes. Typical PEOs cover the
followings attributes: technical skills, professional, ethical, and communicational aspects, management and leadership,
life-long learning and continuous education, advanced and graduate studies pursuing, and other aspects [20]. Based
on PEOs wordings of a number of engineering programs, the following set of common PEOs labels, shown in Table3,
have been identified.
Table 3. PEOs Label Set

No PEOs Category PEO Label

1 Life-long Learning LL
2 Communication C
3 Leadership L
4 Teaming T
5 Ethical Conduct EC
6 Professionalism P
7 Social and Community SC
8 Career Success CS
9 Technical Competency TC
10 Knowledge Competency KC
11 Graduate Studies GS
12 Others O

4.4.2. PEOs Annotation

Using the identified PEOs label set, each data instance is annotated with one or more labels that match its PEO text.
This process has been accomplished by three annotators who initially annotated the data set individually using the
PEOs label set. The three annotators then met to resolve the conflicting cases of PEOs annotation. It should be
mentioned that some PEOs text have been annotated with more than PEOs label, because it might happens that the
PEOs text describes multiple skills and attributes. Table 4 shows excerpts of the dataset after PEOs annotation. It
should be noted that the values 1 in each data instance indicates the presence of SO in the mapping between its PEO
and the SOs set, whereas 0 indicate its exclusion from the mapping .

Table 4. PEOs Labeled Dataset

PEOs Labels a b c d e f g h i j k
TC 1 1 1 1 1 1 1 1 1 1 1
LL GS 1 1 1 0 1 0 0 0 1 0 1
LL P 0 0 0 0 0 0 0 0 1 0 1
 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481 479
8 Author name / Procedia Computer Science 00 (2019) 000–000

4.4.3. Multi-label to Single-label Data Projection

As shown in Table 5, the PEOs annotation step results in a set of data instances with multiple PEOs labels. As the
aim is to discover the rules that governs the relationships of each PEO and the SOs set, each multi-label data instance
is decomposed into a single-label data instances, where each one is annotated with a single PEO of those in the original
data instance. The results of this step is an enlarged dataset with 1196 single-label data instances.

4.5. Association Rules Mining

In this step, Apriori algorithm is applied under WEKA framework [21]. It works by iteratively generating for each
PEO the frequent k-item sets whose their count equal to or greater than to a pre-specified minimum support count
which can be calculated as follows

support(PEO→ SOs) = count(PEO ∪ SOs) / N (1)

A process of joining and pruning is applied to the current frequent k-item sets to generates frequent k + 1 item sets.
The frequent k-item sets are joined together and the k-item sets that cannot satisfy the minimum support count are
pruned since they are infrequent item sets. The process of joining and pruning is repeatedly applied until the frequent
item sets become null. Thereafter, the algorithm is aborted and the association rules generation begins. The confidence
level of the rule must satisfy the user specified confidence threshold. The confidence level of the rules can be calculated
using the equation 2.

confidence(PEO→SOs)=count(PEO∪SOs)/count(PEO) (2)

To implement the Apriori algorithm as described above in WEKA, the minimum support and minimum confidence
parameters must be specified. Since the interest of this research is confined to the relationship between PEOs and SOs,
the application of Apriori algorithm focuses on generating rules that contains a particular PEO in it antecedent and a
combination of SOs in its consequent. This is can be achieved by controlling the minimum support parameter of
WEKA to be in an interval centered by the probability of that PEO. The generated rules for each PEO are manipulated
to select only rules that has a particular PEO in its antecedent and a combination of SOs in its consequent. Thereafter
the generated rules are sorted based on their confidence. Table 5 shows examples of the discovered rules along with
their confidences.
Table 5. Examples of the discovered PEOs-SOs mapping rules
Rule confidence
if (PEO=LL) then i=1 0.85
if (PEO=C) then g=1and i=0 0.79
if (PEO=L) then d=1 0.76
if (PEO=T) then a=0 and b=0 0.76
if (PEO=EC) then f =1 0.85
if (PEO=P) then b=0 and c=0 0.7
if (PEO=SC) then b=0 and k=0 0.81
if (PEO=CS) then a=1 and e=1 0.66
if (PEO=TC) then a=1 0.74
if (PEO=KC) then d=0 0.75
if (PEO=GS) then a=1 and e=1 0.55
480 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481
Author name / Procedia Computer Science 00 (2019) 000–000 9

4.6. Results Evaluation

The set of rules for each PEO has been sorted in descending order according to their confidence values and the top
10 rules have been used to generate a more compact representation of the discovered rules as shown in Table 6. It can
be observed that the PEOs Ethical Conduct (EC) and Social and Community (SC) have the highest average confidence,
whereas Knowledge Competency (KC) and Graduate Study (GS) have the lowest. In this rules representation the PEOs,
Social and Community (SC) and Knowledge Competency (KC), are not dependent on the presence of any SOs. They
mainly depend on the absence of different combinations of SOs. The PEOs Life-long Learning(LL),
Communication(C), Ethical Conduct(EC), and Professionalism(P) depend on the presence of a single SO. This
indicates a one to one mapping between the graduate attributes of the PEO and the corresponding skills of the SO.
Interestingly, the PEOs Leadership(L) and Teaming (T) depend on the presence of the same SOs (d and g), and on
the absence of almost the same combination of SOs. This suggests a correlation between the two graduate attributes
Leadership(L) and Teaming(T) and a correlation between the skills d and g. A somewhat related PEO to them is
Communication(C), which have a similarity in the dependent SOs. Concerning the PEOs Career Success (CS) and
Technical Competency(TC), their dependence on the same combination of SOs suggests a very high correlation
between their corresponding graduate attributes. A closely related PEO to them is Graduate Study (GS), which depends
on the same set of SOs except c. As for PEO Knowledge Competency (KC), it does not show dependency on the
presence of any SOs; however, it depends on absence of seven SOs.

Table 6. Recommended PEOs-SOs mapping rules

SOs
No PEO Conf. Interval Avg. Conf.
a b c d e f g h i j k
1 LL 0 0 0 0 0 0/1 0 0/1 1 0/1 0/1 0.65-0.85 0.693
2 C 0 0 0/1 0/1 0/1 0 1 0/1 0 0/1 0/1 0.77-0.91 0.792
3 L 0 0 0 1 0/1 0/1 1 0/1 0 0/1 0 0.65-0.76 0.679
4 T 0 0 0 1 0 0/1 1 0/1 0 0/1 0 0.66-0.82 0.726
5 EC 0 0 0/1 0/1 0 1 0/1 0/1 0/1 0/1 0 0.81-0.89 0.827
6 P 0 0 0 0 0 1 0 0/1 0/1 0/1 0/1 0.68-0.8 0.711
7 SC 0 0 0/1 0/1 0 0/1 0/1 0/1 0/1 0/1 0 0.81-0.9 0.826
8 CS 1 1 1 0/1 1 0/1 0/1 0/1 0/1 0/1 1 0.69-0.74 0.678
9 TC 1 1 1 0/1 1 0 0 0 0 0 1 0.65-0.74 0.691
10 KC 0/1 0 0/1 0 0/1 0 0 0 0 0/1 0 0.53-0.75 0.62
11 GS 1 1 0/1 0 1 0 0/1 0/1 1 1 1 0.56-0.77 0.63

Finally, the recommended PEOs-SOs mapping rules provide very useful insights to the decision makers of
engineering academic programs during the continuous improvement process. They represent a guidance manual that
can be used to assist academicians when designing a new academic program or reviewing the existing one.

5. Conclusion

Association rules data mining technique is proposed and applied to discover the set of rules that govern the mapping
between PEOs and SOs of academic programs. It is applied to dataset extracted from SSRs of 152 ABET accredited
engineering programs. The discovered rules are useful for academic programs design and assessment. In addition, the
discovered rules disclose a number of interesting correlations between PEOs and ABET SOs that are informative for
programs designing and assessment and deserving further investigation.
 Anwar Ali Yahya et al. / Procedia Computer Science 163 (2019) 472–481 481
10 Author name / Procedia Computer Science 00 (2019) 000–000

References

[1] L. Wallace-Hulecki. [Online]. Available: http://www.semworks.net/white-papers-books/papers/Academic-Innovation-White-Paper.pdf.

[Accessed 30 12 2018].
[2] S. Subbaraman, R. R. Jagtap and S. S. Shinde, "Outcome Based Learning: A Case Study," in IEEE International Conference in MOOC,
Innovation and Technology in Education (MITE), 2013.
[3] R. Shivakumar, S. H. Usha, N. A. Chetan, K. Sainath and M. Samita, "Establishing Program Educational Objectives," Journal of
Engineering Education Transformations, vol. 29, no. 2, pp. 53-58, 2015.
[4] O. Petersen, S. Williams and E. Durant, "Understanding ABET Objectives and Outcomes," in 2007 ASEE Annual Conference, Honolulu,
HI, 2007.
[5] ABET, San Diego, CA: ABET, 2005.
[6] J. Han, M. Kamber and J. Pei, "Data Mining: Concepts and Techniques," in The Morgan Kaufmann Series in Data Management Systems,
Elsevier, 2011.
[7] M. Dunham, Data Mining: Introductory and Advanced Topics, Prentice Hall, 2003.
[8] R. S. J. Baker, "Data mining for education," in International encyclopedia of education, 2011.
[9] C. Romero and S. Ventura, "Educational data mining: A survey from 1995 to 2005," Expert systems with applications, vol. 33, no. 1, 2007.
[10] S. R. Baker and K. Yacef, "The state of educational data mining in 2009: A review and future visions," Journal of Educational Data Mining,
vol. 1, no. 1, pp. 3-17, 2009.
[11] S. Zhong-mei, Q. Qiong-fei and F. Lu-qi, "Educational data mining and analyzing of student learning outcome from the perspective of
learning experience," in 7th International Conference on Educational Data Mining, 2014.
[12] R. Wes Crues, "Untangling The Program Name Versus The Curriculum: An Investigation of Titles and Curriculum Content," in 10th
International Conference on Educational Data Mining, 2017.
[13] K. Goslin and M. Hofmann, "Identifying and visualizing the similarities between course content at a learning object, module and program
level," in 6th International Conference on educational data mining, 2013.
[14] M. Deziel, D. Olawo, L. Truchon and L. Golab, "Analyzing the mental health of engineering students using classification and regression,"
in 6th International Conference of Educational Data Mining, 2013.
[15] T. Winters, Educational Data Mining: Collection and Analysis of Score Matrices for Outcomes-Based Assessment, University of California
Riverside, 2006.
[16] K. Rice and J. L. Hung, "Data mining in online professional development program evaluation: An exploratory case study," International
Journal of Technology in Teaching & Learning, pp. 1-20, 2015.
[17] S. Ougiaroglou and G. Paschalis, "Association Rules Mining from the educational data of ESOG web-based application," in IFIP Advances
in Information and Communication Technology, vol. 382, Berlin, Heidelberg, Springer, 2012.
[18] R. Agrawal and S. Ramakrishnan, "Fast algorithms for mining association rules in large databases," in the 20th International Conference
on Very Large Data Bases, VLDB, Santiago, Chile pp, 1994.
[19] A. Osman, A. A. Yahya, M. B. Kamal, “A Benchmark Collection for Mapping Program Educational Objectives to ABET Student Outcomes:
Accreditation. In: Alenezi M., Qureshi B. (eds) 5th International Symposium on Data Mining Applications. Advances in Intelligent Systems
and Computing, vol 753. Springer, Cham, 2018
[20] N. Abbadeni, A. Ghoneim and A. Alghamdi, "Program Educational Objectives Definition and Assessment for Quality and Accreditation,"
International Journal of Engineering Pedagogy, vol. 3, no. 3, pp. 33-46, 2013.
[21] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I. H. Witten, "The WEKA data mining software: an update," vol. 11, no.
1, pp. 10-18, 2009.