ASSESSMENT OF STUDENT LEARNING OUTCOMES (SLOs) FOR THE

AERONAUTICAL ENGINEERING PROGRAM OF PATTS

A Master’s Thesis
Submitted to the Faculty of the Graduate School of the
Philippine State College of Aeronautics
Manila, Philippines

In Partial Fulfillment
of the Requirements for the Degree of
Master of Education in Aeronautical Management

NEIL ANSAGAY

August 2019
 1

CHAPTER 1

THE PROBLEM AND ITS BACKGROUND

Introduction

Today, higher education institutions are being challenged to show evidence that

students are actually learning and gaining knowledge (Saint Germain, 2008). Education

policy makers and the general public have raised concerns about the quality of educational

programs and student learning outcomes. These concerns translate to losses in enrollment,

funding and accreditation (Wegner, 2008). Stakeholders, including governmental agencies,

parents and industry employers are asking whether students are really learning and if

student outcomes data are being used for program improvement purposes. This is the

primary motivation of the researcher for conducting a study on student learning outcomes

of Aeronautical Engineering graduates.

The concept of identifying learning outcomes for college students is not a new one.

Educators, government officials, and industry managers and employers understand the

need to equip students with the knowledge, skills and behaviors they will need to be

successful in the workplace and to be contributing citizens in society. Many groups have

undertaken the task of identifying the learning outcomes of college students both from a

broad educational perspective and through discipline-specific skills and competencies

(Kuh and Ikenberry, 2009; ABET 2014).

Learning outcomes are undeniably key to a meaningful education, and focusing on

learning outcomes is essential to inform diagnosis and improve teaching processes and

student learning. While there is a long tradition of learning outcomes’ assessment within
 2

institutions’ courses and programs, emphasis on learning outcomes has become more

important in recent years.

Situationer

PATTS College of Aeronautics is one of the country’s number one Aeronautical

Colleges in the country today. It is founded as a joint enterprise of Filipino and American

pioneers in aviation, the Philippine Air Transport and Training Services (PATTS) which

started operations in 1969. Its primary aim was to establish a manufacturing and assembly

plant for trainer aircrafts. Its secondary aim was to put up an Aeronautical School to meet

the domestic and international demands in the fields of aviation and air transportation

industry.

Under the excellent stewardship of the Board of Directors, PATTS continues to

serve as a model to the the industry. It attained the College status in 1989 and transformed

from a small school to a big reputable educational institution now known as the PATTS

College of Aeronautics. The Schools is now located at Lombos Avenue, San Isidro,

Paranaque City.

PATTS course offerings are recognized by the Commission on Higher Education

(CHED), the Technical Education and Skills Development Authority (TESDA), the

Professional Regulation Commission (PRC), and the National Telecommunication

Commission (NTC). The Civil Aviation Authority of the Philippines (CAAP) certifies that

PATTS is an Approved Training Organization (ATO), rated for Airframe, Powerplant and

Avionics.
 3

One of the most honored and notable programs offered by PATTS is the Bachelor

of Science in Aeronautical Engineering. The Aeronautical Engineering curriculum of the

college strives to prepare students for technological service and leadership and to make

them adaptable to a variety of challenges in the aviation industry. Graduates of this

profession are actively engaged in the work of planning, design, structure analysis,

construction and repair, supervision, operation of systems and advanced facilities

necessary in the development of the industry. Graduates of this course are presently

employed in the Civil Airworthiness Authorities, Aircraft Operators, Maintenance, Repair

and Overhaul Companies, and other aviation agencies here and abroad.

To the extent of the researcher’s knowledge and extensive literature search, there

is currently no existing study that assess the student learning outcomes at achieving course

level or program specifically in the field of Aeronautical Engineering although some

attempts have been made for various disciplines such as medicine, accounting and other

engineering courses as part of course accreditation. Thus, this study will add to the limited

literature on this field.

Rationale

Higher education institutions are being challenged to show evidence that students

are actually learning and gaining knowledge (Schwass, 2010). Policymakers and the

general public have raised concerns about the quality educational programs and student

learning outcomes, with concerns and issues such as translating into losses in enrollment,

funding, and accreditation (Immerwahr & Johnson, 2010; Schwass, 2010). Stakeholders,

including governmental agencies, are asking whether students are really learning and if
 4

student outcomes data are being used for program improvement purposes. Recent studies

indicate high level of awareness regarding the need to assess SLOs, but that only a few

institutions in the Philippines have fully implemented student learning assessment systems.

In fact, in an extensive literature search of the author, no existing study was found on the

assessment of students learning outcomes for the Aeronautical Engineering Program of

PATTS College despite the fact that PATTS College of Aeronautics is reputedly one of the

premiere colleges in this field.

Understanding what types of learning outcomes are necessary for Aeronautics

Engineering practice and the mechanism by which those learning outcomes are achieved

is fundamental to uncovering why some Aeronautics students are not prepared for their

careers in the aviation industry. It is important, therefore, to assess and evaluate the student

learning outcomes of Aeronautical Engineering Program and to understand how those

outcomes are measured.

The author, who is currently a faculty member of Aeronautics Engineering

Department of PATTS believes that we must re-examine how the classroom is used in

educating future engineers, broadening the focus to include student learning outcomes. It

is important to understand what learning outcomes Aeronautical Engineering students gain

both from classroom environment and internships and the mechanism by which they gain

them. Thus, this research study attempts to address the paucity of research by examining

the SLOs for the Aeronautical Engineering Program of PATTS as evaluated by four groups

of stakeholders, namely: faculty, students, industry employers and AERO Association.

 5

Conceptual Framework

This study is designed to assess the students learning outcomes (SLOs) in

Aeronautical Engineering Program of PATTS from the perspectives of faculty, students,

industry employers and Aero Association. At the end of the study, innovations in

curriculum and instruction will be proposed to improve the Aeronautical Engineering

program.

Figure 1 illustrates the relationships of the variables that will be considered in this

study using the Input-Process-Output Model. In the IPO model, a process is viewed as a

series of boxes (processing elements) connected by inputs and outputs. The IPO model

provides the general structure and guide for the direction of the study.
 6

PROCESS

1. Assessment of SLOs in
Aeronautical
Engineering Program
of PATTS by four
INPUT groups of respondents
as to:
 Instruction OUTPUT
 Aeronautical  Research
Engineering  Community
Program / Involvement Proposed
Curriculum of Innovations in
PATTS 2. Inputs and Feedbacks Curriculum and
of the four groups of
 Knowledge, Skills, Instruction of the
respondents for the
and Attitudes improvement of the Aeronautical
possessed by Aeronautical Engineering
Aeronautical Engineering Program of Program
Engineering PATTS
students of PATTS
at the completion 3. Test of significant
of the course difference in the
assessment of SLOs
among four groups of
respondents

Figure 1. The Conceptual Paradigm

Statement of the Problem

This study will examine the SLOs in Aeronautical Engineering Program of PATTS

as evaluated by four groups of respondents, namely: faculty, students, industry employers

and Aero Association.

 7

Specifically, this will address the following research questions:

1. What are the student learning outcomes (SLOs) for the Aeronautical

Engineering program of PATTS as assessed by faculty, students, aviation

industry and AERO Association in terms of:

1.1 Instruction,

1.2 Research, and

1.3 Community involvement?

2. What are the inputs and feedbacks of the different stakeholder-respondents for

the improvement of Aeronautical Engineering Program of PATTS:

3.1 Faculty,

3.2 Students,

3.3 Aviation industry, and

3.4 AERO Association?

3. Is there a significant difference in the assessment of SLO’s for the Aeronautical

Engineering Program of PATTS among four groups of respondents?

4. What innovations can be adopted to improve the Aeronautical Engineering

program of PATTS?

Objectives of the Study

The main objective of this study is to assess the SLOs in Aeronautical Engineering

Program offered by PATTS from the perspectives of four groups of stakeholders, namely:

faculty, students, industry employers and Aero Association.

 8

Specifically, this aims to:

1. Determine the SLOs for the Aeronautical Engineering program of PATTS as

assessed by faculty, students, aviation industry and AERO Association in terms

of: instruction, research, and community involvement;

2. Determine the inputs and feedbacks of the different stakeholder-respondents for

the improvement of the Aeronautical Engineering program;

3. Ascertain whether there is a significant difference in the assessment of SLO’s

for the Aeronautical Engineering Program of PATTS among four groups of

respondents; and

4. Based on the findings, propose some innovations that can be adopted to improve

the Aeronautical Engineering program of PATTS.

Assumptions

The following assumptions guide this study:

1. The study participants will answer honestly and truthfully to the

questionnaire since anonymity and confidentiality will be preserved and

that the participants are volunteers who may withdraw from the study at any

time and with no ramifications.

2.

Hypothesis

The hypothesis that will be raised and tested is:

 9

Ho1 There is no significant difference in the assessment of student learning

outcomes for the Aeronautical Engineering graduates of PATTS among four groups of

respondents, namely: faculty, students, Aviation industry and AERO Association.

Scope and Delimitation

This study will primarily assess the SLOs for the Aeronautical Engineering

Program of PATTS from the perspectives of four groups of stakeholders. There will be

four groups of study participants, such as: 1) Aeronautical Engineering faculty members

of PATTS; 2) selected Aeronautical Engineering graduates of PATTS; 3) Aviation

industry; and 4) AERO Association. This will utilize a survey questionnaire that will be

prepared by the researcher.

This study will be confined only to the assessment of SLOs for one academic

program of PATTS College, the Aeronautical Engineering program. The results of this

study cannot be generalized to other institutions offering similar academic program nor

since the scope and study area will only be limited to PATTS.

Significance of the Study

The study had significance for future practice, research and policy. This study will

hopefully benefit the following:

Aeronautical Engineering students and graduates. The results of the study

would provide Aeronautical Engineering students and graduates of PATTS with valuable

information that could be used as basis for improving their own skills, knowledge and

attitudes essential to their career in the industry. Likewise, this could lead to a better
 10

understanding of the necessary employability skills they need to succeed in the industry.

This would give them better preparation for adjustment to the industry and for career

success.

Aeronautical Engineering faculty members. As one of the principal instruments

in developing competent graduates of Aeronautical Engineering, this study will assist

Aeronautical Engineering faculty members in the provision of quality instruction, learning

materials and resources, as well as learning activities that will meet the needs of students

and graduates in the job market they are entering. With the findings from this study, faculty

members will be able to identify the skills that recruiters in today’s aviation industry find

important and be able to design coursework and classes that will enable the students to

obtain the skills that are deemed important.

PATTS Administrators. The results of this study will provide administrators with

clear insights into the actual learning outcomes of Aeronautical Engineering Program of

PATTS. This can provide evidence to assist administrators in strengthening the program

designed for Aeronautical Engineering students. Also, this may provide administrators

with data that will serve as basis for faculty development and resources toward improving

the instruction and learning resources so as to ensure that the students will meet the

objectives of the course or program.

Commission of Higher Education. The CHED will hopefully find this study a

valuable source of information in gaining a deeper understanding of the competencies

needed by Aeronautical Engineering students. In this way, they could address these issues

through the enhancement of the curriculum and development of training programs which
 11

would enable students to have a more enabling learning environment that would enhance

their skills and abilities matching their jobs in the industry.

Future Researchers. The results of the study can be used as a reference or basis

for conducting future researches that are related to the present study.

Definition of Terms

For the purposes of readers to have a common understanding of terms that are used

in the study, the following terms are defined operationally based on how these terms are

used in this study:

Aeronautical Engineering Program. This is an undergraduate degree or program

which represents the branch of engineering that deals with the research, design,

development, construction, testing, science and technology of aircraft. The field also covers

investigation into aerodynamic characteristics of aircraft, including behaviors and related

factors such as airfoil, control surfaces, lift and drag. In recent years, aeronautical

engineering has become one of two major and overlapping branches of aerospace

engineering, with astronautical engineering being the second.

Assessment. This refers to any group of methods used to appraise the knowledge,

skills and attitudes of a group of students at the completion of a course of study. For this

study, the focus will be SLOs of Aeronautical Engineering Program of PATTS.

Competence. This refers to a satisfactory level of performance or achievement of

a standard associated with a learning outcome.

Course. A collection of subjects which when completed satisfactorily constitutes a

coherent program of study. In this study, the focus will be the Aeronautical Engineering

course or program.
 12

Evaluation. The drawing of conclusions and making judgments about

performance of individuals or institutions based on the results of measurement and/or

reporting. In this study, the conclusions and judgments will be on the performance of

Aeronautical Engineering graduates of PATTS.

Measurement. This is the assignment of numerical scores or quantifying the

classification of observations about students’ performance in assessment tasks.

Standard. This is an agreed specification or other criterion used as a rule, guideline

or definition of a level of performance or achievement of the student.

Student learning outcomes (SLOs). These are set of skills, abilities and

knowledge that a student has acquired as a result of their higher education experience. In

this study, this is a description of knowledge, skills and abilities of graduates resulting from

the course Aeronautical Engineering program of PATTS.

 13

CHAPTER 2

REVIEW OF RELATED LITERATURE AND STUDIES

This chapter provides a summary of literature review related to the research

questions raised in this study. The study of and research related to assessment and student

learning outcomes is extensive. In this review, theories, articles and previous studies

relating to assessment of student learning outcomes from several areas and courses are

presented.

Definition and Concepts of Learning Outcomes

A common definition of a higher education learning outcome is the personal and

societal changes or benefits that follow from the experience of learning (Chalmers, 2008;

Nusche & OECD, 2008) but often it is assumed that the meaning of the term being used is

obvious. The above definition recognizes the impact of student learning in higher education

both on the individual and on society as a whole. Similarly, in the more recent Degree

Qualifications Profile (Ewell, Gaston, & Schneider, 2011) developed by the National

Institute for Learning Outcomes Assessment (NILOA), learning outcomes also include

attributes relating to external impacts and are defined as ‘information and skills that
 14

graduates need for work, citizenship, global participation and life’ which are focused on

life after study.

Shavelson’s (2009) learning outcome focuses on personal growth and defines it as

a ‘permanent change in observable behavior over time’. This is a behaviorist definition

from his perspective as a psychometric analyst. It underpins the conceptual framework he

has devised which describes the nature and relationship between types of learning

outcomes. In contrast work done in the UK by Otter uses an operational definition (1992

cited in Martin, 2016) that a learning outcome is a description of a course objective with a

clear statement of assessment requirements is far more pragmatic. The more recent

European Tuning process also links the learning outcome to a standard of achievement and

describes it as the extent and the level of standard of competence including knowledge that

a student will develop by graduation (Lennon et al., 2014). This suggests the need for

distinction between the concepts of standards, competence, achievement and outcomes as

these terms are often blurred with ‘competences’, sometimes used interchangeably with

learning outcomes.

Melton (cited in Martin, 2016) suggests a number of issues of principle that should

be considered when devising sets of learning outcomes. A primary concern is whether a

‘one size fits all’ approach to defining learning outcomes and standards at the institutional

or discipline level is possible or appropriate. His other issues include whether the focus is

on individual learning or the needs of others such as Government and the professions; the

need for a classification scheme or taxonomy for the set of learning outcomes to allow

direct comparison between different entities; and the need for national standards for
 15

measurement of the learning outcomes to allow for comparability of attainment across

institutions and disciplines.

Origins and Feasibility of Specifying Learning Outcomes in Higher Education

There are various views about the origins of the use of learning outcomes in higher

education. Nusche and Organisation for Economic Cooperation and Development, (2008)

states that ‘the term learning outcomes has its origin in outcomes-based education’ in which

curriculum and assessment are both organized around statements of anticipated learning

outcomes. The link between assessment and learning outcome description is a key factor

in the feasibility of the measurement of learning outcomes at the center of this research

study. Nusche’s view is that provided the alignment between the statement of learning

outcomes and the modes of assessment is achieved comparative measurement of learning

outcomes becomes feasible (Nusche & OECD, 2008).

Melton (1996 cited in Martin, 2016) argues that the development of learning

outcomes in higher education arose from the specification of competencies in applied and

vocational courses. He considers that when higher education expanded and changed with

the emergence of broader curriculum and much less obvious career paths for university

graduates on qualification, it became necessary to think about the more general knowledge

and skills needed for university graduates to enter any of a wide range of professional areas

as employees. This led to the desire in universities to ensure graduates developed what

Melton calls ‘core skills’ of ‘problem solving, communication, learning to learn, working

with others, numeracy, information technology, competency in a foreign language’. These

are usually referred to as generic skills.

 16

More recently Ewell (2010) alludes to the growing competition of a global higher

education market for students and funding as drivers for the ongoing need to specify

learning outcomes accurately. He argues that this competition results in greater mobility of

graduates in search of employment and further study options in which they will seek credit

for work undertaken in different jurisdictions. In his view this has led to various

mechanisms which are ‘manifestations of the global standards movement’. Ewell quotes

quality assurance networks such as the Bologna process, the increasing prevalence of

international university rankings, alignment efforts for standards such as the Tuning profile

and approaches to international comparative assessment such as the AHELO project

sponsored by the OECD, as evidence of an increasing need to specify graduate learning

outcomes effectively and measure them accurately. The ongoing focus on learning

outcomes as an approach to quality assurance of teaching and learning in higher education

has a number of positives and negatives.

Ewell (2010) believes that being able to compare outcomes and measure them to

allow comparisons across populations and different settings is a significant positive,

against which he sets a number of pitfalls which include: a) conceptual and operational

difficulties in defining the outcomes; b) Questions from traditional academic staff about

the legitimacy of the exercise; c) Potential to lose important distinctions between the

programs offered by different institutions; d) Potential to lose the sense of the whole by

defining a list of abilities and knowledge rather than looking at the overall interactions of

the parts; and e) The problem of not being able to specify all important outcomes in advance

(Ewell, 2010).
 17

According to Martin (2016), the achievement of a set of course learning outcomes

is the result of complex interactions between expected knowledge and skills executed

through a range of individually taught subjects, many of which are discipline specific. Sets

of learning outcomes also usually include a range of personal attributes expected of

graduates, as identified by Melton and others (cited in Martin, 2016), but in some cases

these may not be directly the result of the higher education experience. This presents one

of the difficulties in the precise specification of such outcomes and their measurement

which must differentiate between what has been learned at a higher education institution

and what other skills or personal characteristics arise solely from individual development

or maturation.

Martin (2016) further explained that while it may be possible to describe a

comprehensive set of learning outcomes which includes knowledge, skills and personal

attributes, only some may be related to the education process, and even fewer will satisfy

the requirement of alignment with assessment.

Learning Outcomes for Engineering Graduates

Learning outcomes for Engineering majors include both discipline-specific and

broad educational outcomes necessary for modern-day practice. Though similar, learning

outcomes across Science, Technology, Engineering and Math (STEM) do reflect

differences in fundamental knowledge and application. Math and computer technology, for

example, are hierarchical in nature, while engineering integrates knowledge of math and

science into application-based outcomes (Council for Higher Education Accreditation,

2010).
 18

According to Hirleman, Groll and Atkinson (2007 mentioned in Brooks, 2016) that

the earliest engineers were trained through apprenticeships and much of the apprentice

style of training remained in the early pedagogy of higher education. In the 1950s concerns

for national security brought on by World War II and the launching of Sputnik shifted

engineering education from a focus on practical training to a focus on research and the

development of new technologies, especially in the defense and space programs. Over the

next 30 years, industry needs began to shift as a result of a growing national

competitiveness within the context of a more global economy (Lucena, et al., 2008). In

1985 the National Research Council sounded the call for engineers to heighten their

professional skills, including communication, teamwork, and developing an understanding

of economic and societal impacts on the engineering profession (National Research

Council, 1985 mentioned in Brooks, 2016). By the mid 1990’s industry and government

officials recognized a disconnect between the skills that engineering graduates were

learning through formal engineering education and those needed to compete in the

engineering workforce (Duderstadt, 2008 in Brooks, 2016).

In 1994, the Accreditation Board for Engineering and Technology (ABET), with

financial support from the National Science Foundation, held a workshop consisting of

representatives from industry, government, and academia. At the workshop,

representatives discussed the disconnect between education and practice, acknowledging

that the accreditation criteria used to evaluate engineering schools were too rigid and

prescriptive to meet the rapidly changing technical environment that engineering graduates

would be entering (Lang, Cruse, McVey, McMasters, 1999 cited in Brooks, 2016). As a

result of these discussions, ABET revised the accreditation criteria to reflect a learning
 19

outcomes-based approach that was more holistic and consisted of eleven outcomes known

collectively as the ABET (a-k) outcomes (ABET, 2012). The ABET (a-k) learning

outcomes represent both technical and non-technical competencies that engineering

students need to demonstrate including: (a) Ability to apply knowledge of mathematics,

science, and engineering; (b) Ability to design and conduct experiments, as well as to

analyze and interpret data; (c) Ability to design a system, component, or process to meet

desired needs; (d) Ability to function on multi-disciplinary teams; (e) Ability to identify,

formulate, and solve engineering problems; (f) An understanding of professional and

ethical responsibility; (g) Ability to communicate effectively; (h) Broad education

necessary to understand the impact of engineering solutions in a global/societal context; (i)

Recognition of the need for, and an ability to engage in life-long learning; (j) Knowledge

of contemporary issues; and (k) Ability to use the techniques, skills, and modern

engineering tools necessary for engineering practice (ABET, 2012).

The criteria, known as Engineering Criteria 2000 (EC2000), represented a radical

change in the approach to engineering education, and remain in place at all U.S. accredited

engineering programs. The criteria also serve as a guideline for engineering programs

worldwide (Hirleman, Groll, & Atkinson, 2007 cited in Brooks, 2016).

While the EC2000 shifted the emphasis from curricula inputs to student learning

outcomes, each institution is responsible for determining how the outcomes are achieved

and assessed (Lang, et al., 1999 cited in Brooks, 2016). The criteria are intentionally

flexible in order to allow individual institutions to utilize their unique capacities in

designing the overall educational experiences necessary to achieve these eleven outcomes.
 20

In 2015, the ABET Engineering Accreditation Commission (EAC) proposed

changes to Criterion 3 as a result of perceived shortcomings in both application and

interpretation of the a-k learning outcomes. The proposed changes reduced the learning

outcomes from eleven to seven outcomes that include: 1) engineering problem solving, 2)

engineering design, 3) measurement, testing and quality assurance, 4) communication

skills, 5) professional responsibility, 6) professional growth, and 7) teamwork and project

management skills (ABET, 2015). The EAC task force concluded that, “some of the (a)-

(k) components were interdependent, broad and vague in scope, or impossible to measure.

As a consequence, program evaluators were inconsistent in their interpretation of how well

programs were complying with Criterion 3” (ABET, 2015). Critics of the proposed changes

argue that the new outcomes will “lower the bar” in engineering education by diminishing

the importance of educational breadth, especially related to having an understanding of

contemporary issues, professional ethics, and working in a multidisciplinary environment

(Riley, 2016).

Previous Studies on the Assessment of Student Learning Outcomes Assessment

Several studies examining the assessment of student learning outcomes were found

in the literature. For example, in a 2009 paper published by the National Institute for

Learning Outcomes Assessment (NILOA), Kuh and Ikenberry reported results from a

national survey of 1500 high level administrators including provosts and chief academic

officers. The study found that most colleges and universities in the United States identify

learning outcomes for their students that include both program-specific and institution-

level assessments. Kuh and Ikenberry (2009) also found out that gaining faculty
 21

involvement and support in assessing learning outcomes remains a major challenge.

Campuses would also like more assessment expertise, resources, and tools”. It was also

found out that institutions and faculty would be more likely to survive the accountability

movement if transparency were promoted and the value of assessing student learning was

apparent. Key findings from the 2009 survey related to faculty engagement include: a) in

order to effectively assess student learning outcomes, 66% of schools indicated the need

for more faculty engagement, and b) about four-fifths of provosts at doctoral research

universities reported greater faculty engagement as their number one challenge. The

authors conclude that assessment of student learning outcomes in higher education remains

a work in progress ((Kuh & Ikenberry, 2009).

A recent study of Rose (2015) on faculty perceptions on the student learning

accountability movement using a qualitative survey of 140 instructional faculty and from

21 semi-structured interviews with instructional faculty, accountability specialists, and

state coordinating board officials showed that faculty members perceived that student

learning assessment requirements have thought of the assessment in a myriad of ways. The

majority of faculty agreed that the purpose of student learning assessment is to improve

learning or to help students achieve competency, whereas, a few faculty indicated that the

purpose was to evaluate teaching. Moreover, survey respondents indicated that student

learning assessment at UK reflects a commitment to competency, improvement, and

quality education. Faculty members acknowledge the importance and meaning behind

assessment, they felt that the institutions requirements were just adequate at best.

In 2007, Passow conducted a meta-analysis of ten different studies representing

almost 6,000 engineers to determine what the relative emphasis on the various learning
 22

outcomes should be. She used the ABET (a-k) outcomes as the common construct for the

study, mapping the various competencies from each of the ten studies onto the ABET

outcomes, and listing separately those competencies that were not considered comparable.

Since Passow (2007) was interested in determining the relative emphasis among the

competencies rather than the absolute importance ratings (using a Likert scale), she chose

to compare the mean value of specific competencies to the overall mean value of the ABET

(a-k) outcomes in a given study. This method resulted in the rank ordering of the

importance of each of the competencies including (in order): problem-solving,

communication, ethics, life-long learning, experiments, teams, use of engineering tools,

and design as the most important competencies; knowledge of math, science and

engineering as being of average importance; knowledge of contemporary issues and impact

as being of least importance (Passow, 2007). In addition, she identified two additional

competencies that were not part of the ABET (a-k) outcomes that ranked between the top

two levels of importance: decision-making and commitment to achieving goals. The results

of this meta-analysis indicate that competencies that include abilities, attitude and skills are

considered to be more important than strictly technical knowledge (Passow, 2007).

 23

CHAPTER 3

RESEARCH METHODOLOGY

This chapter discusses the research design that will be applied in this study, the

study respondents, the research instrument that will be used, the procedure to be employed

in data gathering, and the statistical treatment of data.

Research Design

The descriptive research design utilizing quantitative method will be used to assess

the student learning outcomes for the Aeronautical Engineering Program of PATTS from

the perspectives of faculty members, students, industry employers and Aero Association.

According to Creswell (2010), a descriptive study consists of a set of gathered data or

information, which were analyzed, summarized and interpreted along certain lines of

thought for the pursuit of a specific purpose or study. Meanwhile, a quantitative approach

is context free and the intent is to develop generalization, relying heavily upon statistical

results represented with numbers and is done to determine relationships, effects, and

causes.

Respondents of the Study

There were four (4) groups of respondents in this study, namely: 1) all Aeronautical

Engineering faculty members of PATTS; 2) selected fifth year Aeronautical Engineering

students of PATTS; 3) 20 aviation industry employers; and 4) 50 members of Aero

Association. Table 1 shows the distribution of respondents for this study.

 24

Table 1

Distribution of respondents for the study

Number of
Group Percentage
Respondents
Faculty members of Aeronautical
11 6.08
Engineering Department of PATTS
Fifth year Aeronautical Engineering
100 55.25
students of PATTS
Industry employers and practitioners 20 11.05

SAEP members 50 27.62

TOTAL 181 100.00

The first group of respondents will be composed of faculty members from

Aeronautical Engineering Department of PATTS with a total of 11 faculty members as of

academic year 2019-2020. The student-respondents will be composed of 100 selected fifth

year Aeronautical Engineering students of PATTS enrolled during academic year 2019-

2020. The third group will be 20 aviation industry employers represented by owners,

managers and industry practitioners. The last group of respondents will be 50 members of

the Society of Aerospace Engineers of the Philippines (SAEP) which is the only recognized

AERO Association in the country. Table 1 shows the distribution of respondents by group.

Instrument

A semi-structured questionnaire prepared by the researcher will be used in

gathering information from the respondents. The survey instrument shown in Appendix 1

is constructed by the researcher based on the research questions developed in this study.

The questionnaire is divided into four parts: Part1 – Demographic Background; Part 2-
 25

Assessment of SLOs for the Aeronautical Engineering Program of PATTS; and Part 3 –

Inputs and Feedbacks for the Improvement of Aeronautical Engineering program.

Validation of Instrument. To ascertain the validity and reliability of the survey

instrument, a pre-test will be done using 20 students of Aeronautical Engineering of

PATTS not included in the sample. Thereafter, the pre-test survey questionnaires will be

tested using Cronbach alpaha (α). All Cronbach α coefficients for all scales that will be

used in the questionnaire will be computed by the statistician and all values greater than

0.7 will be considered acceptable for conducting research.

Moreover, the researcher will seek the help of graduate faculty members from the

graduate school of Philippine State College of Aeronautics who are experts in the field of

student learning outcomes to validate the survey instrument. The suggestions and

comments of these experts will be considered in the final draft of the instrument.

Procedure in Data Gathering

Prior to study initiation, permission to gather data and conduct the study will be

secured from the President of PATTS and the chairperson of the Aeronautical Engineering

Department. Thereafter, the researcher will secure permission to conduct the study from

the aviation industry managers and practitioners as well as from the President of SAEP.

After permission is granted, the target respondents will be notified of the study and will be

asked for their participation in this research.

Respondents will be asked to complete the questionnaire in approximately 15-20

minutes. Prior to completing the questionnaire, respondents will be given a brief summary

of the purpose of the study and will then be asked to complete the questionnaire. The
 26

respondents will be informed that by completing the questionnaire, they are providing their

consent to participate.

Statistical Treatment

The data that will be collected will be analyzed with the use of quantitative research

methods. The data will be examined using descriptive statistical tools to answer the

research questions raised in this study.

Descriptive statistical tools such as: frequency distribution, percentages, range,

mean and standard deviation will be computed to describe the assessment of student

learning outcomes (SLOs) in Aeronautical Engineering Program of PATTS, and the inputs

and feedbacks of the following stakeholders for the improvement of Aeronautical

Engineering program as per research questions 1 and 2.

In order to address research question 3, one-way Analysis of Variance or F-test will

be used to determine whether there is a significant difference in the assessment of SLO’s

of Aeronautical Engineering Program of PATTS among four groups of respondents. The

significance level will be set at five percent.

Finally, based on the findings and results of the study, innovations in curriculum

and instruction will be proposed by the researcher that can be adopted to improve the

Aeronautical Engineering program.

 27

REFERENCES

ABET - Accreditation Board for Engineering and Technology. (2014). Criteria for
accrediting engineering programs. Baltimore, MD.
ABET - Accreditation Broad for Engineering and Technology. (2015). Rationale for
revising Criteria 3 and 5. Retried from
http://www.abet.org/accreditation/accreditationcriteria/accreditation-
alerts/rationale-for-revising-criteria-3/
Brooks, C. L. (2016). Understanding STEM learning outcomes using a phenomenographic
approach. (2016). Doctoral Dissertations May 2014 - current. 833. Retrieved from
http://scholarworks.umass.edu/dissertations_2/833
Chalmers, D. (2008). Review of Australian and national performance indicators and
measures of the quality of teaching and learning in higher education. Canberra,
Australia: Australian Learning and Teaching Council.
Council for Higher Education Accreditation. (2010). The value of accreditation.
Washington, D.C.
Ewell, P.T. (2010). Conversations about Standards and Institutional Funding. L H Martin
Professional Development Session. Melbourne, Australia.
Ewell, P.T., Gaston, P., & Schneider, C.G. (2011). The Degree Qualifications Profile.
Indianapolis, United States: Lumina Foundation.
Kuh, G., & Ikenberry, S. (2009). More than you think, less than we need: Learning
outcomes assessment in American Higher Education. Urbana, IL: University of
Illinois and Indiana University, National Institute for Learning Outcomes
Assessment (NILOA).
Lennon, M. C., Frank, B., Humphreys, J., Lenton, R., Madsen, K., Momri, A., et al. (2014).
Tuning: Identifying and measuring sector-based learning outcomes in post-
secondary education. Toronto, Canada: Higher Education Quality Council of
Ontario (HEQCO).
Lucena, J., Downey, G., Jesiek, B., & Elber, S. (2008). Competencies beyond countries:
The reorganization of engineering education in the United States, Europe, and
Latin America. Journal of Engineering Education, 433-447.
Martin, L. M. (2016). Using assessment of student learning outcomes to measure
university performance: towards a viable model. (Dissertation). Melbourne Centre
for the Study of Higher Education Melbourne Graduate School of Education,
University of Melbourne.
 28

Nusche, D. & Organisation for Economic Cooperation and Development. (2008).

Assessment of Learning Outcomes in Higher Education: A Comparative Review of
Selected Practices. OECD Education Working Papers, No. 15: OECD Publishing.
Passow, H. J. (2007). What competencies should undergraduate engineering programs
emphasize? A dilemma of curricular design that practitioners’ opinions can
inform. Published in proceedings from the 3rd International CDIO Conference.
Cambridge, MA
Riley, D. (2016). Against ABET: Defending the broad education of engineers. Retrieved
from https://aabet.org/2016/02/
Rose, T. (2015) Faculty perceptions on the student learning accountability movement.
(Doctoral Dissertation) Walden University. Retrieved from
http://scholarworks.waldenu.edu/cgi/viewcontent.cgi?article=3031&context=diss
ertations
Saint Germain, M. A. (2008). Challenges to Higher Education in the 21st Century. Paper
presented at the annual meeting of the Western Political Science Association,
Manchester Hyatt, San Diego, California. Retrieved from
http://www.allacademic.com/meta/p238373 index.html
Shavelson, R. (2009). Measuring college learning responsibly: accountability in a new
era. Palo Alto: Stanford University Press.
Wegner, G. (2008). Partnerships for public purposes: engaging higher education in
societal challenges of the 21s ' century (April, 2008, #07-5). San Jose, CA: The
National Center for Public Policy and Higher Education.
Schwass, C. M. (2010). Assessing student learning outcomes in AQIP accredited
community and technical colleges (Order No. 3470411). Available from Education
Database. (743814748). Retrieved from
https://search.proquest.com/docview/743814748?accountid=38643

Immerwahr, J. & Johnson, J. (2010). Squeeze play 2010: continued public anxiety oncost,
harsher judgments on how colleges are run. San Jose, CA: A Joint Project of The
National Center for Public Policy and Higher Education and Public Agenda.
 29

Appendix 1

QUESTIONNAIRE

Part 1 – Demographic Background

Instruction: Please provide the necessary information. Check (√) your answer on the
box provided or write your response on the space provided.

1. Name: _____________________________________________________
(Optional)
2. Respondent group:
( ) Faculty
( ) Aero Engineering student
( ) SAEP member
( ) Aviation industry representative, pls specify occupation __________________

3. Age:
( ) 61 – 65 years old ( ) 31 – 40 years old
( ) 51 – 60 years old ( ) 21 – 30 years old
( ) 41 – 50 years old

4. Gender: ( ) Male ( ) Female

5. Educational Attainment:

( ) College level

( ) BS/AB Degree, please specify degree/major field

__________________________________

( ) Master's Level, please specify degree/major field

__________________________________

( ) Master's Degree, please specify degree/major field

__________________________________

( ) Doctoral Level, please specify degree/major field

__________________________________
 30

Part 2- Assessment of the Student Learning Outcomes in

Aeronautical Engineering Program of PATTS

Instructions: The statements listed on the table are the student learning outcomes
(SLO) for the Aeronautical Engineering Program of PATTs. In each SLO, please provide
two responses by checking one response in each of the two set of columns. Below is the
scale of responses for your guidance.

In the LEFT column, indicate your expected student learning outcome for the
Aeronautical Engineering Program of PATTS. The response scale for this column is as
follows:

Level of Expectation
5 = Very High
4 = High
3 = Moderate
2 = Fair
1 = Low

In the RIGHT column, indicate the extent to which each of this expectation is met
by the graduate of Aeronautical Engineering Program. The response scale for this
column is as follows:

Extent to which the Expectation is met

5 = Very Great Extent
4 = Great Extent
3 = Moderate Extent
2 = Little Extent
1 = Very Little Extent
 31

Extent to which
Level of
expectation
STUDENT LEARNING OUTCOMES Expectation
is met
5 4 3 2 1 5 4 3 2 1
Instruction
On successful completion of the
Aeronautical Engineering program, students
should be able to:
1) apply knowledge of mathematics, physical
sciences, engineering sciences to the practice
of aeronautical engineering.
2) demonstrate the practical engineering skills to
carry out technical work in both laboratories
and workshops
3) sesign, improve, innovate, and to supervise
systems or processes to meet desired needs
within realistic constraints.
4) identify, formulate, and apply the knowledge
of mathematics, science, engineering
fundamentals to the solution of engineering
problems.
5) utilize the knowledge of
aeronautical/aerospace engineering in
innovative, dynamic and challenging
environment for design and development of
new products
6) use the techniques, skills, and modern
engineering tools necessary for engineering
practice.
7) work effectively in multi-disciplinary and
multi-cultural teams in diverse fields of
practice.
8) apply quantitative technical tools and
demonstrate the ability to provide novel
solutions to aeronautical problems,
particularly in the design of aircraft
9) be familiar with emerging technologies for
aircraft design and analysis
10) demonstrate the professional and ethical
responsibility and norms of the engineering
practice.
 32

Extent to which
Level of
expectation
STUDENT LEARNING OUTCOMES Expectation
is met
5 4 3 2 1 5 4 3 2 1
Research
On successful completion of the Aeronautical
Engineering program, students should be able to:
11) Conduct and complete research at colleges /
universities and in work settings.
12) Design and conduct experiments to test
hypotheses and verify assumptions, as well as
to analyze and interpret data and to simulate
processes.
13) participate in the generation of new knowledge
or in research and development projects
14) use research-based knowledge and research
methods to design and conduct experiments,
as well as to analyze and interpret data.
15) identify, formulate, review research literature,
and analyze complex engineering problems
reaching substantiated conclusions using first
principles of mathematics, natural sciences,
and engineering sciences.

Community Involvement
On successful completion of the Aeronautical
Engineering program, students should be able
to:
16) understand the effects and impact of the
aeronautical engineering profession on the
environment and the society,
17) apply acquired aeronautical engineering
knowledge and skills for community and
national development.
18) participate in various types of development
activities, and public discourses, particularly
in response to the needs of the communities
he/she serves
19) participate in professional societies and
community service.
20) apply knowledge to assess societal, health,
safety, and cultural issues and the consequent
responsibilities relevant to the professional
engineering practice.
 33

Part 3 - Inputs and Feedbacks for the Improvement of Aeronautical Engineering

Program

1. Based on your assessment of the SLOs of Aeronautical Engineering Program, what

suggestions can you propose to improve the curriculum of the Aeronautical
Engineering Program of PATTS? Please list below:
__________________________________________________________________
__________________________________________________________________
_________________________________________________________________
__________________________________________________________________

2. Based on your assessment of the SLOs of Aeronautical Engineering Program, what

suggestions can you propose to improve the instruction of the Aeronautical
Engineering Program of PATTS? Please list below:

__________________________________________________________________
__________________________________________________________________
_________________________________________________________________
__________________________________________________________________

3. What innovations can you propose for the improvement of the Aeronautical
Engineering Program?

__________________________________________________________________
__________________________________________________________________
_________________________________________________________________
__________________________________________________________________

Thank you very much for your participation.