102 PROJECT WRITE Setting new benchmarks for teacher education is no mean task. This is so in the instance of Project WRITE (Writing Resources for Integrative Teacher Education) which we initiated on August 4, 2006 culmi- nating in the production and publication of workbooks and textbooks. It was'a yeoman’s task to gather our colleagues from Luzon, Visayas, and Mindanao and motivate them to get involved in a textbook writing process. But the best happened such that we were able to form a true partnership as proven by this WRITE publication that may help bring teacher education to new heights This book on Advanced Methods in Educational Assessment and Evaluation Assessment of Learning 2 aims to align teacher education with the CHED- prescribed new teacher education curriculum that is reflective of the National Competency-Based Teacher Standards with the end in view of contributing to the formation of the ideal professional teacher as concretely described in the seven integrated domains of the teach- ing and learning process. The activities or exercises at the end of every lesson were intended not only for drill and evaluative purposes but also for purposes of ad- vocacy for current trends in education such as reflective education, multiple intelligences, multicultural education, in- teractive and brain-based teaching, authentic assessment and of bridging the gap between theory and practice. We gratefully acknowledge Dr. Rosita G Santos, for her desire and concer to share her expertise in the teaching profession. We are glad to say that this Project WRITE publication is but a beginning, With confidence, we look forward to more publications that will do justice to the prescribed new teacher education curriculum and improve teacher performance and advance the enhance- ment of both the pre-service and in-service phases of, teacher education Onward to WRITE! Paz I. Lucido, Ph.D. President, PAFTECChapler7 ~ easing and Roparting References De Fina, A. 1992. Portfolio Assessment: Getting Started. Scho: last. Grabe, W. & R. Kaplan, 1996. Theory and Practice of Writing ‘Addison Wesley, Longman. Hedge, T. 1988. Writing. Oxford University Press. _ Johnson, K.E. 1996. “Portfolio assessment in second language teacher education,” TESOL Journal, Volume 2. O'Malley, JM. & L. Valdez Pierce. 1996. Authentic Assessment for English Language Learners. Addison-Wesley Publishing Company, Paulson F.L., PR. Paulson and C.A. Meyer. 1991. “What makes «8 portfolio a portfolio?” Educational Leadership. February. Puhl, C.A. 1997, “Develop, not judge: Continuous assessment in the ESL classroom.” Forum 35:2. pp. 2-9. Santos, M.G. 1997. “Portfolio assessment and the role of leaner reflection.” Forum 35:2. pp.10-16. Tierney R.J., M.A. Carter and L.E. Desai. 1991. Portfolio As- sessment in the Reading-Writing Classroom. Norwood: Christopher Gordon Publishers. Toperoff, Debby. 1995. Portfolio Assessment in Literature Teaching. M.A. Dissertation. Surrey University. Wolf, K. & Y. sib 30-36. DAC Network on Development Evalu uu-Runyan. 1996, Portfolio purposes and pos- -s, Evimal of Adolescent & Adult Literacy 40:1. pp. iuiding Prine House, E. R. (1978) “Assumptions underlying evaluation moc- els.” Educational Researcher. 1(3), 4-12. Stufflebeam, D. L., & Webster, W. J. (1986). “An analysis of el- temative approaches to evaluation.” Educational Evaluation and Policy Analysis. 2(3), 5-19. 101100 Aowncee Merioosw EowcaowatAssesouex m0 Eisen (Assrssuex oF Leatnc2} : 6. When using absolute grading, the standard for passing should be predetermined LEARNING GOAL: Distinguishes between desirable and undesir- able practices in conducting a parent-teacher conference. Directions: Indicate :vhether each of the following statements describes a DESIRABLE practice or an UNDESIRABLE practice in conducting, parent-teacher conferences. 1, Before the conference, assemble a portfolio of specific infor- mation about and examples of the student’s learning progress. 2. Present examples of the student's work to parents. 3. Begin the conference by describing the student's learning difficulties. 4, Make clear to parents that, as a teacher, you know what is best for the student’s learning and development. 5. In the concluding phase, review your conference notes with the parents. 6. End the conference with a po: student ve comment about the100 Aowcen Merieosm EoucaTons Assess no Ex (ASSESSUENTF LesONG 2) 1 ‘When using absolute grading, the standard for passing should be predetermined. LEARNING GOAL: Distinguishes between desirable and undesir- able practices in conducting a parent-teacher conference. Directions: Indicate -vhether each of the following statements describes a DESIRABLE practice or an UNDESIRABLE practice in conducting, parent-teacher conferences, Before the conference, assemble a portfolio of specific infor- mation about and examples of the student’s learning progress. Present examples of the student's work to parents. Begin the conference by describing the student’s lea difficulties, Make clear to parents that, as a teacher, you know what is best for the student’s learning and development. In the concluding phase, review your conference notes with the parents, End the conference with a positive comment about the student.Chapter? ~ Grading and Reporting D. Parent-teacher conference LEARNING GOAL: Distinguishes between desirable and unde- sirable practices in assigning relative letter grades. Directions: Indicate whether each of the following statements describes a DESIRABLE practice or an UNDESIRABLE practice in assigning relative letter grades on the blank. Put a D if the practice is desirable and a U if not. 1. The grades should reflect the learning outcomes speci- fied for the course. 2. To give test scores equal weight in a composite score, the scores should be simply added together. If it is decided to assign different weights to some scores, the weighting should be based on the maxi- mum possible score on the test. 4. Grades should be lowered for tardiness or misbehavior. 5 . Grading typically should be based on the normal curve. [Hint] 6 . Pass-fail decisions should be based on an absolute standard of achievement. LEARNING GOAL: \guishes between desirable and unde- sirable practices in assigning absolute grades. Directions: Indicate whether each of the following statements describes a DESIRABLE practice or an UNDESIRABLE practice in assigning absolute letter grades. Write D for Desirable and U for Undersirable. 1. Absolute grades should be used with mastery learning, 2. Clearly defined domains of learning tasks should pro- vide the basis for grading. 3. If all students pass a test, a harder test should be given before grades are assigned, The distribution of grades to be assigned should be predetermined and explained. 5. Grades should be based on the amount of improve- ment shown. 9998 ‘Abwurcen Merion» Eoucaras ASSESSUEXT NO EUUATEN (ASSESEIEN OF LEAWNG 2) 1 best fits each statement listed below. . Provides for two-way reporting. ‘A. Traditional letter grade (A B. Tworletter grade (pass, fail) C. Checklist of objectives DF) D. Parent-teacher conference . Provides most useful learning guide to student. ‘A. Traditional letter grade (A,B,C,D,F) B, Tworletter grade (pass, C. Checklist of objectives D. Parent-teacher conference . Provides least information conceming leaming, ‘A. Traditional letter grade (A,B,C.D.F) B. Two-letter grade (pass, fail) C. Checklist of objectives D, Parent-teacher conference « Most preferred by college admissions officers. A, Traditional letter grade (A,B,C,D,F) B. Two-letter grade (pass, fail) C. Checklist of objectives D. Parent-teacher conference May be too complex to be understood by parents, A. Traditional letter grade (A,B, B. Tworletter grade (pass, fail C. Checklist of objectives D. Parent-teacher conference « Most widely used method of reporting in high school. ‘A. Traditional letter grade (A,B,C,D,F) B, Tworletter grade (pass, fail) C. Checklist of objectivesCheplr 7 ~ Grading and Reporting carefully planned and the teacher is skilled in handling such confer- ences. Skills in conducting parent-teacher conferences can be deve- loped. Here are some hints on conducting good conferences: Guidelines for a good conference 1. Make plans + Review your goal + Organize the information to present. + Make list of points to cover and questions to ask. + If bring portfolios, select and review carefully. 2. Start positive—and maintain a positive focus. + Present student's strong points first. + Be helpful to have example of work to show strengths and needs. + Compare early vs. later work to show improvement. 4, Encourage parents to participate and share information. + Be willing to listen, + Be willing to answer questions. 5. Plan actions cooperatively. + What steps you can each take? + Summarize at the end. 6. End with positive comment, + Should not be a vague ‘generality. + Should be true. 7. Use good human relations skills CHAPTER EXERCISES Directions: Indicate which type of marking and reporting system 796 Amc Menicos m Enucaron Assessuent no Ewuinnon (ASSESENT oF ARNG 2) On the other hand, the criterion-referenced or absolute (have assessed absolute levels of knowledge) grading system is not an issue because: a. it seldom uses letter grades alone b. it often includes checklists of what has been mastered c. the distribution of grades is not predetermined Guidelines for Effective Gra es The following guidelines for effective grading and reporting are recommended for use in Philippine schools: 1. Describe grading procedures to students at beginning of instruction. 2. Clarify that course grade will be based on achievement only. 3. Explain how other factors (effort, work habits, etc.) will be reported. 4, Relate grading procedures to intended learning outcomes. 5. Obtain valid evidence (tests, etc.) for assigning grades. 6. Try to prevent cheating. 7, Return and review all test results as soon as possible. 8. Properly weight the various types of achievements included in the grade. 9. Do not lower an achievement grade for tardiness, weak effort, or misbehavior. 10. Be fair. Avoid bias. When in doubt, review the evidence. If still in doubt, give the higher grade. 7.7 Conducting Parent-Teacher Conferences Parent-teacher conferences become productive when they are(Chapter ~ Grating and Reporting a. grade does NOT depend on what group you are in, but only on your own performance compared to a set of performance standards, . grading is a complex task, because grades must: i. clearly define the domain. clearly define and justify the performance standards, iii be based on criterion-referenced assessment. ¢. conditions are hard to meet except in complete mastery learning settings. Finally, grades may also reflect learning ability or improvement performance i.e. score compared to learning “potential” or past perfor- mance, In such a system: a, grades are inconsistent with a standards-based system be- cause now, each child is his/her own standard. b, reliably estimating learning ability (separate from achievement) very difficult, €. one cannot reliably measure change with classroom mea- sures, 4d, therefore, should only be used as a supplement. 76 Distribution of Grades and Guidelines for Effective Grading How should grades be distributed? The norm-referenced and criterion referenced distribution of grades have been in practice for several years now. The norm-referenced or relative (have ranked the students) distribution is a big issue because of the following con- siderations: a, normal curve is defen: unselected group b. when “grading on the curve,” school staff should set fair ranges of grades for different groups and courses c.” when “grading on the curve,” any pass-fail decision should be based on an absolute standard (i.e., failed the minimum essentials) 4. standards and ranges should be understood and followed by le only when we have large, all teachers 95 Grades may aiso reflect absolute perfomance ie ‘score compared to specified performance standards (what ‘you can do). This is more ‘commonly called a citeon- referenced grating‘Rowen Merion n Eovcaronas Asses ano Ewen (Aorssven oF Letrne2) 94 7.4 Assigning Letter Grades and Computing Grades Grades assigned to students must include only achievement. It is very important to avoid the temptation to include effort for less able students because it is difficult to assess effort or potential and it is difficult to distinguish ability from achievement. Furthermore, if achievement and effort are combined in some way, grades would mean different things for different individuals. Grades reflected on report cards are numbers or numerical quan- tities arrived at after several data on the students” performance are combined. The following guidelines may be considered in combining such data: * Properly weight each component to create a composite. The weights used are normally agreed upon by the school offi- cials e.g. how many percent goes for quizzes , unit tests, periodic tests ete. The more scientific approach is to use a principal components analysis which is hardly practiced in schools because of the difficulty involved, + Put all components on same scale to weight properly a. equate ranges of scores bb. oF, convert all to T-scores or other standard scores 7.5 Norm or Criterion-Referenced Grading Grades may reflect relative performance i.e. score compared to other students (where you rank). This is more commonly called a norm-referenced grading system. In such a system: Grades may raft relative performance i. score compared a, grade (like a class rank) depends on what group you are in, to other students not just your own performance. {where you rank). b. typical grade may be shifted up or down, depending on Ts fs move group’s ability a jely used because much classroom testing is norm- crading sytem referenced Grades may also reflect absolute performance i.e. score compared to specified performance standards (what you can do). This is more commonly called a criterion-referenced grading. In such a system:7.3 Development of a Grading and Reporting System Grading and reporting systems are guided by the functions to be served by such in the educative process. The system will most probably be a compromise because of the hosts of factors to be con- sidered in the preparation of such. The important thing to note, however, is that we should always keep achievement reports separate from effort expended. Ideally, grading and reporting systems should be developed | cooperatively (parents, students, school personnel) in order to ensure development of a more adequate system, and a system that is understandable to all. They should thus be: Chapter? ~ Grading and Reporting dents about objectives/standards they are to meet Parent-teacher conferences. Parent-teacher conferences are mainly used in elementary schools. This requires that parents of pupils come for a conference with the teacher to discuss the pupils’ progress. Portfolios, when available, are useful basis for discussion. Such conferences are useful for a two-way flow of information and getting more information and cooperation from the parents. They are , however, of limited value as a report because most parents do not come for such conferences. Based on clear statement of learning objectives. The grading and reporting system needs to be based on the-same set of learning objectives that the parents, teachers and students agreed at the beginning. Consistent with school standards. The system must support the school standards rather than oppose the school standards already set. Based on adequate assessment. The grading and reporting system should be easily verifiable throvgh adequate system of testing, measurement and assessment methods. Based on the right level of detail. The system must be detailed enough to be diagnostic but compact enough to be practical: not 100 time consuming to prepare and use, under- standable to users and easily summarized for school records purposes. Providing for parent-teacher conferences as needed 9392 ‘Anucen Merionsn Eoucanow Assessue 0 EyuaTon (ASSESBIENT OF LEARNING 2) a, Traditional letter-grade system. In the traditional letter grade system, students’ performance are summarized by means of letters. Thus, A stands for excellent, C stands for average, D stands for needing improvement and an F as a failure, The traditional letter grade system is easy to under- stand but it is of limited value when used as the sole report because they end up being a combination of achievement, effort, work habits, behavior. As such, they become difficult to interpret and they do not indicate pattems of strengths and weaknesses. b. Pass-fail. The pass or fail system utilizes a dichotomous grade system. Either a student has complied and reached certain standards, in which case he passes or he failed to do so and he gets a failing mark. This is popular in some courses in college (but not very much practice in basic education). In fact, the pass-fail system should be kept to a minimum because it does not provide much information, students tend to work to the minimum (ust to pass), and in mastery learning courses, no grades are reflected until “mastery” threshold is reached. Checklists of objectives. In this system, the objectives of the course are enumerated. After each objective, the students’ level of achievement is indicated: Outstanding, Very Good, Good, Fair or Poor. This is a very detailed reporting system and tends to be more informative for the parents and pupils at the same time. It is, hewever, also very time con- suming to prepare. There is also the potential problem of keeping the list manageable and understandable. d. Letters to parents/guardians. Letters to parents and guard- ians are useful supplement to grades. However, they have limited value as sole report because they are very time ‘consuming to prepare, the accounts of weaknesses are often misinterpreted by parents and guardians, and they are not characterizéd as systematic nor cumulative, e. Portfolios. As already explained, a portfolio is a set of pur posefully seleeted work, with commentary by student and teacher . Portfolios are useful for showing students” strengths and weaknesses, illustrating range of students’ work, show- ing progress over time or stages of a project, teaching ‘stu-: Chapter 7 ~ Grading and Reporting 7.1 Functions of Grading and Reporting Systems Grading and reporting systems have some very important functions in the educative process. Some of these important functions are enumerated below: 1. Enhancing students’ learning through: clarifying instruc- tional objectives for them, showing students’ strengths and weaknesses, providing information on personal-social development, enhancing students’ motivation (¢.g., short-term goals) and indicating where teaching might be modified. These can be achieved through day-to-day tests and feedback and integrated periodic tests., 2. Reports to parents/guardians Grading and reporting systems also inform parents and guardians of students on the progress of their wards. Likewise, grades and reports communicate objectives to parents, so they can help promote leaning and likewise, communi- cate how well objectives were met, so parents can better plan. 3. Administrative and guidance uses. The administrative and guidance purpose of grading and reporting consist in :helping to de- cide promotion, graduation, honors, athletic eligibility, reporting ‘achieve ment to other schools or to employers , providing input for re educational, vocational, and personal counseling. ‘These three main purposes of grading and reporting by no means exhaust all possible uses of the activity, The main point, however, is that grades and report cards should promote and enhance learning rather than frustrate and discourage students. In many schools, report cards are given to students and then parents are asked to sign these report cards, Such a practice is not encouraged in the Philippine edu- cational system. Instead, it may be a good practice to call for parent- teacher conferences at this time in order that the report cards will effectively function as motivation for further learning. 7.2. Types of Grading and Reporting Systems In practice, we find a number of ways in which grades are made and reported. Normally, the type of grading used depends on the extent to which more descriptive and informative summaries are required by school authorities or by the stakeholders in general. Thus, the major types of grading and marking found in practice include: 1Cuerer 7 ‘The main aim of grading and reporting system is {0 prove results in brio, under. standabie form for varied users GRADING AND REPORTING One of the more frustrating aspects of teaching is that of grad- ing and reporting student progress since there are so many factors to consider, and so many decisions to be made. This chapter attempts to simplify this task and to minimize some of the complexities by describ- ing the various types of grading and reporting systems and providing idelines for their effective use. The main aim of grading and report- ing system is to. provide results in brief, understandable form for var- ied users which lead to several big questions: .What should I count— Just achievement, or effort, too? How do I interpret a student’s score? Do I compare it to other students’ scores (norm-referenced), a stan- dard of what they can do (criterion-referenced), or some estimate of what they are:able to do (learning potential, or self-referenced)? What should my distribution of grades be, and how do I determine it? How do I display student progress or strengths and weaknesses, to students and their parents? Of course, answers to these questions may be obtained from: your school which may have some policies or guidelines, applying what you learn in this Chapter; consulting your teaching colleagues; and then applying your good judgment or, learning from first-hand experience. Measurement and assessment of learning are the first ‘two steps in educational evaluation. Grading and reporting are the two terminal stages in the process, Measurements are simply numerical aids which guide our decisions in the educative process. It is, there- fore, very important that we also pay attention to the process of ssiving grades and reporting these to students, parents and other stake- holders in the system,(Chapter 6 - Educational Evaaton CHAPTER EXERCISES A Discuss the following evaluation philosophies: + subjectivist ethics + objectivist ethics + subjectivist epistemology + objectivist epistemology + utilitarianism B. State and discuss the basic principles in educational assess- ‘ment and evaluation. C. Consider Stufflebeam’s CIPP evaluation model. What are the essential components of: + Context? + Input? + Process? + Product? D. Using Stufflebeam’s CIPP model, conduct a self-assessment to evaluate your own Mathematics program in your school. E, What are the political dimensions involved in educational evaluation? Discuss these dimensions thoroughly. References 1. DAC Network on Development Evaluation Home Page 2. Joint Committee on Standards for Educational Evaluation 3. American Evaluation Association Guiding Principles for 5. Evaluators House, E. R. (1978). “Assumptions underlying evaluation models.” Educational Researcher. 1(3), 4-12. Stufflebeam, D. L., & Webster, W. J. (1980). “An analysis of alternative approaches to evaluation.” Educational Evaluation and Policy Analysis. 2(3), 5-19. 8988 -Abwuicen Merions n EoucaowusAssessue mo Ewen (Asessuen oF Learn 2) can ow: These guide questions are not answered by the teacher only or by a single individual. Instead, there are many ways in which they be answered. Some of the more common methods are listed be- + discussion with class * informal conversation or observation + individual student interviews + evaluation forms + observation in class/session of teacher/trainer by colleagues + video-tape of own teaching (micro-teaching) ‘+ organizational documents + participant contract + performance test + questionnaire + self-assessment + written test 6.4 Summary of Keywords and Phrases: + Assessment is the process of gathe information as part of an evaluation. ig and analyzing specific + Competency evaluation is a means for teachers to determine the ability of their students in other ways besides the standard- ized test. + Course evaluation is the process of evaluating the instruction of a given course. + Educational evaluation is evaluation that is conducted spe- cifically in an educational setting. + Immanent evaluation, opposed by Gilles Deleuze to value Judgment + Performance evaluation is a term from the field of language testing. It stands in contrast to competence evaluation. + Program evaluation is essentially a set of philosophies and techniques to determine if a program ‘works’.(Capler 6 - Edvatona Evaluation How supportive is the classroom environment? How many students are there? How many teachers are there? How is the course organized? ‘What regulations relate to the training? What is the workload of students? How well/actively do students participate? Are there any problems related to teaching? Are there any problems related to learning? Is there an effective 2-way communication? Is knowledge only transferred to students, or do they use and apply it? ‘Are there any problems which students face in using/apply- ing/analysing the knowledge and skills? Are the teaching and leaming process continuously evaluated? ‘Are teaching and learning affected by practical/institutional problems? What is the level of cooperation/interpersonal relations between teachers/students? How is discipline maintained? Is there one final exam at the end or several during the course? Is there any informal assessment? What is the quality of assessment (i.e. what levels of KSA are assessed?) What are the students? KSA levels after the course? Is the evaluation carried out for the whole[*_| In-line, WMF 1 process? How do students use what they have learned? How was the overall experience for the teachers and for the students? : What are the main ‘lessons learned”? Is there an official report? Has the teacher's reputation improved or been ruined as a result? a786 ‘owe Meno n Eoucavon Assess mo Exe (Asessuen orLesmine2) In this approach, the user is asked to go through a series of ques- tions In the context, inputs, process and product stages. These ques- tions are reproduced below for convenience: Context What is the relation of the course to other courses? Is the time adequate? What are critical or important extemal factors (network, min- istries)? Should courses be integrated or separate? What are the links between the course and research/exten- sion activities? Is there a need for the course? Is the course relevant to job needs? What is the entering ability of students? What are the learning skills of students? What is the motivation of students? What are the living conditions of students? What is the students’ existing knowledge[* | In-line. WMF “2 Are the aims suitable? Do the objectives derive from aims? Are the objectives ‘smart’? Is the course content clearly defined? Does the content (knowledge, skills and attitudes{*_| In- line. WME *}) match student abilities? Is the content relevant to practical problems? What is the theory/practice balance? What resources/equipment are available? What books do the teachers have? What books do the students have? How strong are the teaching skills of teachers? What time is available compared with the workload, for preparation? 7 ‘What knowledge, skills and attitudes, related to the subject, do the teachers have?(Chapter 6 - Edvostonal Evaluation = Accelerated aging | + Course evaluation | - interview + Quentiatve } Acion research | « Deta mining + Marketing research | research + Advanced Product | + Delphi Technique | - Mete-enalysis Questonnaie Quaity Panning | « Discourse anaiyss | - Metis + Questonnaie + Alematve + Bectonic porioio | + Most sigifcant constuction assessment + Envtonmental change Root cause analysis + Appreciative inquiry] scanning + Mutivatote stasios | Rubrics + Assessment + Ethnography + Natrasic ‘Sampling I Aadomatic design | - Experiment observation + School aoreitation : Benchmarking — | + Experimenta + Observational + Sehassessment + Case study teciques teciniques + Sx Sigma + Change + Fectr anatysis + Opinion pong |» Standacized testing management + Factorial experiment | « Orgarizatonal + Stace process + Clinical tal + Feasbilly sty tearing contol + Coho study | » Field experiment Pertipant observa. |. Statistical suney + Competitor anaiysis | Fitureless In-Circut | fon + Salis + Consensus Test + Partcpatory Impact | Strategic planning decision-making | « Focus group Pathways Anays's | Structured + Consensusseeking | « Foroe field analysis | - Policy analysis interiwing decisionmaking | + Game theory + Process + Systems theory + Content analysis | + Grading improvervent + Student ting + Conversation | - Historcal method |» Project management + Total Quality analysis Inquiry + Quaitatve research | Management + Costbenelt + Qualty audit + Triangulation analysis + Qual cece + Quaity cont + Cualty management + Quality Met 6.4 The CIPP evaluation model Stufflebeam (1983) developed a very useful approach in educa~ tional evaluation knowsi as the CIPP or Context, Input; Process, Prod- uct approach (although this model has since then beef expanded to CIPPOI (where the last two stand for Outcome and Impact respec- tively). The approach essentially systematizes the way we evaluate the different dimensions. and aspects of curriculum development and the, sunytotal of student experiences in the educative process. The model requires that stakeholders be involved in the evaluation process. The approach is illustrated in the diagram below: ‘The ‘CIPP’ model of evaluation84 Avance Meriooen Eovcxtona Assess aio Euan (Asessue or Leang2) use established guidelines to determine if the applicant should be ap- proved to perform specified functions. However, unless performance- based standards are used, attributes of applicants and the processes they perform often are overemphasized in relation to measures of outcomes or effects. ‘Connoisseur studies use the highly refined skills of individuals intimately familiar with the subject of the evaluation to critically char- acterize and appraise it. This approach can help others see programs in a new light, but it is difficult to find @ qualified and unbiased con- noisseur, . ayo Subjectivist, mass, true evaluation The adversary approach focuses on drawing out the pros and cons of controversial issues through quasi-legal proceedings. This helps ensure a balanced presentation of different perspectives on the issues, but it is also likely to discourage later cooperation and heighten animosities between contesting parties if “winners” and “los- ers” emerge. Client-centered studies address specific concems and issues of practitioners and other glients of the study in a particular setting. These studies help people understand the activities and values involved from a variety of perspectives. However, this responsive approach can lead to low external credibility and a favorable bias toward those ‘who participated in the study. 6.3 Evaluation methods and techniques Evaluation is methodologically diverse using both qualitative methods and quantitative methods, including case studies, survey research, statistical analysis, and model building among others. A more detailed list of methods, techniques and approaches for conduct- ing evaluations would include the following:Chapt 6 ~ Educatoral Evaluation might not adequately sample what is taught or expected. Objectives-based approaches relate outcomes to prespecified objectives, allowing judgments to be made about their level of attain- ment, Unfortunately, the objectives are often not proven to be impor- tant or they focus on outcomes too narrow to provide the basis for determining the value of an object. Content analysis is a quasi-evaluation approach because con- tent analysis judgments need not be based on value statements. In- stead, they can be based on knowledge. Such content analyses are not evaluations. On the other hand, when content analysis judgments are based on values, such studies are evaluations. Objectivist, mass, quasi-evaluation Accountability is popular with constituents because it is in- tended to provide an accurate accounting of results that can improve the quality of products and services. However, this approach quickly can turn practitioners and consumers into adversaries when imple- mented in a heavy-handed fashion, Objectivist, elite, true evaluation Decision-oriented studies are designed to provide a knowledge base for making and defending decisions. This approach usually re- quires the close collaboration between an evaluator and decision- maker, allowing it to be susceptible to corruption and bias. Policy studies provide general guidance and direction on broad issues by ig and assessing potential costs and benefits of competing policies. The drawback is these studies can be corrupted or subverted by the politically motivated actions of the participants. Objectivist, mass, true evaluation Consumer-oriented studies are used to judge the relative merits of goods and services based on generalized needs and values, along with a comprehensive range of effects. However, this approach does not necessarily help practitioners improve their work, and it requires a very good and credible evaluator to do it well. Subjectivist, elite, true evaluation Accreditation / certification programs are based on self-study and peer review of organizations, programs, and personnel, They draw on the insights, experience, and expertise of qualified individuals who 8382 ‘Amancen Menoos n Educanowt Assessuen ao Ewuanon (ASsesuenr oF Lene 2) We briefly describe each of these below: Pseudo-evaluation Politically controlled and public relations studies are based on an objectivist epistemology from an elite perspective. Although both of these approaches seek to misrepresent value interpretations about some object, they go about it a bit differently. Information obtained ‘through politically controlled studies is released or withheld to meet the special interests of the holder. Public relations information is used to paint a positive image of an object regardless of the actual situation, Neither of these ap- Proaches is acceptable evaluation practice, although the seasoned reader can surely think of a few examples where they have been used. Objectivist, elite, quasi-evaluation As a group, these five approaches represent a highly respected collection of disciplined inquiry approaches. They are considered quasi- evaluation approaches because particular studies legitimately can focus only on questions of knowledge without addressing any questions of value, Such studies are, by definition, not evaluations. These approaches can produce characterizations without producing appraisals, although specific studies can produce both. Each of these approaches serves its intended purpose well. They are discussed roughly in order of the extent to which they approach the objectivist ideal. Experimental research is the best approach for determining causal relationships between variables. The potential problem with using this as an evaluation approach is that its highly controlled and stylized methodology may not be sufficiently responsive to the dynamically changing needs of most human service programs. Management information systems (MISs) can give detailed information about the dynamic operations of complex programs. How- ever, this information is restricted to readily quantifiable data usually available at regular intervals. Testing programs are familiar to anyone who has attended school, served in the military, or worked for a large company. These programs are good at comparing individuals or groups to selected norms in a number of subject areas or to a set of standards of per- formance. However, they only focus on teste performance and theyChapter 8 ~ Educational Evan at “bute ‘Approach | Organizer | Purpose Key strengths | Key weaknesses besin: | Deasins Provide atnon- | Ereauoge use of | Neoesaryclabrton ofented edge & value base | evaluation to plan &| between evaluator & formakng & | inpbmet nested | decison raker pees celering eases. pograns Hobe | ecprinty Bee justify decisions results, boi pans & ors Palo es Try ad assess | Prove gener | On corsa er eae) potential costs & | direction for broadly | subverted by poliically Donal cf carein east acs. | mobi aos ok pales, partons ‘Consumer: Generaized needs | Judge the relative | Independent ‘Might not help practtio- ciel | Gvabes, fects | mers of aemate | pra topo | 68 ober goods & sevice. | patra & | Requios ele & ume tom | compet evar. Sho pos & servos, igh pubic ety Tatiana | Sanda & | Daemnet | Hope pubic mae | Sena & gusanes catfcain | gues instars, infomed sections | typicaly nce proans & fatty of | icc be Perse shou telaranzaons & | eentigs ages topced i pafor| qsateae | Cu ‘peed tinct | somone comisseur | Cia! Caticaly describe, |explas nigh | Dependent on smal ates Sean eae NON cg | ruber of exes, ila a ote. [on sajeat cf | aa ern inerest. Can inspire | Susan to sublecve others to more iy, Bias, and comuption. isch ot fae r an deonrage oesay | Yor issues | Present ne pos & | Ess bdanae ons of an ue, [ernie of | cope, hen ‘pects snmoaten perpectes. Glatatoed | Specie omens reser ndasendeg Tow exena aed, Biss | Gocwtes thw [OME 2S, | sorte was a they are valued in | ing own evan, favor of pericpats, a gen stig & : fom a vat of ‘perspectives. Noe: Adapted and codensed primary from House (1978) and Suflebeam & Webster (1980),80 Amoxcen Mericos m Eoucanen Assessienr wo Eun (ASSESSMENT OF LesUNG 2) Table 4. Summary of Approsches for Conducting Evaluation Tbe ‘Approach Organizer Purpose Key strengths ‘Key weaknesses: Get, keep or ‘Secure evidence | Violates the principle of paitiaty | Teas isoease interes, | atvenageeus | ‘ul & Frank dsclosue scaled power or money. | to the dentin acai Create positive ‘Secure evidence | Violates the principles of a Popeume | bc mage. most ey to | balers eprig bake pubic | jsfedconcsions, & a ‘support objectivity. Datsine causd | Stongist Requtes cored Enaineid | Coma rns bavee| pad tr | sting, is rere of rea stcstips | varie. Individual Compare te8 8°55) elabe evidence | teste ‘performance, ‘Giierences ot divas & in many ‘overemphasizes test- Tesing yorps 0 seeded | cerammence | taking skits, can be pogams ‘norms. ‘areas. Very (poor sample of what is. ‘amar to pubic. | taught or expected, Raia cacores | Objecives Commen sense | Leads tein objectives. ‘appeal, widely ‘evidence offen 00 decves. used, dses | aon to provide bass xed behavra fe jong to vue of 2 ctjecves & | progam, tetg sol ses. el Desaite & fan Sennen | SINS] ote | meeiane Content commuricaicn. | anajysis of large. | overwhelming in volume, mails volumes of | Anais design on vestuetued, | overly snp fr symboie cuesin rata. Peformance | Fonds OoSIU=T | poner qin | Crees ues bacon erpedatons | win an azuate | coruens, | rates & consume a fs, Poles ofen ces ‘Accountbilty ae Aimed ot bon improving qualty | Peete studs, of praia series.(Chapter 6 - Educational Evauaton tation promotes a positive or negative view of an object regardless of ‘what its value actually might be. They call this pseudo-evaluation. The question orientation includes approaches that might or might not pro- vide answers specifically related to the value of an object. They call this quasi-evaluation. The values orientation includes approaches prima- rily intended to determine the value of some object. They call this true evaluation. When the above concepts are considered simultaneously, fifteen (15) evaluation approaches can be identified in terms of epistemology, ‘major perspective (from House), and orientation (from Stufflebeam & Webster). Two pseudo-evaluation approaches, politically controlled and public relations studies, are represented. They are based on an objec- ivist epistemology from an elite perspective. Six quasi-evaluation approaches use an objectivist epistemology. Five of them—experimen- tal research, management information systems, testing programs, objec- tives-based studies, and content analysis—take an elite perspective. Accountability takes a mass perspective. Seven (7) true evaluation approaches are included. Two (2) approaches, decision-oriented and policy studies, are based on an objectivist epistemology from an elite perspective, Consumer-oriented studies are based on.an objectivist epistemology from a mass perspective. Two approaches—accredita- tion/certification and connoisseur studies—are based on a subjectivist epistemology from an elite perspective. Finally, adversary and client- centered studies are based on a. subjectivist epistemology from a mass perspective. ‘The following table is used to summarize each approach in terms of four attributes—organizer, purpose, strengths, and weaknesses. ‘The organizer represents the main considerations or cues that practi- tioners use to organize a study. The purpose represents the desired outcome for a study at a very general level. Strengths and weaknesses represent other attributes that should be considered when deciding whether to use the approach for a particular study. The following narrative highlights differences between approaches grouped together. 7978 Aowacen Metso m Eoucanon Assesses Ese (ASSESSMENT OF Lean 2) 6.2 Evaluation approaches Evaluation approaches are the various conceptual arrangements made for designing and actually conducting the evaluation process Some of the evaluation approaches used today in the educational setting are original approaches while others are refinements and exten- sions of these original approaches. The first major classification of evaluation is that authored by House (1980) This considers all major evaluation approaches to be based on a common ideology which is that of liberal democracy. In liberal democracy, it is believed that an individual has freedom of choice, he is unique and that the evaluation process is guided by em- pirical inquiry truly based on objective standards. However, all evalua- tions are based on subjectivist ethics in which the individual's subjec- fe experiences figure prominently. For instance, utilitarianism is one form of subjectivist ethics in which what is good is defined as that which maximizes the happiness of society as a whole. In contrast, another form of subjectivist ethics is the intuitionist or pluralist form in which no single interpretation of “the good” is assumed and these in- terpretations need not be explicitly stated nor justified. Each of these ethical positions has its own cotresponding ways of obtaining knowledge or epistemology. The objectivist epistemology is associated with the utilitarian ethics. In this case, knowledge is acquired which is capable of external verification and evidence (intersubjective agreement) through methods and techniques universally accepted and through the presentation of data, The subjectivist episte- mology is associated with the intuitionistipluralist ethic. It is used to acquire new knowledge based on existing personal knowledge and experiences that are (explicit) or are not (tacit) available for public inspection, House’s approach further subdivides the epistemological approach in terms of two main political perspectives. Thus, an approach can be elitist in which the idea is to focus on the perspectives of managers and top echelon people and professionals. The approach can also be mass-based in which the focus is on consumers and the approaches are participatory, On the other hand, Stufflebeam and Webster (1980) place approaches into one of three groups according to their orientation toward the role of values, an ethical consideration. The political orien-Chapter 6 - Educational Evaluation In order to systematize the evaluation process, The American Evaluation Association has created a set of Guiding Principles for evaluators which can equally apply in the Philippine context... These guiding principles are hereunder stated: + Systematic Inquiry: Evaluators conduct systematic, data- based inquiries about whatever is being evaluated. Inquiry cannot be based on pure hearsay or perceptions but must be ‘based on concrete evidence and data to support the inquiry process, + Competence: Evaluators provide competent performance to stakeholders. The evaluators must be people or persons of known competence and generally acknowledged in the educa- tional field. + Integrity / Honesty: Evaluators ensure the honesty and integrity of the entire evaluation process. As such, the integ- rity of authorities who conduct the evaluation process must be beyond reproach. + Respect for People: Evaluators respect the security, dignity and self-worth of the respondents, program partici- pants, clients, and other stakeholders with whom they interact. They cannot act as if they know everything but must listen patiently to the accounts of those whom they are evaluating. + Responsibilities for General and Public Welfare: Eva- luators articulate and take into account the diversity of interests and values that may be related to the general and public welfare. These evaluation guiding principles can be used at various levels: at the institutional level when we evaluate learning , at the policy level when we evaluate insti rank and evaluate the performance of various institutions of higher learning. In whatever level of evaluation these principles are used, it is well to bear in mind that these guiding principles serve as benchmarks for good practices in educational evaluation s, and at the international level when we 7Corer 6 EDUCATIONAL EVALUATION Book 1 and most of Chapters 1 through 5 of this textbook concern themselves with assessment. Evaluation is the next stage in ‘the process. “Evaluation” when viewed from the micro or classroom level may be defined as a systematic, continuous and comprehensive process of determining the growth and progress of the pupil towards objectives or values of the curriculum. However, when viewed more generally, evaluation may be characterized as the systematic determi- nation of merit, worth, and significance of something or someone. Evaluation ofien is used to characterize and appraise subjects of interest in a wide range of human enterprises, including the Arts, business, computer science, criminal justice, engineering, foundations and non-profit organizations, government, health care, and other human services. 6.1 Educational Evaluation In the United States, there is a Joint Committee on Standards for Educational Evaluation which has developed standards for educational programmes, personnel, and student evaluation. The Joint Committee standards are broken into four sections: Utility, Feasibility Propriety, and Accuracy. In the Philippines, there is:also a society which looks into educatiortal evaluation, the Philippine Society for Educational Research and Evaluation (PSERE) but mainly, educational evaluation standards are set by the Department of Education. Various Europeen institutions have also prepared their own standards, more or less related to those produced by the Joint Committee in the“United States. They provide guidelines about basing valve judgments on systematic inquiry, evaluator competence and integrity, respect for people, and regard for the general and public welfare,Chapt 5 - Porto Assessment Nehads 6. Appendices and Dates of Drafts D. Enumerate and discuss the various types of portfolios. E, What is the importance of student-teacher conferences? Discuss its importance in light of portfolio assessment. F, What is the main philosophy behind portfolio assessment? Discuss this basic philosophy. G. How does portfolio assessment differ from traditional te and from other authentic assessment methods? Chapter References De Fina, A. 1992. Portfolio Assessment: Getting Started. Scholastic Grabe, W. & R. Kaplan. 1996, Theory and Practice of Writing, Addison Wesley, Longman, Hedge, T. 1988. Writing. Oxford University Press. Johnson, K.E. 1996, “Portfolio assessment in second language teacher education,” TESOL Journal, Volume 2. O™Malley, J.M. & L. Valdez Pierce. 1996. Authentic Assessment ‘for English Language Learners. Addison-Wesley Publishing ‘Company. : Paulson F.L., PR. Paulson and C.A. Meyer. 1991. “What makes « portfolio a portfolio?” Educational Leadership. February. Puhl, C.A. 1997. “Develop, not judge: Continuous assessment in the ESL classroom.” Forum 35:2. pp. 2:9. Santos, M.G. 1997. “Portfolio assessment and the role of leamer reflection.” Forum 35:2. pp.10-16. Tierney R.J., M.A. Carter and L.E. Desai. 1991. Portfolio Assessment in the Reading-Writing Classroom. Norwood: Christopher Gordon Publishers. Toperoff, Debby. 1995. Portfolio Assessment in Literature Teaching. M.A. Dissertation. Surrey University. Wolf, K. & Y. Siu-Runyan, 1996. “Portfolio purposes and possibilities.” Journal of Adolescent & Adult Literacy 40:1. pp. 30-36. 7574 ‘Aowwcen Merions n Eovcion AssesSuen 80 EATON (ASESSUEN OF LEARN 2) 5.7 Student-Teacher Conferences The main philosophy embedded in portfolio assessment is “shared and active assessment”. To this end, the teacher should have short individual meetings with each pupil, in which progress is discussed and goals are set for a future meeting. Throughout the process, the student and the teacher keep careful documentation of the meetings noting the significant agreements and findings in each individual session. Through meetings of this kind, the formative evaluation process for portfolio assessment is facilitated. Indeed, the use of portfolio assessment takes time but in the end, the gains are well worth the time and effort ‘expended by the teacher. Finally, student-teacher conferences can also be used for summative evaluation purposes when the student presents his final portfolio product and where final grades are determined together with the teacher. Even at this stage of the process, students can negotiate for the appropriate grade to be given using as evidence the minutes of the regular student-teacher conferences. It is for this reason that notes from these conferences have be included in the portfolio as they contain joint decisions about the individual's strengths and weaknesses. These conferences can be prepared for in pairs, where students practice presenting their portfolios. CHAPTER EXERCISES ‘A. What is portfolio assessment? What are the key elements of «a portfolio? B. What are the purposes of portfolio assessment? Discuss the merits of portfolio assessment over traditional testing. C. For each of the following main elements of a portfolio, construct a rating scale or rubrics for evaluating students’ portfolio on the topic: “The EDSA Revolution I": 1. Cover letter 2, Table of Contents and Introduction 3. Entries 4, Reflections 5. Summative Statements(Chapfer 5 - Portolo Assessment Mehas the test items. Moreover, students and teachers work collaboratively to determine grades or scores to be assigned. In this particular dimen- sion, rubrics, rules, and scoring keys can be designed for a variety of portfolio components. In addition, letter grades might also be assi where appropriate. It is especially recommended that for summative purposes, a panel of interviewers be designated to evaluate the stu- dents’ portfolio based on the agreed set of criteria at the beginning. It important that the panel of interviewers understand the decision- ‘making process that the student underwent in selecting the items to be included in the finished portfolio.(.Source: Mindful School: Portfolio connection, Burke, Fogerty, IRI/Skylight Publishing (1994) Each portfolio entry needs to be assessed with reference to its specific goal(s). Since the goals and weighting of the various portfolio ‘components have been clearly fixed in advance, assessing the portfo- lios is not difficult. Self and peer-assessment can be used too, as a ‘ool for formative evaluation, with the students having to justify their ‘grade with reference to the goals and to specific pages in the portfo- lio. This actually makes the teacher's job of assessing the portfolio much simpler, because the pupil has done the groundwork of proving how far each goal is met in the portfolio. It takes some of the burden off the teacher and helps students to internalize criteria for quality work After all the efforts that your students have invested in their portfolios, it is recommended that the teacher provides feedback on the portfolios that is more than just a grade. One possibility is to write a letter about the portfolio, which details strengths and weaknesses and generates a profile of a student's ability, which is then added to the portfolio. Another option is to prepare certificates which comment on the portfolio strengths and suggest future goals, Finally, the teacher needs to give guiding feedback. The finished portfolio may be due only at the end of the semester, but it is a good idea to set regular dates at which time several portfolio-ready items (ie, with drafts and refléctions) will be handed in, so that students know whether they are on the right track. Alternatively, you can have 1 portfolio project on a single unit of material so that both teacher and students will acquire experience in this kind of assessment over a shorter period of time. 7372 ‘Anwwoso Meroos wEoucanons Assessuer no ExunTon (ASSESSMENT OF LEAHING 2) criteria to be used may be formative (ie, throughout an instructional time period) or summative (i.e., as part of a culminating project, activity, or related assessment to determine the extent to which iden- tified curricular expectancies, indicators, and standards have been achieved). Below is an example of a scoring form for a portfolio covering the “cover letter” SAMPLE OF RATING SCALE FOR COVER LETTER Grade Description 1-3 | shows limited awareness of portfolio goals has difficulty understanding the process of revision demonstrates litle evidence of progress over time limited explanation of choices made has difficulty relating to selfpeer assessment 4-7 | reflects awareness of some portfolio goals Understands the process of revision to a certain extent demonstrates some evidence of progress over time explains choices made in a relevant way relates to selfipeer assessment 8-10 | reflects awareness of portfolio goals Understands the process of revision demonstrates evidence of progress over time fully explains choices made reaches high level of reliability in selfipeer assessment draws conclusions about his/her leaming Cd One of the more significant aspects of portfolio assessment is its “collaborative approach’” in which students and teacher work together to identify especially significant or important artifacts and processes to be captured in the portfolio. In traditional testing, students do not have any influence nor significant inputs as to what the teacher will ask inChapter 5 = Pttoo Assessment Methods type of portfolio is best used for summative evaluation of stu- dents’ mastery of key curriculum outcomes. It should include students’ very best work, determined through a combination of student and teacher selection. Only completed work should be included. In addition, this type of portfolio is especially compatible with audio-visual artifact development, including photographs, videotapes, and electronic records of students? completed work. The showcase portfolio should also include vritten analysis and reflections by the student upon the deci- sion-making process(es) used to determine which works are included. 5.6 Assessing and Evaluating the Portfolios ‘According to Paulson, Paulson and Meyer, (1991, p. 63): “Portfo- lios offer a way of assessing student learning that is different from the traditional methods. Portfolio assessment provides the teacher and students an opportunity to observe students in a broader context: taking risks, developing creative solutions, and learning to make judgments about their own performances.” Detailed rating criteria may be evolved to evaluate the finished portfolio presented by students. In general, however, they should in- clude the following: + Thoughtfulness (including evidence of students’ monitoring of their own comprehension, metacognitive reflection, and productive habits of mind) + Growth and development in relationship to key curriculum ‘expectancies and indicators + Understanding and application of key processes + Completeness, correctness, and appropriateness of products and processes presented in the portfolio + Diversity of entries (e.g., use of multiple formats to demon- strate achievement of designated performance standards) In evolving an evaluation criteria, teachers and students must ‘work together and agree on the criteria to be applied to the portfolio, Such evaluative criteria need to be set and agreed prior to the deve- lopment of the portfolio. The criteria themselves will serve as guide to the students when they actually prepare the portfolio requirement. The m4‘Aowncen Menon Eovcanown Assessuer io Exunron (Assessuent or LEAN 2) 70 as a core item together with reflection on what the student leamed from doing the test and revising it. Furthermore, you may ask the stu- dents to explain in their reflections who helped them to" improve their work (a peer, a parent, a spell-checker) and what they leamed from revising their work. 5.5 Types of Portfolios Different types of portfolios exist for assessing student perfor- mance. These differ from each other depending on the purposes or objectives set for the overall classroom assessment program. As a general rule, portfolio assessment is used where traditional testing would be inadequate to measure desired skills and competencies. Essentially, three types of portfolios are normally cited in the literature with appropriate variants for each: + Documentation Portfolio: As the name implies, this ap- proach involves a collection of work over time showing growth and improvement reflecting students’ leaming of iden- Documentation tified outcomes. This portfolio is also called a “growth portfo- Porto inokes lio” in the literature. The documentation portfolio can include evra everything from brainstorming activities to drafts to finished showing growth products. The collection becomes meaningful when specific and improverent items are selected out to focus on particular educational ex- Senet ering periences or goals. It can include the best and weakest of of identified student work. It is important to realize here that even drafts ‘outcomes, ‘and scratch papers should be included in the portfolio for they actually demonstrate the growth process that the students a have been through. parca ost | +. Process Portfolio: The process portfolio in contrast demon- in decurening strates all facets or phases of the learning process. As such, students’ overall these portfolios contain an extensive number of reflective leaning process. : ‘ journals, think logs and other related forms of metacognitive Showcase Porto processing. They are particularly useful in documenting stu- Petco) dents’ overall learning process. It can show how students howe the best of integrate specific knowledge or skills and progress towards the students? both basic and advanced mastery. oe + Showcase Portfolio: The showcase portfolio only shows the best of the students’ outputs and products. As such, this(Chaplor 5~ Potoo Assessment Meads Stage 5. Informing key school officials, parents and other stake- holders. Do not attempt to use the portfolio assessment method without notifying your department head, dean or prin precaution in case students will later complain about your new assess- ment procedure. 1. This will serve as a Stage 6. Development of the Portfolio Both students and teacher need support and encouragement at this stage in the process of portfolio development. The students par- icularly should get this from an understanding and patient teacher. Teachers will get it by doing portfolio assessment as teamwork in their staff or joining or initiating a support group to discuss questions with colleagues as they arise. Some portion of the class-time can be de- voted to student-teacher dialogues and conferences with other teachers in relation to the task of preparing the portfolio. It is necessary to stress the importance of reflection and self-assessment while prepar- ing the portfolio itself since these are essentially new skills for the students. Reflection and self-assessment require practice. There are certain essential questions that the teachers can use to guide students in reflections and self-assessment such as: ‘+ What did I learn from that activity? + Whi ‘+ How can I improve this? This can be done by class brain- storming (what are some possible reasons for including an item in your portfolio?) or in pairs - “portfolio partners” - who help each other select samples of their work (written comments on their work from a peer can also be included in the portfolio). my best piece? Initially, we advise teachers to begin with more guided and closed forms of reflection and slowly proceed to more open reflective comments, Writing reflections is a life-skill which is very essential for the students and is, therefore, time well spent for the teacher. Finally, since portfolios are essentially done by the students out- side of the regular class-time, we need to ensure that indeed the port- folio represents the students’ work and accomplishment. In order to do this, require that some items be done completely in class. You might also decide to have a test (preferably with corrected version) included 6968 ‘Avwwcen Merioosm Eovcaron Assessuen snp Euan (ASSssuENt oF LesRANG 2) Porftolio assessment is a new thing for many students who are used to traditional testing. For this reason, it is important for the teacher to introduce the concept to the class. Perhaps, he can start by explaining the meaning of the word “portfolio” from portare (carry) and foglio (sheet of paper). If you can invite artists or achitects to come and visit your class bringing with them their own set of portfo- ios, then this will help convey the principle of a portfolio as a selec- tion of a student’s work, showing progress in different areas or skills It is also a good idea to show the students examples of existing portfolios prepared by other classes or by other students. If you hi ‘your own portfolio, then this will also help in conveying the information to the students (exemplar method), In order to convince your students that portfolio assessment is worth a try, ask them to enumerate their problems with traditional testing. Tell them that portfolio assessment will assess them in a much fairer way than traditional testing would. It is also important to inform the students how much weight the portfolio will have in the computa- tion of their final grades and just what is going to be replaced by the portfolio, Stage 3. Specification of Portfolio Content Specify what and how. much have to be ifcluded in the portfolio - both core and options.(it is important to include options as these enable self-expression and independence). Specify for each entry how it will be assessed. The students should be acquainted with the scoring ‘guides/rating scales that will be used before performing the task. Port- folio entries can take many forms - written, audio and video-recorded items, artifacts (¢.g., a T-shirt, an annotated drawing, a model), dialogue journals, etc, Stage 4. Giving clear and detailed guidelines for portfolio presentation There is a tendency for students to present as many evidence of learning as they can when left on their own. The teacher must there- fore set clear guidelines and detailed information on how the portfolios will be presented. Explain the need for clear and attractive presenta- tion, dated drafts, attached reflections or comment cards. Moreover, it will help if the teacher explains how the portfolio will be graded and when it needs to be ready (final and mid-way dates)Chapter 5 ~ Porto Assessment Methads cess (for formative and/or summative purposes.) and at the Tower levels can be written in the mother tongue or by stu- dents who find it difficult to express themselves in English, For each item - a brief rationale for choosing the item should be included. This can relate to students’ performance, to their feelings regarding their progress and/or themselves as leamers. Students can choose to reflect upon some or all of the following: + What did I learn from it? + What did I do well? + Why (based on the agreed teacher-student assessment criteria) did I choose this item? + What do I want to improve in the item? + How do I feel about my performance? + What were the problem areas? 5.4 Stages in Implementing Portfolio Assessment The following constitute the stages in the implementation of portolio assessment in classrooms: Stage 1: Identifying teaching goals to assess through portfolio ‘The usual first step of organizing portfolio assessment is to estab- lish the teaching goals. It is very important at this stage to be very clear about what the teacher hopes to achieve in teaching. These goals will guide the selection and assessment of students’ work for the portfolio. In order to do this, the teacher should ask himself the ques- tion “What do T want the students to learn?” and choose several goals to focus on; for example, general goals such as improvement in flu- ency of speech or independent reading, and specific goals such as scanning a text or telling a story. The Revised Basic Education Cur- riculum (RBEC) contains many examples of goals (called “bench- ‘marks”) that show progress towards the overall standards of a subject matter to be learned. It is often suggested that this stage be done together with the students asking them what they want and what they need to leam in the particular subject matter. Stage 2. Introducing the Idea of Portfolio Assessment to your class 6766 Amawcen Menicos mn EoveaTona Asessvent wo EWLLAnON (ASSESUENT OF LERANG) pairs so’ that they necessarily have to interact and collaborate to com- plete the tasks. Eighth, portfolio assessment develops independent and ac- tive learners. Students must select and justify portfolio choices; rnoni- tor progress and set learning goals. Traditional testing cannot achieve this educational objective no matter how skillfully the tests are con- structed. Ninth, portfolio assessment can improve motivation for learn- ing and thus achievement. When students are empowered to prove their own achievement and worth they become highly motivateD to pursue the learning tasks. It is when they lose this feeling of empow- ‘erment that they feel inadequate and become less motivated as in tra~ ditional classroom testing... Tenth, portfolio assessment provides opportunity for student- teacher dialogue. It enables the teacher to get to know every student. Morevoer, portfolio assessment promotes joint goal-setting and negotiation of grades which can never happen in traditional tes 5,3 Essential Elements of the Portfolio Every portfolio must contain the following essential elements: 1. Cover Letter “About the author” and “What my portfolio shows about my progress as a learner” (written at the end, but put at the beginning). The cover letter summa- rizes the evidence of a student’s learning-and progress. 2. Table of Contents with numbered pages. 3. Entries - both core (items students have to include) and op- tional (items of student’s choice). The core elements will be required for each student and will provide a common base from which to make decisions on assessment, The optional items will allow the folder to represent the uniqueness of each student. Students can choose to include “best” pieces of work, but also a piece of work which gave trouble or one that was less successful, and give reasons why. Dates on all entries, to facilitate proof of growth over time, 5. Drafts of aural/oral and written products and revised versions; ie, first drafts and corrected/revised versions. 6. Reflections can appear at different stages in the leaming proChapter 5 - Prtolo Assessment Methods classroom activities. Unlike test items which mainly measure cognitive skills, portfolio assessment can assess other components of the students’ formed abilities based on classroom discussions. Second, portfolio assessment has clear goals. In fact, they are decided on at the beginning of instruction and are clear to teacher and students alike. In cognitive testing, the objectives are set at the beginning but the actual items may or may not reflect achievement of such objectives. In portfolio assessment , however, the students control the items to be included and therefore are assured that the goals are achieved. Third, portfolio assessment gives a profile of learner abilities in terms of depth, breadth, and growth, In terms of depth, portfolio assessment enables the students to demonstrate quality work done without pressure and constraints of time present in traditional testing through the help of resources such as reference materials and the help of other students, In terms of breadth, portfolio assessment can show a wide range of skills to be demonstrated in the final output. Finally, in terms of growth , portfolio assessment shows efforts to improve and develop and clearly demonstrates students” progress over time, Fourth, portfolio assessment is a tool for assessing a variety of skills not normally testable in a single setting for traditional testing. The portfolio can show written, oral and graphic outputs of students in a variety of ways which demonstrate skills developed by the students. Fifth, portfolio assessment develops awareness of own learning by the students, Students have to reflect on their own progress and the quality of their work in relation to known goals. This is achieved at each stage of the process since the students continually refer to the set of goals and objectives set at the beginning. Sixth, portfolio assessment caters to individuals in a hetero- geneous class. Such a flexibility is attributed to the fact that portfolio assessment is open-ended so that students can demonstrate their abili- ties on their own level and caters to differential learning styles and expression of varying strengths. Seventh, portfolio assessment develops social skills. Students interact with other students in the development of their own port- folios. Sometimes, they are assessed on work done in groups or in 6564 Atencen Mericosin Eoucarcn Asessuey wo EWLAANON (ASSESSENT OF LeArOMG 2) 5.1 Features and Principles of Portfolio Assessment Porfolio assessment possesses several features and essential characteristics which are enumerated below: 1. A portfolio is a form of assessment that students do together with their teachers, The teachers guide the students in the planning, execution and evaluation of the contents of the portfolio. Together, they formulate the overall objectives for constructing the portfolio. As such, students and teachers in- teract in every step of the process in developing a portfolio. 2. A portfolio represents a selection of what the students be- lieve are best included from among the possible collection of things related to the concept being studied. It is the teachers? responsibility to assist the students in actually choosing from among a possible set of choices to be included in the portfo- lio. However, the final selection should be done by the students themselves since the portfolio represents what the students believe are important considerations. 3. A portfolio provides samples of the student’s work which show growth over time, By reflecting on their own leaming (self-assessment), students begin to identify the strengths and ‘weaknesses in their work. These weaknesses then become improvement goals. 4. The criteria for selecting and assessing the portfolio contents must be cléar to the teacher and the students at the outset of the process, If the criteria are not clear at the beginning, then there is a tendency to include among unessential compo- nents in the portfolio and to include those which happen to be available at the time the portfolio is prepared. At each step of the process, the students need to refer to the agreed set of criteria for the construction and development of the portfolio 5.2 Purposes of Portfolio Assessment Why should we resort to portfolio assessment methods? Portfolio assessment has several purposes and rationale for its use. First, portfolio assessment matches assessment to teaching. The final outputs to be assessed are products of classroom discussions and classroom work and are not simple diversions from the tedium ofPORTFOLIO ASSESSMENT METHODS Curren 5 A portfolio is a purposeful collection of student work that exhibits the student's efforts, progress and achievements in one or more areas. The collection must include student participation in selecting contents, the criteria for selection, the criteria for judging merit and evidence of student self-reflection (Paulson, Paulson, Meyer 1991). Within the con- text of this definition, a portfolio continually grows and accumulates as the student progresses in the particular learning task. Each addition to the portfolio is carefully planned and selected by the student and dem- onstrates his progress. In fact, the overall purpose of creating a portfolio is to enable the student to demonstrate to others learning. and progress. The greatest value of portfolios is that, in building them, stu- dents become active participants in the learning process and its as- sessment. In sharp contrast, students do not have any control or influ- ence in traditional testing i.e. the teacher decides on the items to be included in the test. The sense of “ownership” on the part of the stu- dents that goes with portfolio assessment makes it quite attractive to learners in general. This Chapter concentrates on the topic called portfolio assess- ment methods. Portfolio assessment is one of the several authentic and non-traditional assessment techniques in education. The use of portfolio assessment became popular in the early to late 1980°s in response to the growing clamor for more “reasonable” and authentic means of assessing students’ growth and development in school. One area of application of portfolio assessment, for instance, may be in the ac- creditation of experiences towards a degree (see for example CHED's Expanded Tertiary Education Equivalency and Accreditation Program (ETEEAP). In this modality, experiences of managers or workers are accredited towards a Bachelor's degree (or higher) depending on the portfolio presented by the students to a panel of expert evaluators. 6362 Aoucen Menicos m Eoucavonn Assessuen ko Exon (ASSESBNENT OF LEARANG2) Chapter References Citations 1. Ormrod, J. E. (2006). Educational Psychology: Developing. Learners (Sth ed.), “glossary”. N.J., Merrill: Upper Saddle River (companion website) 2. httpvAwww.des.emory.edu/mfp/effalk htm! 3. Goal Setting and Self-Efficacy During Self-Regulsted Learn- ing. 4. httpviwww.des.emory.edu/mfp/eff htm! 5. Karyn Ainsworth, Fall Quarter Seminar Paper: What is Teaching? / What is Learning? 6. Diffusion of the Internet within a Graduate School of Education, 2. Conceptual Framework 2.3.3.2 Bandura: Efficacy x Value 7. Albert Bandura, Gian Vittorio Caprara, Claudio Barbaranelli, and Concetta Pastorelli, “Sociocognitive Self-Regulatory Mechanisms Governing Transgressive Behavior” [PDF] 8. Albert Bandura, Gian Vittorio Caprara, Claudio Barbaranelli3, Maria Gerbino, and ConcettaPastorelli, “Role of Affective Self-Regulatory Efficacy in Diverse Spheres of Psychosocial Functioning” 9. Kwak, K., & Bandura, A. (1998). Role of perceived self- efficacy and moral disengagement in antisocial conduct. Manuscript, Osan College, Seoul, Korea. 10, Christine Galbreath Jernigan, What do Students Expect to Lear? The Role of Learner Expectancies, Beliefs, and Attri- butions for Success and Failure in Student Motivation. General + Bandura, A. (1997). Selfefficacy: The exercise of control. New York: WH Freeman and Company. + Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, 52, 1-26. + Baron, A. Robert (2004). Social Psychology, Tenth Edition. External articles and further reading ‘+ Information on Self-Efficacy; A Community of Scholars, + Pajares, F., & Urdan, T. (Eds.). (2006). Adolescence and Education, Vol. 5: Self-Efficacy Beliefs of Adolescents. Greenwich, CT: Information Age Publishing, + Seifert, Timothy L.., Understanding Student Motivation. Memorial University of Newfoundland, St John’s, Newfoundland. 2004 + Banyard, Philip (2002). Psychology in Practice: Health. Hodder and Stoughton. ISBN 0-340-8449%Chaplr 4 Assessment inthe Afetve Domain rectly facing you tum their chairs and bodies around to face you when signaled to do so. CHAPTER EXERCISES ‘A. What is a self-report? What should such a self-report essen- tially contain? How should a teacher instruct the students in writing self-reports? B. What is a rating scale? Enumerate the various types of rat- ing scales and explain how these rating scales are forinu- lated? C. What is the difference between a Thurstone scale and a Likert Scale? In what way does Guttman scaling improve on both types of scales? D. What is a semantic differential scale? Illustrate the use of a semantic differential scale in the measurement of attitude E. Construct a rating scale for each of the following situations: . Measuring attitude towards Mathematics . Measuring interest in literary arts 1 2 3. Motivation to study instrument 4, Predisposition to Work Instrument 5. Measuring attitude towards language studies F. Construct a checklist for each of the following activities: 1, Classroom observation for a practice teacher 2. Good interpersonal relations checklist 3. Checklist for behavior demonstrating good manners and right conduct 4, Checklist for good practices in typing 5. Checklist for good practices in laboratory experiments60 Aowsceo Metso w Eoueanonn ASsesoenr wo Ewwuten (Assess oF Leanne 2) Checklists for Teachers By: Sandra F. Rief, M.A.E74 (1997) Getting students’ attention ‘Ask an interesting, speculative question, show a picture, tell a little story, or read a related poem to generate discussion and terest‘in the upcoming lesson. Try “playfulness,” silliness, a bit of theatrics (props and storytelling) to get attention and peak interest Use storytelling, Students of all ages love to hear stories, ‘especially personal stories. It is very effective in getting at- tention Add a bit of mystery. Bring in an object relevant to the up- coming lesson in a box, bag, or pillowcase. This is a wonderful ‘way to generate predictions and can lead to excellent discus- s. sions or writing acti Signal students auditorily: ring a bell, use a beeper or timer, play a bar of music on the piano or guitar, etc. Vary your tone of voice: loud, soft, whispering. Try making a louder command “Listen! Freeze! Ready!” followed by a few seconds of silence before proceeding in a normal voice to give directions. Use visual signals: flash the lights or raise your hand which signals the students to raise their hands and close thei mouths until everyone is silent. Frame the visual material you want students to be focused con with your hands or with @ cotored box around it, If using an overhead, place an object (e.g, little toy car or plastic figure) to be projected on the sereen to get attention. Clearly signal: “Everybody... Ready...” Color is very effective in getting attention. Make use of col- red dry-erase pens on white boards, colored overhead pens for transparencies and overhead projectors, and colored paper to highlight key words, phrases, steps to computation prob- lems, spelling patterns, ete. Model excitement and enthusiasm about the upcoming lesson. Use eye contact. Students should be facing you when you are speaking, especially while instructions are being given. If students are seated in clusters, have those students not di-(Copter 4 - Assessment in the Aectve Domain between the degree to which one values algebraic equations. For ex- ample, the difference between strong agreement and agreement is not necessatily the same as the difference between disagreement and agreement. Strictly, even demonstrating that categories are ordinal re- quires empirical evidence based on pattems of responses (Andrich, 1978). In 1944, Guttman suggested that attitude should be measured by ‘multidimensional scales, as opposed to unidimensional scales such as those developed by Thurstone and Likert, Guttman pointed out that there should be a multidimensional view of the attitude construct. He developed the Scalogram Analysis, Cumulative Scaling, or as usually called, Guttman scalling. The major characteristic of this scale is that the response to one item helps predict the responses to other items For instance, if the individual responds negatively to the item “I like oranges”, he is not likely to respond positively to the item “Oranges are great for breakfast”, ‘Checklists ‘The most common and perhaps the easiest instrument in the affective domain to construct is the checklist. A checklist consists of simple items that the student or teacher marks as “absent” or “present”, Here are the steps in the construction of a checklist: + Enumerate all the attributes and characteristics you wish to observe relative to the concept being measured. For instance, if the concept is “interpersonal relation”, then you might want to identify those indicators or attributes which constitute an evidence of good interpersonal relation. + Arrange these attributes as a “shopping” list of characteris- tics + Ask the students to mark those attributes or characteristics which are present and to leave blank those which are not Below is an example of a checklist for Teachers (Observational Guide) with emphasis on the behavior: “Getting Students’ Attention” by Sandra F. Rief (1997). Notice that the observational guide will probably consist of several other items like: focusing students’ atten- tion, maintaining students’ attention, and keeping students on-task dur- ing seatwork, each of which requires corresponding series observable behaviors like the example given for “getting students” atte 59 ‘The mast common and perhaps the easiest instrument inthe affective domain to ‘construct is the checks A checks consists of spe tems that the student or teacher marks as “absent” or ‘present.58 Anwaceo Merion m Eoucrnous ASSESNEXT Ato EL (ASSESGuEn o eX 2) 3. Lam not interested in how blacks rate socially, (scale value = 5.4) 4, A refusal to accept blacks is not based on any fact of nature, but on a prejudice which should be overcome. (scale value = 7.9) I believe that blacks deserve the same social privledges as whites. (scale value = 10.3) In 1932, Likert developed the method of summated ratings (or Likert’s scale), which is still widely used. The Likert scale requires that individuals tick on a box to report whether they “strongly agree”, “agree”, are “undecided”, “disagree”, or “strongly disagree”, in response to a large number of items concerning an attitude object or stimulus. Likert scales are derived as follows: First, you pick individual items to include. You choose individual items that you know correlate highly with the total score across items. Second, you choose how to scale each item. For example, you construct labels for each scale value (¢.g., | to 11) to represent the interpretation to be assigned to the number (¢.g., disagree strongly = 1, disagree slightly = 2, etc.)Third, you ask your target audience to mark each item. Fourth, you derive a target's score by adding the values that target identified on each item, Below is an example of the use of a Likert Scale: Statement: I do not like to solve algebraic equations. Response options: * 1. Strongly Disagree * 2. Disagree *3. Agree * 4, Strongly Agree It is common to treat the numbers obtained from a rating scale directly as measurements by calculating averages, or more generally any arithmetic operations. Doing so is not however justified. In terms of the levels of measurement proposed by S.S. Stevens, the data are ordinal categorizations. This means, for example, that to agree strongly with the above statement implies a least favourable perception of al- gebraic equations than does to agree with the statement. However, the numbers are not interval-level measurements in Stevens’ schema, which means that equal differences do not represent equal intervalsChaplr 4 ~ Assessment in he Afectve Domain affective responses. The EPA system is notable for being a ‘multi-variate approach to affect measurement. It is also a ‘generalized approach, applicable to any concept or stimulus, and thus it permits comparisons of affective reacti widely disparate things. EPA ratings have been obtained for hundreds of word concepts, for stories and poems, for social roles and stereotypes, for colors, sounds, shapes, and for in- dividual persons. + The SD has been used as a measure of attitude in a wide variety of projects. Osgood, et al., (1957) report exploratory studies in which the SD was used to assess attitude change as a result of mass media programs and as a result of mes- sages structured in different ways . Their chapter on attitude balance or congruity theory also presents significant applica- tions of the SD to attitude measurement. The SD has been used by other investigators to study attitude formation (e.g., Barclay arid Thumin, 1963), attitudes toward organizations (eg, Rodefeld, 1967), attitudes toward jobs and occupations (eg, Triandis, 1959; Beardslee and O’Dowd, 1961; Gusfield and Schwartz, 1963), and attitudes toward minorities (e.g., Prothro and Keehn, 1957; Williams, 1964; 1966). The results in these, and many other studies, support the validity of the SD as a technique for attitude measurement. son Thurstone and Likert Scales Thurstone is considered the father of attitude measurement. He addressed the issue of how favorable an individual is with regard to a given issue. He developed an attitude continuum to determine the Position of favorability on the issue. Below is an example of a Thurstone scale of measurement. Example (from Thurstone, 1931): Directions. Put a check mark in the blank if you agree with the item. 1. Blacks should be considered the lowest class of human beings. (scale value = 0.9) 2. Blacks and whites must be kept apart in all social affairs where they might be taken as equals. (scale value = 3.2)56 Aowncen Mevioos m Eoucncval Assessver no Ew (Asessve oF Lenn 2) bipolar scales defined with contrasting adjectives at each end. An example of an SD scale is: + Good —— — — —-——_— Bad oe ee ee Usually, the position marked 0 is labeled “neutral,” the 1 positions are labeled “slightly,” the 2 positions “quite,” and the 3 positions “extremely.” In the illustration above, for instance, a ‘“3” close to good ‘would mean an “extremely good” reaction while a “3” close to bad would be an “extremely bad” reaction. The scale actually measures two things: directionality of a reaction (e.g, good versus bad) and also intensity (slight through extreme). Usually, a person is presented with some concept of interest without any other explanatory remarks, ¢.g., Math teacher, and asked to rate it on a number of such scales. Ratings are combined in various ways to describe and analyze the person’s feelings. A number of basic considerations are involved in SD methodology: * Bipolar adjective scales are a simple, economical means for obtaining data on people’s reactions. With adaptations, such scales can be used with adults or children, persons from all walks of life, and persons from any culture. + Ratings on bipolar adjective scales tend to be correlated, and three basic dimensions of response account for most of the co-variation in ratings. The three dimensions, which have been labeled Evaluation, Potency, and Activity (EPA), have been verified and replicated in an impressive variety of stud- ies. + Some adjective scales are almost pure measures of the EPA dimensions; for example, good-bad for Evaluation, powerful- powerless for Potency, and fast-slow for Activity. Using a few pure scales of this sort, one can obtain, with considerable economy, reliable measures of a person’s overall response to something, Typically, a concept is rated on several pure scales associated with a single dimension, and the results are aver- aged to provide a single factor score for each dimension. Measurements of a concept on the EPA dimensions are re- ferred to as the concept’s profile. + EPA measurements are appropriate when one i interested inChapter 4 = Assessment inthe Aecive Domain 5. The phrase: “Motivation-Hygiene” is often used to describe Herzberg’s two-factor theory. What are hygiene factors? Why are they important in the educative process? 6. What is “self-efficacy”? How does this concept figure in the educational measurement field? Why is this concept important to teachers? 7. Give one example of a leaming competency objective in the affective domain for each of the levels in the taxonomy of Kratwoh! et. al. Assume that you are teaching an English literature subject. 8. Describe the potential of tapping the affective domain in en- hancing the learning of students. 4.3 Development of Assessment Tools Assessment tools in the affective domain, in particular, those which are used to assess attitudes, interests, motive is, and self-ef- ficacy, have been developed. There are certain good practices that are considered good practices in developing these instruments, We consider a few of the standard assessment tools in the affective domain. ‘Self-Report. Self report is the most common measurement tool in the affective domain. It essentially requires an individual to provide ‘an account of his attitude or feelings toward a concept or idea or people. Self reports are also sometimes called “written reflections”, In using this measurement tool, the teacher requires the students to write his/her thoughts on a subject matter, like, “Why I Like or Dislike Mathematics”. The teacher ensures that the students write something which would demonstrate the various levels of the taxonomy e.g. low- est level of receiving up to characterization, Rating Scales. A rating scale is a set of categories designed to elicit information about a quantitative attribute in social science. Com- mon examples are the Likert scale and 1-10 rating scales for which a person selects the number which is considered to reflect the perceived quality of a product. The basic feature of any rating scale is that it consists of a number of categories. These are usually assigned inte- gers. Semantic ferential Scales The Semantic Differential (SD) tries to assess an individual's reaction to specific words, ideas or concepts in terms of ratings on 55 Seif report isthe ‘most common ‘measurement foo in the affective domain. The basi feature of any rating sca) Is that & consists of a number of categories. Thes: are usualy assigned ifegenons Eowcaronn Assessuer ao Euarion (ASSESSuEN OF Lenn 2) execute the courses of actions required to manage prospective situa- tions. Unlike efficacy, which is the power to produce an effect (in essence, competence), self-efficacy is the belief (Whether or not accu- rate) that one has the power to produce that effect. : It is important here to understand the distinction between self steem and self-efficacy. Self esteem relates to a person’s sense of self-worth, whereas self efficacy relates to @ person’s perception of their ability to reach a goal, For example, say a person is a terrible rock climber. They would likely have a poor self-efficacy in regard to rock climbing, but this wouldn't need to affect their self-esteem; most people don’t invest much of their self esteem in this activity. Research on leaming has indicated that in certain circumstances, having less self-efficacy for a subject may be helpful, while more negative attitudes towards how quickly/well one will lea, can actually prove of benefit. One study uses the foreign language classroom to examine students’ beliefs about learning, perceptions of goal attain- ‘ment, and motivation to continue language study. (Christine Galbreath Jernigan, What do Students Expect to Learn? The Role of Learner Expectancies, Beliefs, and Attributions for Success and Failure in Student Motivation.) Survey and interview results indicated students’ attributions for success and failure and their expectations for certain subjects’ learning ability played a role in the relationship between goal attainment and volition. It appears that over-efficaciousness negatively affected student motivation. For other students who felt they were “bad at languages,” their negative beliefS increased their motivation to study. EXERCISES 1. Enumerate the different levels in the taxonomy of the affec- tive domain. Discuss each of these levels. 2. What is an “attitude”? Why is the study of attitude impor- tant? How will understanding of attitudes and predisposition enhance teaching? What is “motivation”? Discuss the need theories on motiva- tion: Maslow, Herberg and ERG theories. 4. Why are the “needs” as presented by Maslow arranged in a hierarchy? What do you mean by a hierarchy of needs?Chapt & ~ Assessment inthe fete Domain consistently recorded low scores against management/leadership - Em- ployees were optimisitic about success but happy to complain about leadership since their hygiene factors had not been addressed. Mes- sage - sort the hygiene, then drive the motivation. Finally, created by Clayton Alderfer, Maslow’s hierarchy of needs ‘was expanded, leading to his ERG theory (existence, relatedness and growth). Physiological and safety, the lower order needs, are placed in the existence category, Love and self esteem needs in the relatedness category. The growth category contained the self actualization and self ‘esteem needs. Motivation is of particular interest to educational psychologists because of the crucial role it plays in student learning. However, the specific kind of motivation that is studied in the specialized setting of education differs qualitatively from the more general forms of motiva- tion studied by psychologists in other fields. Motivation in education can have several effects on how students learn and their behavior towards subject matter (Ormrod, 2003). It can: 1. direct behavior toward particular goals. 2. lead to increased effort and energy: 3. increase initiation of, and persistence in, activities 4 enhance cognitive processing. 5. determine what consequences are reinforcing. 6. lead to improved performance. Because students are not always internally motivated, they some- ‘times need situated motivation, which is found in environmental condi- tions that the teacher creates, There are two kinds of motivation: + Intrinsic motivation occurs when people are internally m¢ vated to do something because it either brings them pleasure, they think it is important, or they feel that what they are learning is morally significant + Extrinsic motivation comes into play when, a student is com- pelled to do something or act a certain way because of fac- tors external to him or her (like money or good grades). Finally, the last concept relates to self-efficacy. Self-efficacy is fn impression that one is capable of performing in a certain manner or attaining certain goals. It is a belief that one has the capabilities to 5352 Ao Mena a Coven. ASSESS 0 Eun (ASSESSuET oF Lea 2) are many theories that explain human motivation. The need theory is one of these theories. Abraham Maslow’s hierarchy of human needs theory is the most widely discussed theory of motivation. The theory can be summarized as thus: E + Human beings have wants and desires which influence their behaviour; only unsatisfied needs can influence behavior, satisfied needs cannot + Since needs are many, they are arranged in order of impor- tance, from the basic to the complex. + The person advances to the next level of needs only after the lower level need is at least minimally satisfied. + The further the progress up the hierarchy, the more individu- ality, humanness and psychological health a person will show. The needs, listed from basic (lowest, earliest) to most complex (highest, latest) are as follows: + Physiological: food, clothing, shelter + Safety and security: home and family + Social: being in a community + Self esteem + Self actualization Herzberg’s two factor theory is another need theory of moti- vation, Prederick Herzberg’s two factor theory, concludes that certain factors in the workplace result in job satisfaction, while others'do not, but if absent lead to dissatisfaction. He distinguished between: + Motivators; (e.g. challenging work, recognition, responsibility) which give positive satisfaction, and + Hygiene factors: (e.g. status, job security, salary and fringe benefits) which do not motivate if present, but if absent will result in demotivation. ‘The name Hygiene factors is used because, like hygiene, the presence will make you healthier, but absence can cause health deterioration ‘The theory is sometimes called the “Motivator-Hygiene Theory.” From a practical point of view (vs academic) Herzberg’s two factor theory has proven more powerful than Maslow since its concepts are simpler to understand. Steve Bicknell did considerable research into Employee Engagement Data. In particular, the analysis of verbatim comments cover 50 companies found there was a common theme between low hygiene - high motivator and low Employee Engagement. Employees(Chapter 4 ~ Assessment inthe Atexive Dorain associate it with “calm and peace”. C. Behavioral Intentions- Behavioral-inientibné te-our goals; aspirations, and our expected responses to the attitude object. D. Evaluation- Evaluations are often considered the central component of attitudes. Evaluations consist of the imputation of some degree of goodness or badness to an attitude object. When we speak of a positive or negative attitude toward an object, we are referring to the evaluative compo- nent. Evaluations are a function of cognitive, affect and behavioral intentions of the object. It is most often the evaluation that is stored in memory, often without the corres- ponding cognitions and affect that were responsible for its formation (Robert Scholl, University of Rhode Island, 2002), Why study attitudes? Obviously, attitudes can influence the way wwe act and think in the social communities we belong. They can func~ tion as frameworks and references for forming conclusions and inter- preting or acting for or against an individual; individuals, a concept or ‘an idea. For instance, think about your attitudes toward “drinking alco- holic beverages” or “gambling” or “going on an all-night bar hopping spree every night”. Or, perhaps, think about your attitude towards “mathematics and mathematical equations”, Do these attitudes shape the way you think and correspondingly act? What is your response? How is your response informed by each of these attitudes? Several studies in the past, for instance, concluded that poor performance in school mathematics cannot be strictly attributable to differential mental abilities but to the students’ attitudes toward the subject. When mathematics classes are recited, students with negative attitude towards mathematics tend to pay less attention and occupy minds with something else. Thus, attitudes may influence beha- vior. People will behave in ways consistent with their attitudes. Motivation. Motivation is a reason or set or reasons for engag- ing in a particular behavior, especially human behavior as studied in psychology and neuropsychology. The reasons may include basic needs (eg, food, water, shelter) or an object, goal, state of being, or ideal that is desirable, which may or may not be viewed as “positive,” such as seeking a state of being in which pain is absent. The motivation for a behavior may also be attributed to less-apparent reasons such as altruism or morality. According to Geen (1995), motivation refers to the initiation, direction, intensity and persistence of human behavior. There 5150 ‘ounce Nencos w Eaveanou Assesses Euan (ASSESSMENT Liason 2) Likewise, we provide some examples of verbs or behavioral terms that can be used to express learning competencies or objectives in the affective domain. We cannot stress enough the importance of using behavioral terms in specifying our learning competencies. Behavioral terms tend to simplify the assessment and measurement methodologies that are suggested in this textbook. Behavioral objectives focus on observable behaviors which can then be easily translated in quantitative terms. Receiving | Responding | Valuing Organization | Charactorization + accept [+ complete | + accept * cody * intemal + atend + comply * detend + dsciminate | + verity + deveion | + cooperate | + devote + display + recognize | + discuss * pursue + order + examine | + seek + organize + obey + systomate + respond + weigh Figure 2 - Behavioral Verbs Appropriate for the Affective Domain In the affective domain, and in particular, when we consider Teaming competencies, we also consider the following focal concepts: Attitudes. Attitudes are defined as a mental predisposition to act that is expressed by evaluating a particular entity with some degree of favor or disfavor. Individuals generally have attitudes that focus on objects, people or institutions. Attitudes are also attached to mental categories. Mental orientations towards concepts are generally referred to as values. Attitudes are comprised of four components: ‘A. Cognitions- Cognitions are our beliefs, theories, expectan- cies, cause-and-effect beliefs, and perceptions relative to the focal object. This concept is not the same as “feelings” but just a statement of beliefs and expectations which vary from one individual to the next. B. Affect- The affective component refers to our feeling with respect to the focal object such as fear, liking, or anger. For instance, the color “blue” evokes different feelings for differ- ent individuals: some like the color blue but others do not. Some associate the color blue with “loneliness” while othersChepler 4 ~ Assessment in ne Afectie Donain censure that learning is focused clearly enough that both stu- dents and teacher know what is going on, and so learning can be objectively measured. Different archers have differ- ent styles, so do different teachers. Thus, you can shoot your arrows (objectives) in many ways. The important ‘thing is that they reach your target (goals) and score that, bullseye! We have reproduced the taxonomy of educational objectives in the affective domain below as adopted from Krathwohl. Each level of the affective domain is given a description and an example of an appropriate objective or learning competency is provided. Notice that it far more difficult to state an objective in the affective domain because they often refer to feelings and internal processes of the mind and body that cannot be tested and measured using traditional methods. We also mention in passing that we assess and measure the affective domain in schools but such measurements will not be used to grade students on this domain, Level Definition Example Receiving Being aware of or Individual would aendng to something | read a book passage in the emiconment about cil rights, Responding | Showing some new | Individual would answer questons behaviors 2s a result | about the book, read enother of experience book by the same author, another book about civil rights, ee Valuing Showing some definite | The individual might involvement or ddemonstte this by voluntarily conmient tending a lecture on civil rights. Organization | Inlegraing a new The individual might arange value into one's 2 Gil rights ral general set of values, ‘ving it some ranking among one's general Priore Characterization | Acting consistent wih | The individual is firmly committed by Value the new value to the valve, perhaps becoming 2 Gil rights leader. Figure 1 ~ The Taxonomy inthe Afectve Domain ‘Adapted from: Krathwohl, educational objectives. Handbook i Mekay. Bloom, B., & Masia, B. (1956). Taxonomy of fective domain. New York: David 4948 ‘Avicen Merioosn Eoucaions AssesovexT wo EwuunTon (ASSessuen o LENING 2) scientists, learning science and specific science topics. We want to find teaching methods that encourage students and draw them in. Affective topics in educational literature include attitudes, motivation, communication styles, classroom management styles, leaning styles, use of technology in the classroom and nonverbal communication, inter- ests, predisposition and self-efficacy As teachers, we need to be careful about our own actions that may negatively impact on students? attitudes which go straight into the affective domain. For instance, facial expressions that reveal sarcasm, body movements that betray distrust and dislike, should all be avoided. The affective domain is the least studied and most often over- looked domain in educational literature despite the fact that almost every researcher or author begins with a premise on the importance of the affective domain in the teaching-leaming process. The reason, perhaps, is the fact that the affective domain is the most nebulous and the hardest to evaluate of Bloom’s three domains. Traditional assessment procedures, for instance, concentrate on the cognitive aspects of leaming and as teachers majority of us typically focus our efforts on the development of tests and instruments for measuring cognitive leaning. However, it is important to realize that by tapping the potentials of the affective domain in enhancing learning, we increase the likelihood of real and authentic learning among our stu- dents, Similarly, students may experience affective roadblocks to lean- ing that can neither be recognized nor solved when using a purely cognitive approach. 4.2 Affective Learning Competencies. Affective desired learning competencies are often stated in the form of instructional objectives. What then are instructional objectives? ‘+ Instructional objectives are specific, measurable, short-term, observable student behaviors. * Objectives are the foundation upon which you can build lessons and assessments that you can prove meet your overall course or lesson goals. + Think of objectives as tools you use to make sure you reach your goals. They are the arrows you shoot towards your target (goal). + The purpose of objectives is not to restrict spontaneity or constrain the vision of education in the dis.Chapter 4 ~ Assessment inthe Alectve Domain 4.1 The Taxonomy in the Affective Domain ‘The taxonomy in the affective domain contains a large number of objectives in the literature expressed as interests, attitudes, apprecia- tions, values, and emotional sets or biases. [from Krathwohl et al, 1964 ] The descriptions of each step in the taxonomy culled from Kratwohl’s Taxonomy of Affective Domain (1964) are given as follows: Receiving is being aware of or sensitive to the existence of certain ideas, material, or phenomena and being willing to tolerate them. Examples include: to differentiate, to accept, to listen (for), to respond to. Responding is committed in some small measure to the ideas, ‘materials, or phenomena involved by actively responding to them. Examples are: to comply with, to follow, to commend, to volunteer, to spend leisure time in, to acclaim. ‘Valuing is willing to be perceived by others as valuing certai ideas, materials, or phenomena. Examples include: to increase mea- sured proficiency in, to relinquish, to subsidize, to support, to debate. Organization is to relate the value to those already held and bring it into a harmonious and internally consistent philosophy. Examples are: to discuss, to theorize, to formulate, to balance, to examine, Characterization by value or value set is to act consistently in accordance with the values he or she has internalized. Examples include: to revise, to require, to be rated high in the value, to avoid, to resist, to manage, to resolve, If we are desirous to apply the continuum of Krathwohl et al. to our teaching, then we are encouraging students to not just receive information at the bottom of the affective hierarchy. Instead, as teachers, we would like for them to respond to what they learn, to value it, to organize it and maybe even to characterize themselves as environmentalists, geology majors or earth scientists, Notice that in these science subjects, for instance, it is important to also mention and perhaps study the biographies of great scientists since these serve as inspiration for them to emulate the way that great scientists have led simple lives and devoted their talents to the cause of science. We are also interested in students’ attitudes toward science, a7Counrer 4 ASSESSMENT IN THE AFFECTIVE DOMAIN , ‘hike the Zogntve domain ach emphasizes “reasurements of “easoning end the rental facies of ‘he student, the fective domain descnbes learning objectives that emphasize @ feeing tone, an motion, oF @ degree of soceptance or rejection. Is, admittedly, afar more diet somain to bjectvely analyze Ind assess since afectve objectives vary from simple attention to selected phenom ena fo complex but intoraly consistent quaites of character and conscience. 46 The affective domain is part of a system that was published in 1965 for identifying, understanding and addressing how people learn, In the Bloom's taxonomy published in 1965, three domains were identified: Cognitive, affective and psychomotor domains. In this Chapter, we shall be concerned with the second of these domains which is the affective domain Unlike the cognitive domain which emphasizes measurements of reasoning and the mental faculties of the student, the affective domain describes learning objectives that emphasize a feeling tone, an emotion, or a degree of acceptance or rejection. It is, admittedly, a far more difficult domain to objectively analyze and assess since affective objec- tives vary from simple attention to selected phenomena to complex but internally consistent qualities of character and conscience. Nevertheless, much of the educative process needs to deal with assessment and measurement of students” abilities in this domain, For instance, it is often heard that certain people are “schooled” but not “educated”, This cliché” simply refers to the fact that much of the processes in education today are aimed at developing the cognitive aspects of development and very little or no time is spent on the development of the affective do- main, The development of the psychomotor domain is also an important consideration in education. However, due to space and time limitations, this book shall not cover the appropriate measurement and evaluation techniques in the psychomotor domain. The interested reader, however, is referred to the list of references given at the end of this chapter for more information We shall first discuss the taxonomy developed in the affective domain as a starting point of our discussions on measurement and evaluation in this particular educational domain of ‘interest.‘Chaper 3~ Product Ooned Peomance-Sasod Assessment Chapter References Brookhart, S. M. (1999). The Art and Science of Classroom Assess- ‘ment; The Missing Part of Pedagogy. ASHE-ERIC Higher Edu- cation Report (Vol. 27, No.1). Washington, DC: The George Washington University, Graduate School of Education and Human Development. Chicago Public Schools (1999). Rubric Bank. [Available online at: http://intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/ Rubric_Bank/rubric_bank.html]. Danielson, C. (1997a). A Collection of Performance Tasks and Ru- ries: Middle School Mathematics. Larchmont, NY: Eye on Edu- cation Ine. Danielson, C. (1997). A Collection of Performance Tasks and Ru- brics: Upper Elementary Schoo! Mathematics. Larchmont, NY: Eye on Education Inc. Danielson, C. & Marquez, E. (1998). 4 Collection of Performance Tasks and Rubrics: High School Mathematics. Larchmont, NY: Eye on Education Inc. ERIC/AE (2000a). Search ERIC/AE draft abstracts. [Available online at: http://ericae.nev/sinprog htm], ERIC/AE (2000b). Scoring Rubrics - Definitions & Construction [Available online at: _http://ericae.net/faqs/rubrics/ scoring_rubrics.htm]}. Knecht, R., Moskal, B. & Pavelich, M. (2000). “The Design Report Rubric: Measuring and Tracking Growth through Success”, Paper presented at the annual meeting of the American Society for Engineering Education, Leydens, J. & Thompson, D. (August, 1997), Writing Rubrics Design (EPICS) I, Internal Communication, Design (EPICS) Program, Colorado Schoo! of Mines. Moskal, B. (2000). Assessment Resource Page. (Available online at: ittp:/Avww.mines.edu/Academic/assess/Resource.htm} Schrock, K. (2000). Kathy Schrock’s Guide for Educators. [Available online at: http://school.discovery.com/schrockguide/ assess.html]. State of Colorado (1998). The Rubric. [Available online at: http:// www.edestate 45Aowarcee Mericos mw Eoucanos ASSESSuNT 0 Eton (ASsessue or Lene 2) 44 10. 13, u. 12, 4 15. Writing a short computer program on “Roots of a qua- Menions nEoucaena Assessuer aso Eien (ASsessueu or Lennie 2) 3.2. Task Designing How should a teacher design a task for product-oriented perfor- mance based assessment? The design of the task in this context de- pends on what the teacher desires to observe as outputs of the stuc dents. The concepts that may be associated with task designing in- clude: a. Complexity. The level of complexity of the project needs to be within the range of ability of the students. Projects that are too simple tend to be uninteresting for the students while projects that are too complicated will most likely frustrate them, b. Appeal. The project or activity must be appealing to the stu dents. It should be interesting enough so that students are ‘encouraged to pursue the task to completion. It should lead to self-discovery of information by the students, c, Creativity. The project needs to encourage students to exer- cise creativity and divergent thinking. Given the same set of ‘materials and project inputs. how does one best present the project? It should lead the students into exploring the various possible ways of presenting the final output. d. Goal-Based. Finally, the teacher must bear in mind that the project is produced in order to attain a learning objective. Thus, projects are assigned to students not just for the sake of producing something but for the purpose of reinforcing learning. Example: Paper folding is a traditional Japanese art. However, it can be used as an activity to teach the concept of plane and solid figures in geometry. Provide the students with a given number of col- ored papers and ask them to construct as many plane and solid fig- ures from these papers without cutting them (by paper folding only) Exercise 3.1 Design a project or task for each of the following leaning objec- tives: 1. Analyze the events leading to Rizal’s martyrdom, 2. Differentiate between monocotyledon and dicotyledon. 3. Find an approximate value of the gravitational constant, g. 4, Illustrate the concept of “diffusion”,