Test items

A test or examination (or "exam") A test may be administered orally, on paper, on a computer, or in a confined area that requires a test taker to physically perform a set of skills. A test may be administered formally or informally. An example of an informal test would be a reading test administered by a parent to a child. An example of a formal test would be a final examination administered by a teacher in a classroom A standardized test is any test that is administered and scored in a consistent manner to ensure legal defensibility. Standardized tests are often used in education, professional certification, psychology (e.g., MMPI), the military, and many other fields. A non-standardized test is usually flexible in scope and format, variable in difficulty and significance. A non-standardized test may be used to determine the proficiency level of students, to motivate students to study, and to provide feedback to students. In some instances, a teacher may develop non-standardized tests that resemble standardized tests in scope, format, and difficulty for the purpose of preparing their students for an upcoming standardized test. A class instructor may for example, administer a test on a weekly basis or just twice a semester. Depending of the policy of the instructor or institution, the duration of each test itself may last for only five minutes to an entire class period.

Advantages
One of the main advantages of standardized testing is that the results can be empirically documented; therefore, the test scores can be shown to have a relative degree of validity and reliability, as well as results which are generalizable and replicable. This is often contrasted with grades on a school transcript, which are assigned by individual teachers. It may be difficult to account for differences in educational culture across schools, difficulty of a given teacher's curriculum, differences in teaching style, and techniques and biases that affect grading. This makes standardized tests useful for admissions purposes in higher education, where a school is trying to compare students from across the nation or across the world. Another advantage is aggregation. A well designed standardized test provides an assessment of an individual's mastery of a domain of knowledge or skill which at some level of aggregation will provide useful information. That is, while individual assessments may not be accurate enough for practical purposes, the mean scores of classes, schools, branches of a company, or other groups may well provide useful information because of the reduction of error accomplished by increasing the sample size. Standardized tests, which by definition give all test-takers the same test under the same (or reasonably equal) conditions, are also perceived as being more fair than assessments that use different questions or different conditions for students according to their race, socioeconomic status, or other considerations.

Disadvantages and criticism

"Standardized tests can't measure initiative, creativity, imagination, conceptual thinking, curiosity, effort, irony, judgment, commitment, nuance, good will, ethical reflection, or a host of other valuable dispositions and attributes. What they can measure and count are isolated skills,

specific facts and function, content knowledge, the least interesting and least significant aspects of learning." Standardized tests are useful tools for assessing student achievement, and can be used to focus instruction on desired outcomes, such as reading and math skills. However, critics feel that overuse and misuse of these tests harms teaching and learning by narrowing the curriculum. According to the group FairTest, when standardized tests are the primary factor in accountability, schools use the tests to define curriculum and focus instruction. Critics say that "teaching to the test" disfavors higher-order learning. While it is possible to use a standardized test without letting its contents determine curriculum and instruction, frequently, what is not tested is not taught, and how the subject is tested often becomes a model for how to teach the subject. Uncritical use of standardized test scores to evaluate teacher and school performance is inappropriate, because the students' scores are influenced by three things: what students learn in school, what students learn outside of school, and the students' innate intelligence. The school only has control over one of these three factors. Value-added modeling has been proposed to cope with this criticism by statistically controlling for innate ability and out-of-school contextual factors.[19] In a value-added system of interpreting test scores, analysts estimate an expected score for each student, based on factors such as the student's own previous test scores, primary language, or socioeconomic status. The difference between the student's expected score and actual score is presumed to be due primarily to the teacher's efforts. Supporters of standardized testing respond that these are not reasons to abandon standardized testing in favor of either non-standardized testing or of no assessment at all, but rather criticisms of poorly designed testing regimes. They argue that testing does and should focus educational resources on the most important aspects of education imparting a pre-defined set of knowledge and skills and that other aspects are either less important, or should be added to the testing scheme.

Test Development
When testing aviation students, the instructor is usually concerned more with criterionreferenced testing than norm-referenced testing. Norm-referenced testing measures a student's performance against the performance of other students. Criterion-referenced testing evaluates each student's performance against a carefully written, measurable, standard or criterion. There is little or no concern about the student's performance in relation to the performance of other students. The FAA knowledge and practical tests for pilots and aircraft maintenance technicians are all criterion referenced because in aviation training, it is necessary to measure student performance agaitist a high standard of proficiency consistent with safety. The aviation instructor constructs tests to measure progress toward the standards that will eventually be measured at the conclusion of the training. For example, during an early stage of flight training, the flight instructor must administer a presolo written exam to student pilots. Since tests are an integral part of the instructional process, it is important for the aviation instructor to be well informed about recommended testing procedures.

Aviation instructors can follow a four-step process when developing a test. This process is useful for tests that apply to the cognitive and affective domains of learning, and also can be used for skill testing in the psychomotor domain. The development process for criterionreferenced tests follows a general-to-specific pattern.

Determine Level-of-Learning Objectives

The first step in developing a test is to state the individual objectives as general, level-of-learning objectives. The objectives should measure one of the learning levels of the cognitive, affective, or psychornotor domains described in Chapter 1. The levels of cognitive learning include knowledge, comprehension, application, analysis, synthesis, and evaluation. For the comprehension or understanding level, an objective could be stated as, "Describe how to perform a compression test on an aircraft reciprocating engine." This objective requires a student to explain how to do a compression test, but not necessarily perform a compression test (application level). Further, the student would not be expected to compare the results of compression tests on different engines (analysis level), design a compression test for a different type of engine (synthesis or correlation level), or interpret the results of the compression test (evaluation level). A general level-of-learning objective is a good starting point for developing a test because it defines the scope of the learning task. List Indicators/Samples of Desired Behavior The second step is to list the indicators or samples of behavior that will give the best indication of the achievement of the objective. Some level-of-learning objectives often cannot be directly measured. As a result, behaviors that can be measured are selected in order to give the best evidence of learning. For example, if the instructor is expecting the student to display the comprehension level-of-learning on compression testing, some of the specific test question answers should describe appropriate tools and equipment, the proper equipment setup, appropriate safety procedures, and the steps used to obtain compression readings. The overall test must be comprehensive enough to give a true representation of the learning to be measured. It is not usually feasible to measure every aspect of a level of-learning objective, but by carefully choosing samples of behavior, the instructor can obtain adequate evidence of learning.

Establish Criterion Objectives The next step in the test development process is to define criterion (performance-based) objectives. In addition to the behavior expected, criterion objectives state the conditions under which the behavior is to be performed and the criteria that must be met. If the instructor developed performance based objectives during the creation of lesson plans, criterion objectives have already been formulated. The criterion objective provides the framework for developing the test items used to measure the level-of-learning objectives. In the compression test example, a criterion objective to measure the comprehension level of learning might be stated as, "The student will demonstrate comprehension of compression test procedures for reciprocating aircraft engines by completing a quiz with a minimum passing score of 70%." Develop Criterion-Referenced Test Items The last step is to develop criterion-referenced test items. The actual development of the test questions is covered in the remainder of this chapter. While developing questions, the instructor should attempt to measure the behaviors described in the criterion objective(s). The questions in the exam for the compression test example should cover all of the areas necessary to give evidence of comprehending the procedure. The results of the test (questions missed) identify areas that were not adequately covered. Performance-based objectives serve as a reference for the development of test items. If the test is the presolo knowledge test, the objectives are for the student to comprehend the regulations, the local area, the aircraft type, and the procedures to be used. The test should measure the student's knowledge in these specific areas. Individual instructors should develop their own tests to measure the progress of their students. If the test is to measure the readiness of a student to take a knowledge test, it should be based on the objectives of all the lessons the student has received. Another source of test items includes FAA knowledge test guides for a particular knowledge test. These sample questions are designed to measure the level-of-leaming desired for pilots or aviation maintenance technicians. As a result, they are a good source of example questions to be used in measuring a student's preparedness to take the knowledge test. However, care must be taken not to teach questions to ensure the student does not merely memorize answers or the letter of the answer. When using questions from any source, whether from a publisher or developed by individual instructors, periodically revising the questions used and changing the letters and positions of the answers will encourage learning the material rather than learning the test.

Types of test items Selection Type

Written tests made up of selection-type items are highly objective. That is, the results of such a test would be graded the same regardless of the student taking the test or the person grading it. Tests that include only selection-type items make it possible to directly compare student accomplishment. For example, it is possible to compare the performance of students within one class to students in a different class, or students under one instructor with those under another instructor. By using selection-type items, the instructor can test on many more areas of knowledge in a given time than could be done by requiring the student to supply written

responses. This increase in comprehensiveness can be expected to increase validity and discrimination. Another advantage is that selection-type tests are well adapted to statistical item analysis. There are five types of selection type test in common use. They are written test, multiple-choice, matching, true-false, and supply type. The most used type is the multiple-choice test item in one of several forms. Listed below are examples of some of the more widely used forms.

Written tests

Indonesian Students taking a written test Written tests are tests that are administered on paper or on a computer. A test taker who takes a written test could respond to specific items by writing or typing within a given space of the test or on a separate form or document. A test developer's choice of which style or format to use when developing a written test is usually arbitrary given that there is no single invariant standard for testing. Be that as it may, certain test styles and format have become more widely used than others. Below is a list of those formats of test items that are widely used by educators and test developers to construct paper or computer-based tests. As a result, these tests may consist of only one type of test item format e.g., multiple choice test, essay test) or may have a combination of different test item formats e.g., a test that has multiple choice and essay items.

Multiple-Choice Test Items

Multiple-choice test items consist of a stem or question and three or more alternative answers with the correct answer sometimes called the keyed response and the incorrect answers called distracters

A multiple-choice test item consists of two parts; the stem which includes the question, statement, or problem, and a list of alternatives or responses. Incorrect answers are called distractors. When properly devised and constructed, multiple-choice items offer several advantages that make this type more widely used and versatile than either the matching or the true-false items.

Multiple-choice test questions may be used to determine student achievement, ranging from acquisition of facts to understanding, reasoning, and ability to apply what has been learned. It is appropriate to use when the question, statement, or problem has the following characteristics. Stem Presented as a Question. This form is generally better than the incomplete stem because it is simpler and more natural. Who is primarily responsible for maintaining an aircraft in an airworthy condition? A. Pilot in command or operator. B. Owner or operator of the aircraft. C. The lead mechanic responsible for that aircraft. Stem as an Incomplete Statement. When using this form, care must be exercised to avoid ambiguity, giving clues, and using unnecessarily complex or unrelated alternatives. VFR cruising altitudes are required to be maintained when flying A. at 3,000 feet or more AGL, based on true course. B. more than 3,000 feet AGL, based on magnetic course. C. at 3,000 feet or more above MSL, based on magnetic heading. Stem Supplemented by an Illustration. This form is useful for measuring the ability to read instruments, or identify objects. (Refer to figure 1.) The acute angle A is the angle of A. Attack. B. Dihedral. C. Incidence.

Multiple Responses is required. This form is a variation of the previous forms in that it contains more than one correct answer, and students are instructed to select all correct answers. There are two families of multiple choice questions. The first family is known as the True/False question and it requires a test taker to choose all answers that are appropriate. The second family is known as One-Best-Answer question and it requires a test taker to answer only one from a list of answers. There are several reasons to using multiple choice questions in tests. In terms of administration, multiple choice questions usually requires less time for test takers to answer, are easy to score and grade, provide greater coverage of material, allows for a wide range of difficulty, and can easily diagnose a test taker's difficulty with certain concepts. As an educational tool, multiple choice items test many levels of learning as well as a test taker's ability to integrate information, and it provides feedback to the test taker about why distractors were wrong and why correct answers were right. Nevertheless, there are difficulties associated with the use of multiple choice questions. In administrative terms, multiple choice items that are effective usually take a great time to construct. As an educational tool, multiple choice items do not allow test takers to demonstrate knowledge beyond the choices provided and may even encourage guessing or approximation due to the presence of at least one correct answer. Which of the following statements is/are generally true regarding the charging of several aircraft batteries together? 1. Batteries of different voltage (but similar capacities) can be connected in series with each other across the charger, and charged using the constant current method. 2. Batteries of different ampere-hour capacity and same voltage can be connected in parallel with each other across the charger, and charged using the constant voltage method. 3. Batteries of the same voltage and same ampere-hour capacity must be connected in series with each other across the charger, and charged using the constant current method. A. 3. B. 1 and 2. C. 2 and 3. Negative Variety Type. This form is not suggested but, if used, always emphasizes the negative word. Which of the following is NOT considered a method of heat transfer? A. Diffusion. B. Conduction. C. Convection. Association Type. This form is useful if a limited number of associations are to be made. Which aircraft has the right-of-way over the other aircraft listed? A. Airship. B. Gyroplane. C. Aircraft towing other aircraft. Definition Type. This form is used to determine knowledge of a specific definition.

Aspect ratio of a wing is defined as the ratio of the A. Wingspan to the wing root. B. Wingspan to the mean chord. C. Square of the chord to the wingspan.

Matching Test Items

Matching test items are used to test a student's ability to recognize relationships and to make associations between terms, parts, words, phrases, clauses, or symbols in one column with related alternatives in another column. When using this form of test item, it is a good practice to provide alternatives in the response column that are used more than once, or not at all, to preclude guessing by elimination. Matching test items may have either an equal or unequal number of selections in each column. Matching-Equal Columns. When using this form, providing for some items in the response column to be used more than once, or not at all, can preclude guessing by elimination. Directions: In the blank before each electrical term in the left-hand column, write the letter corresponding to the unit of measurement which is most closely associated with that term. Each unit of measurement may be used more than once and some units may not be used at all. 1. ____ Electromotive force 2. ____ Electrical power, apparent 3. ____ Electrical power, true 4. ____ Resistance 5. ____ Capacitance 6. ____ Inductance 7. ____ Current 8. ____ Impedance A. Watt B. Volt C. Ampere D. Coulomb E. Ohm F. VAR G. Farad H. Henry

Matching-Unequal Columns. Generally preferable to equal columns. Directions: In the blank before each phrase in the left-hand column, write the letter(s) corresponding to the type(s) of drag which is/are most closely associated with that phrase. Each type of drag may be used more than once, and some types may not be used at all. 1. ____ Occurs when varied currents over an airplane meet and Interact. 2. ____ Results from the turbulent wake caused by the separation Of airflow from the surface of a structure. 3. ____ Caused by the roughness of the airplane's surfaces. 4. ____ Generated by the airflow circulation around the airfoil as It creates lift. A. Form drag B. Induced drag C. Skin friction drag D. Static drag E. Interference drag F. Rolling drag G. Sliding drag

True-False Test Items

A True-False test item requires the student to determine whether a statement is true or false. The chief disadvantage of this type is the opportunity for successful guessing. The true-false test item consists of a statement followed by an opportunity for the student to determine whether the statement is true or false. This item type, with all its variations, has a wide

range of usage. It is well adapted for testing knowledge of facts and details, especially when there are only two possible answers. The chief disadvantage is that true-false questions create the greatest probability of guessing. True-false test items are probably used and misused more than any other selection-type item. Frequently, instructors select sentences more or less at random from textual material and make half of them false by inserting negatives. When tests are constructed in this way, the principal attribute being measured is rote memory rather than knowledge of the subject. Such test construction has aroused antagonism toward selection tests in general and true false questions in particular. It has also decreased the validity of educational evaluations. Some of the principles that should be followed in the construction of true-false items are contained in the accompanying list. Directions: Circle the correct response to the following statements. 1. True or False. To operate within Class B airspace, the aircraft must have two-way radio communication capability and a Mode C transponder. 2. True or False. An aviation maintenance technician must hold an Inspection Authorization to legally conduct annual inspections on small aircraft. Include only one idea in each statement. Use original statements rather than verbatim text. Statements should be entirely true or entirely false. Avoid the unnecessary use of negatives. They tend to confuse the reader. If negatives must be used, underline or otherwise emphasize the negative. Avoid involved statements. Keep wording and sentence structure as simple as possible. Make statements both definite and clear. Avoid the use of ambiguous words and terms (some, any, generally, most times, etc.) Whenever possible, use terms which mean the same thing to all students. Avoid absolutes (all, every, only, no, never, etc.) These words are known as determiners and provide clues to the correct answer. Since unequivocally true or false statements are rare, statements containing absolutes are usually false. Avoid patterns in the sequence of correct responses because students can often identify the patterns. Instructors sometimes deliberately use patterns to make hand scoring easier. This is a poor practice. Make statements brief and about the same length. Some instructors unconsciously make true statements longer than false ones. Students are quick to take advantage of this tendency. If a statement is controversial (sources have differing information), the source of the statement should be listed.

Supply-Type Test Items

The aviation instructor is able to determine the students' level of generalized knowledge of a subject through the use of supply-type questions. Short-answer essay test items are the most common. The supply-type item may be required where a selection-type cannot be devised to properly measure student knowledge. The supply-type requires the students to organize their knowledge.

It demands an ability to express ideas that is not required for a selection-type item. This type item is valuable in measuring the students' generalized understanding of a subject. On the other hand, a supply-type item may evaluate the students' ability to write rather than their specific knowledge of the subject matter. It places a premium on neatness and penmanship. The main disadvantage of supply-type tests is that they cannot be graded with uniformity. There is no assurance that the grade assigned is the grade deserved by the student. The same test graded by different instructors would probably be assigned different scores. Even the same test graded by the same instructor on consecutive days might be assigned altogether different scores. Still another disadvantage of a supply-type test is the time required by the student to complete it and the time required by the instructor to grade it. Everything considered, the disadvantages of the supply-type test appear to exceed the advantages to such an extent that instructors prefer to use the selection-type test. It should be noted that although selection-type tests are best in many cases, there are times where the supply-type is desirable. This would be when there is a need to thoroughly determine the knowledge of a person in a particular subject area. An example of this would be the presolo knowledge exam where it would be difficult to determine knowledge of procedures strictly with selection-type test items. 1. Describe the position of the elevator and ailerons when taxiing a tricycle-gear airplane into a right quartering headwind. _______________________________________________________________________ _________________________________ _______________________________________________________________________ _________________________________
2. What conditions are used in determining the published value of VMC? _____________________________________________________________ ___________________________________________ _____________________________________________________________ ___________________________________________

References
http://en.wikipedia.org/wiki/Test_(assessment) http://en.wikipedia.org/wiki/Test_%28assessment%29#cite_noteConstructing_Written_Test_Questions_For_the_Basic_and_Clinical_Sciences-14 http://www.dynamicflight.com/avcfibook/appendix_a/