Assignment 4 Summary Principles of Language Assessment
SUMMARY PRINCIPLES OF LANGUAGE ASSESSMENT
A. PRACTICALITY
An effective test is practical :
• is not excessively expensive,
• stays within appropriate time constraints,
• is relatively easy to administer, and
• has a scoring/evaluation procedure that is specific and time-efficient.
A test that is prohibitively expensive is impractical. A test of language profi-ciency that takes a student five hours to complete is impractical-it consumes more time (and money) than necessary to accomplish its objective. A test that requires individual one-on-one proctoring is impractical for a group of several hun-dred test -takers and only a handful of examiners.
A test that takes a few minutes for a student to take and several hours for an examiner to evaluate is impractical for most classroom situations. A test that can be scored only by computer is impractical if the test takes place a thousand miles away from the nearest computer. The value and quality of a test sometimes hinge on such nitty-gritty, practical considerations.
B. RELIABILITY
A reliable test is consistent and dependable. If you give the same test to the same student or matched students on two different occasions, the test should yield similar results. The issue of reliability of a test may best be addressed by considering a number of factors that may contribute to the unreliability of a test.
Student-Related Reliability
The most common learner-related issue in reliability is caused by temporary illness,
fatigue, a "bad day," anxiety, and other physical or psychological factors, which may make an "observed"score deviate from one's "true" score. Also included in this category are such factors as a test-taker's "test-wiseness" or strategies for efficient test taking (Mousavi, 2002, p. 804).
Rater Reliability
Human error, subjectivity, and bias may enter into the scoring process. Inter-rater reliability occurs when. two or more scorers yield inconsistent scores of the same test, possibly for lack of attention to scoring criteria, inexperience, iriattention, or even preconceived biases. In the story above about the placement test, the initial scoring plan for the dictations was found to be unreliable-that is, the two scorers were not applying the same standards.
Test Adminlstration Reliability
Unreliability may also result from the conditions in which the test is administered. Administration of a test of aural comprehension in which a tape recorder played items for comprehension, but because of street noise outside the building, students sitting next to windows could not hear the tape accurately. This was a clear case of unreliability caused by the conditions of the test administration. Other sources of unreliability are found in photocopying variations, the amount of light in different parts of the (oom, variations in 'temperature, and even the condi-don of desks and chairs.
Test Reliabllity
Sometimes the nature of the test itself can cause measurement errors. If a test is too ,long, test-takers may become fatigued by the time they reach the later items and hastily respond incorrectly. Timed tests may discriminate against students who do not perform well on a test with a time limit.
C. VALIDITY
By-far the most complex criterion ofan effective test-and arguably the most impor-tant principle-is validity, "the extent to which inferences made from assessment results are appropriate, meaningful and useful in terms of the purpose of the assess-ment" (Gronlund, 1998, p. 226). How is the validity of a test established? There is no final, absolute measure of validity, but several different kinds of evidence may be invoked in support. In some cases, it may be appropriate to examine the extent to which a test calls for performance that matches that of the course or unit of study being tested In other cases, we may be concerned with how well a test determines whether or not students have reached an established set of goals or level of competence. We will look at these five types of evidence below:
• Content-Related Evidence
• Criterion-Related Evidence
• Construct Related Evidence
• Consequential Validity
• Face Validity
D. AUTHENTICITY
A fourth major principle oflanguage testing is authenticity, a concept that is a little slippery to define, especially within the art and science of evaluating and designing tests. Bachman and Palmer, (1996, p. 23) In a test, authenticity may be 'present . in the following ways:
• The language in the test is as natural as possible.
• Items are contextualized rather than isolated.
• Topics are meaningful (relevant, interesting) for the learner.
• Some thematic organization to items is provided, such as through a story line
or episode.
• Tasks represent, or closely approximate, real-world tasks.
E. WASHBACK
A facet of consequential validity, discussed above, is "the effect of testing on teaching and learning" (Hughes, 2003, p. 1), otherwise known among language-testing , specialists as washback. In large-scale assessment, washback generally refers to the effects the tests have on instruction in terms of how students prepare for the test."Cram" courses and "teaching to the test" are examples of such washback. Another form ofwashback that occurs more in classroom assessment is the information that"washes back" to students in the form of useful diagnoses of strengths and weaknesses.
Washback also includes the effects of an assessment on teaching and learning prior to the assessment itself, that is, on preparation for the assessment. Informal per-formance assessment is by nature more likely to have built-in washback effects because the teacher is usually providing interactive feedback. Formal tests can also have positive washback, but they provide no washback if the students receive a simple letter grade or a single overall numerical score. A little bit of washback may also help students through specification of the numerical scores on the various subsections of the test.
Finnaly, washback also implies that students have ready access to you to discuss the feedback and evaluation you have given.
References :
Brown, H. Douglas. Language Assessment: Principles and Classroom Practices. Longman.
A. PRACTICALITY
An effective test is practical :
• is not excessively expensive,
• stays within appropriate time constraints,
• is relatively easy to administer, and
• has a scoring/evaluation procedure that is specific and time-efficient.
A test that is prohibitively expensive is impractical. A test of language profi-ciency that takes a student five hours to complete is impractical-it consumes more time (and money) than necessary to accomplish its objective. A test that requires individual one-on-one proctoring is impractical for a group of several hun-dred test -takers and only a handful of examiners.
A test that takes a few minutes for a student to take and several hours for an examiner to evaluate is impractical for most classroom situations. A test that can be scored only by computer is impractical if the test takes place a thousand miles away from the nearest computer. The value and quality of a test sometimes hinge on such nitty-gritty, practical considerations.
B. RELIABILITY
A reliable test is consistent and dependable. If you give the same test to the same student or matched students on two different occasions, the test should yield similar results. The issue of reliability of a test may best be addressed by considering a number of factors that may contribute to the unreliability of a test.
Student-Related Reliability
The most common learner-related issue in reliability is caused by temporary illness,
fatigue, a "bad day," anxiety, and other physical or psychological factors, which may make an "observed"score deviate from one's "true" score. Also included in this category are such factors as a test-taker's "test-wiseness" or strategies for efficient test taking (Mousavi, 2002, p. 804).
Rater Reliability
Human error, subjectivity, and bias may enter into the scoring process. Inter-rater reliability occurs when. two or more scorers yield inconsistent scores of the same test, possibly for lack of attention to scoring criteria, inexperience, iriattention, or even preconceived biases. In the story above about the placement test, the initial scoring plan for the dictations was found to be unreliable-that is, the two scorers were not applying the same standards.
Test Adminlstration Reliability
Unreliability may also result from the conditions in which the test is administered. Administration of a test of aural comprehension in which a tape recorder played items for comprehension, but because of street noise outside the building, students sitting next to windows could not hear the tape accurately. This was a clear case of unreliability caused by the conditions of the test administration. Other sources of unreliability are found in photocopying variations, the amount of light in different parts of the (oom, variations in 'temperature, and even the condi-don of desks and chairs.
Test Reliabllity
Sometimes the nature of the test itself can cause measurement errors. If a test is too ,long, test-takers may become fatigued by the time they reach the later items and hastily respond incorrectly. Timed tests may discriminate against students who do not perform well on a test with a time limit.
C. VALIDITY
By-far the most complex criterion ofan effective test-and arguably the most impor-tant principle-is validity, "the extent to which inferences made from assessment results are appropriate, meaningful and useful in terms of the purpose of the assess-ment" (Gronlund, 1998, p. 226). How is the validity of a test established? There is no final, absolute measure of validity, but several different kinds of evidence may be invoked in support. In some cases, it may be appropriate to examine the extent to which a test calls for performance that matches that of the course or unit of study being tested In other cases, we may be concerned with how well a test determines whether or not students have reached an established set of goals or level of competence. We will look at these five types of evidence below:
• Content-Related Evidence
• Criterion-Related Evidence
• Construct Related Evidence
• Consequential Validity
• Face Validity
D. AUTHENTICITY
A fourth major principle oflanguage testing is authenticity, a concept that is a little slippery to define, especially within the art and science of evaluating and designing tests. Bachman and Palmer, (1996, p. 23) In a test, authenticity may be 'present . in the following ways:
• The language in the test is as natural as possible.
• Items are contextualized rather than isolated.
• Topics are meaningful (relevant, interesting) for the learner.
• Some thematic organization to items is provided, such as through a story line
or episode.
• Tasks represent, or closely approximate, real-world tasks.
E. WASHBACK
A facet of consequential validity, discussed above, is "the effect of testing on teaching and learning" (Hughes, 2003, p. 1), otherwise known among language-testing , specialists as washback. In large-scale assessment, washback generally refers to the effects the tests have on instruction in terms of how students prepare for the test."Cram" courses and "teaching to the test" are examples of such washback. Another form ofwashback that occurs more in classroom assessment is the information that"washes back" to students in the form of useful diagnoses of strengths and weaknesses.
Washback also includes the effects of an assessment on teaching and learning prior to the assessment itself, that is, on preparation for the assessment. Informal per-formance assessment is by nature more likely to have built-in washback effects because the teacher is usually providing interactive feedback. Formal tests can also have positive washback, but they provide no washback if the students receive a simple letter grade or a single overall numerical score. A little bit of washback may also help students through specification of the numerical scores on the various subsections of the test.
Finnaly, washback also implies that students have ready access to you to discuss the feedback and evaluation you have given.
References :
Brown, H. Douglas. Language Assessment: Principles and Classroom Practices. Longman.
Komentar
Posting Komentar