131:
119:
243:
considering only the right answers. This massive loss of information can be explained by the fact that the "wrong" answers are removed from the information being collected during the scoring process and are no longer available to reveal the procedural error inherent in right-wrong scoring. The procedure bypasses the limitations produced by the linear dependencies inherent in test data.
219:
Third, although topic-based subtest scores are sometimes provided, the more common practice is to report the total score or a rescaled version of it. This rescaling is intended to compare these scores to a standard of some sort. This further collapse of the test results systematically removes all the
156:
The first shows scoring information loss. The teacher knows whether the student got the right answer, but does not know how the student arrived at the answer. If the answer is wrong, the teacher does not know whether the student was guessing, made a simple error, or fundamentally misunderstands the
242:
This RSE approach provides an interpretation of every answer, whether right or wrong, that indicates the likely thought processes used by the test taker. Among other findings, this chapter reports that the recoverable information explains between two and three times more of the test variability than
234:
This commentary suggests that the current scoring procedure conceals the dynamics of the test-taking process and obscures the capabilities of the students being assessed. Current scoring practice oversimplifies these data in the initial scoring step. The result of this procedural error is to obscure
204:
answers reflect interpretation departures from the expected one, these answers should show an ordered relationship to whatever the overall test is measuring. This departure should be dependent upon the level of psycholinguistic maturity of the student choosing or giving the answer in the vernacular
45:
interpretation, or occasionally both. A norm-referenced interpretation means that the score conveys meaning about the examinee with regards to their standing among other examinees. A criterion-referenced interpretation means that the score conveys information about the examinee with regard to a
238:
A solution to this problem, known as
Response Spectrum Evaluation (RSE), is currently being developed that appears to be capable of recovering all three of these forms of information loss, while still providing a numerical scale to establish current performance status and to track performance
87:
of 18 and 6 (ACT), and 500 and 100. The upper and lower bounds were selected because an interval of plus or minus three standard deviations contains more than 99% of a population. Scores outside that range are difficult to measure, and return little practical value.
75:
that a score of 65% on form 1 is equivalent to a score of 68% on form 2. Scores on both forms can be converted to a scale so that these two equivalent scores have the same reported scores. For example, they could both be a score of 350 on a scale of 100 to 500.
235:
diagnostic information that could help teachers serve their students better. It further prevents those who are diligently preparing these tests from being able to observe the information that would otherwise have alerted them to the presence of this error.
171:
answers or the sum of item scores (where partial credit is given) is assumed to be the appropriate and sufficient measure of current performance status. In addition, a secondary assumption is made that there is no meaningful information in the
188:
result. The fact that the answer is correct does not indicate which of the several possible procedures were used. When the student supplies the answer (or shows the work) this information is readily available from the original documents.
183:
without any profound understanding of the underlying content or conceptual structure of the problem posed. Second, when more than one step for solution is required, there are often a variety of approaches to answering that will lead to a
64:. A raw score is a score without any sort of adjustment or transformation, such as the simple number of questions answered correctly. A scaled score is the result of some transformation(s) applied to the raw score, such as in
95:
properties of a test; it is something that occurs after the assessment process (and equating, if present) is completed. Therefore, it is not an issue of psychometrics, per se, but an issue of interpretability.
83:
that have scaled scores are the ACT and the SAT. The ACT's scale ranges from 0 to 36 and the SAT's from 200 to 800 (per section). Ostensibly, these two scales were selected to represent a mean and
71:
The purpose of scaled scores is to report scores for all examinees on a consistent scale. Suppose that a test has two forms, and one is more difficult than the other. It has been determined by
34:. One formal definition is that it is "a summary of the evidence contained in an examinee's responses to the items of a test that are related to the construct or constructs being measured."
349:
Powell, Jay C. (2010) Testing as
Feedback to Inform Teaching. Chapter 3 in; Learning and Instruction in the Digital Age, Part 1. Cognitive Approaches to Learning and Instruction. (
231:
answers, 2) what led them astray towards unacceptable answers and 3) where within the body of the test this departure from expectation occurred.
295:
216:
answers are discarded during the scoring process, analysis of these answers for the information they might contain is seldom undertaken.
208:
In this second case it should be possible to extract this order from the responses to the test items. Such extraction processes, the
358:
328:
399:
404:
384:
394:
42:
31:
212:
for instance, are standard practice for item development among professionals. However, because the
292:
200:
guesses, there would be no information to be found among these answers. On the other hand, if
30:
is a piece of information, usually a number, that conveys the performance of an examinee on a
353:, Dirk Ifenthaler, Pedro Isaias, Kinshuk and Demetrios Sampson, Eds.), New York: Springer.
252:
38:
8:
350:
108:
84:
354:
130:
118:
362:
257:
65:
299:
389:
366:
378:
277:
92:
80:
324:
209:
20:
167:, an important assumption has been made about learning. The number of
56:
72:
46:
specific subject matter, regardless of other examinees' scores.
16:
Number, that conveys the performance of an examinee on a test
179:
In the first place, a correct answer can be achieved using
111:. Compare the information provided in these two answers.
107:
A test question might require a student to calculate the
312:
The
Journal of Educational and Psychological Measurement
293:
Iowa
Testing Programs guide for interpreting test scores
220:
information about which particular items were missed.
376:
280:(2001). Test Scoring. Mahwah, NJ: Erlbaum.
99:
377:
91:Note that scaling does not affect the
227:loses 1) how students achieved their
310:Powell, J. C. and Shklov, N. (1992)
54:There are two types of test scores:
37:Test scores are interpreted with a
13:
331:from the original on 30 April 2015
14:
416:
129:
117:
343:
317:
304:
286:
270:
205:in which the test is written.
1:
263:
79:Two well-known tests in the
7:
246:
136:Base = 5 cm; Height = 3 cm
10:
421:
325:"Welcome to the Frontpage"
18:
367:10.1007/978-1-4419-1551-1
100:Scoring information loss
49:
19:Not to be confused with
400:Educational psychology
163:When tests are scored
405:Psychological testing
223:Thus, scoring a test
253:Grading in education
43:criterion-referenced
385:School examinations
314:, 52, 847–865
282:Page 1, sentence 1.
276:Thissen, D., &
395:Standardized tests
351:J. Michael Spector
298:2008-02-12 at the
109:area of a triangle
85:standard deviation
359:978-1-4419-1551-1
142:(Base × Height)
412:
369:
347:
341:
340:
338:
336:
321:
315:
308:
302:
290:
284:
274:
258:Percentile score
133:
121:
66:relative grading
420:
419:
415:
414:
413:
411:
410:
409:
375:
374:
373:
372:
348:
344:
334:
332:
323:
322:
318:
309:
305:
300:Wayback Machine
291:
287:
275:
271:
266:
249:
192:Second, if the
161:
160:
159:
158:
153:
152:
151:
149:
147:
143:
141:
137:
134:
126:
125:
122:
113:
112:
102:
52:
39:norm-referenced
24:
17:
12:
11:
5:
418:
408:
407:
402:
397:
392:
387:
371:
370:
342:
316:
303:
285:
268:
267:
265:
262:
261:
260:
255:
248:
245:
155:
154:
148:(5 cm × 3 cm)
145:
139:
135:
128:
127:
123:
116:
115:
114:
106:
105:
104:
103:
101:
98:
51:
48:
15:
9:
6:
4:
3:
2:
417:
406:
403:
401:
398:
396:
393:
391:
388:
386:
383:
382:
380:
368:
364:
360:
356:
352:
346:
330:
326:
320:
313:
307:
301:
297:
294:
289:
283:
279:
273:
269:
259:
256:
254:
251:
250:
244:
240:
236:
232:
230:
226:
221:
217:
215:
211:
206:
203:
199:
196:answers were
195:
190:
187:
182:
177:
175:
170:
166:
132:
124:Area = 7.5 cm
120:
110:
97:
94:
89:
86:
82:
81:United States
77:
74:
69:
67:
63:
62:scaled scores
59:
58:
47:
44:
40:
35:
33:
29:
22:
345:
333:. Retrieved
319:
311:
306:
288:
281:
272:
241:
237:
233:
228:
224:
222:
218:
213:
207:
201:
197:
193:
191:
185:
181:memorization
180:
178:
173:
168:
164:
162:
93:psychometric
90:
78:
70:
61:
55:
53:
36:
27:
25:
225:right–wrong
210:Rasch model
165:right-wrong
379:Categories
278:Wainer, H.
264:References
57:raw scores
28:test score
21:Score test
176:answers.
329:Archived
296:Archived
247:See also
239:change.
157:subject.
150:= 7.5 cm
138:Area = /
73:equating
229:correct
186:correct
357:
390:Tests
335:2 May
214:wrong
202:wrong
198:blind
194:wrong
174:wrong
169:right
50:Types
355:ISBN
337:2015
60:and
32:test
363:doi
144:= /
41:or
381::
361:,
327:.
68:.
26:A
365::
339:.
146:2
140:2
23:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
