|
|
Dr. Gregg's Testing & Measurement Corner
Norm Referenced TestingIf someone tells us that they received a score of 86 on an exam we are not sure if that's a good or bad score. If this person appears pleased with himself or herself we would probably assume that they got 86 points out of a possible 100, or 86% of the total number of points possible. On the other hand, if this person got 86 points out of a possible 250 or 34.4% of the total points possible it would paint an entirely different picture. What if we knew that the average score for this exam was 52 points out of 250? With this additional information we would be back to thinking that this person received a good score. We can see from this example that we need more information than just a score in determining how well a person did on a norm-referenced test.
The majority of tests that we are familiar with in education are referred to as "norm referenced" tests. That is, a particular score only has meaning in reference to other scores. The group of scores that a particular score is being compared to is often referred to as the "normative group". To give meaning to a norm referenced test score we need to know the particular score and the mean (average) score of the normative group. Given this information we can determine if a score is good, bad or average. On the other hand, to know just how good a score is we need another piece of information. The additional information that is needed is the "standard deviation" on the test. The standard deviation tells us the "average" amount of dispersion of scores around the mean. If for example almost everyone taking the test gets a score very close to the mean, then the standard deviation of the test would be small. If on the other hand test scores vary dramatically, the standard deviation would likely be large. Given a particular test score along with the mean and standard deviation we can tell not only if a score is good, but also how good it is. This judgement is made possible because of our knowledge of the "normal distribution".
Given a normal distribution we know that half the scores fall below the mean and half the scores fall above the mean (symmetrical). We also know that approximately 34.13 % of the scores are between the mean and one standard deviation above the mean. Given knowledge of the normal curve we can say that a person who obtains a score that is one standard deviation above the mean scored better than approximately 84.13% (50% + 34.13%) of the normative group. Given that all norm referenced test refer to the same (normal) distribution the actual score values become less important. That is, a scores location in the distribution of all scores becomes the essential determiner in the assessment of a scores quality. A scores position in the normal distribution is the "standard" by which all scores are judged.
Since all norm-referenced scores are evaluated in terms of its position along the normal distribution, the scale that the scores are expressed in can be changed or set without restriction. As long as the mean and standard deviation are known the scores will have meaning. We refer to these equivalent scores as "standard scores". The figure below shows a number of standard scores and how they relate to the normal distribution.
As can be seen Wechsler IQ's have a mean of 100 and a standard deviation of 15. T-scores on the other hand have a mean of 50 and a standard deviation of 10. If a person obtained an IQ score of 115 we would know that this person scored higher than approximately 84% of the population. The same statement could be made of a person obtaining a T-score of 60, both scores are one standard deviation above the mean.
Normal Curve Equivalents (NCE) NCE scores are also standard scores with a mean of 50 and a standard deviation of 21.06. An NCE score one standard deviation above the mean or 71 (50 + 21) would be higher than approximately 84% of the population with only 16% of the population scoring higher. Since the normal distribution is symmetrical the reverse is also true, a person scoring one standard deviation below the mean would be scoring higher than 16% of the population with 84% scoring higher. Standard scores like the NCE can easily be converted to another standard scale or score. As long as the mean and standard deviation of a standard score is known it can be converted to a scale with a different mean and standard deviation (linear transformation). This will have no effect on the relative standing of the score or the distribution of scores on the newly formed scale. The formula for converting standard scores is as follows:
The following program will convert scores to a number of standard scores. To convert a score simply enter the score you want converted along with the mean and standard deviation of all scores. Then select the desired standard scale you want the score converted to.
| Score | |
| Mean | |
| SD | |
| Scale? | |
| T-Score | |
| Z-Score | |
| College Board | |
| NCE | |
With criterion referenced testing scores are compared to a particular "cut score" or criterion. Often times the cut score is expressed as a percentile. Scores that meet or exceed for example the 80 percent correct are considered proficient and those scores that fall short are considered not proficient. An example of how a criterion referenced test might be used would be to test for minimal competency. For example students might be required to demonstrate minimal competency in a number of content area before progressing to the next grade, or a more advanced class. A cut off score might be set at the 80 percent for competency and 90 percent for mastery. A test for professional certification might require examinees to demonstrate mastery of a particular content area(s) before being certified. Cut scores or criterion scores are most often selected arbitrarily. On the other hand cut scores need not be arbitrary. If for example a company wanted to select the top ten percent of all examinees then selection of a criterion score equivelent to the 90th percentile would no longer be arbitrary. It should be noted that in this example, the initial selection of a criterion score is based on the normal distribution of scores (norm referenced).
A term often used synonymously with test reliability is "test consistency", referring to a test's ability to repeatedly yield the same score given equivalent levels of a trait or ability. It is possible to draw an analogy between a test and an archer. The ability of the archer to hit the bull's-eye of a target could be thought of as the test construct validity (is the test measuring what it is supposed to be measuring?), while the grouping of the arrows on the target can be thought of as the test reliability or consistency. A "good test" then would be one that repeatedly hit the bull's-eye of the target (valid and reliable). One method of determining a test's reliability is to test the same people on two different occasions and comparing the scores. The correlation between the two test scores is referred to as the test's test-retest reliability. Test-Retest methods of estimating a test's reliability have largely been superceded by statistical methods designed to measure a test's "internal consistency". A simple example of a measure of internal consistency is to correlate the odd items with the even items (split-half reliability). A more sophisticated method (Cronbach's Alpha) is a statistical method for finding the average of all possible split half correlations. Cronbach's Alpha is probably the most common measure of test reliability reported in testing manuals. The idea is that if a test is internally consistent, that is all items measure one trait or ability (unidimensional) then the test will also be reliable.
There are a several types of test validity. When the type is test validity is not specified it is commonly assumed that one is refering to "construct validity". Construct validity refers to the test measuring what it is supposed to measure. When talking about achievement tests we are usually concerned with the test's "content validity". When testing achievement we want the content of the test to match as closely as possible the content of the course or curriculum we are evaluating. There are several additional types of validity that I will not be discussed here since content validity and construct validity are the two main types of validity.
