For example, imagine a vocabulary test of 100 items designed for fifth graders. When we give it to a representative sample of these fifth graders, it will yield a more or less normal distribution of scores, with a mean of 50. Why does it do that? Because the test developers designed it that way.

Now give it to a sample of fourth graders. The mean will be lower, say 40, and the distribution will be skewed to the left, but not too far from normal. If you give it to a sample of sixth graders, the mean will be higher, say 60, and the distribution will be skewed a somewhat to the right.

With these data in hand, when an individual student makes a score of 40 we say that she/he scored at the fourth-grade level, because that was the average score for fourth graders. Next year, if she/he makes a score of 60 on the fifth-grade test, we say that she/he scored at the sixth-grade level. Why doesn't it make sense to say she/he gained two years?

As long as we just consider grades four, five and six it might almost make sense. But what happens if we give this fifth-grade vocabulary test to a representative sample of first graders? The mean will be very, very low, because nobody will know any of the answers, and the distribution will be sharply skewed to the left, with most of the scores packed at the bottom end and just a few spreading out to the right.

What about second graders? Much of the vocabulary on a fifth grade test may also be out of their reach, so their scores may be very low, too, and the distribution will still be seriously skewed. We are beginning to see the problem: to get from first-grade level to second-grade level a student may not have to get very many more (fifth-grade) items correct, but to get from fourth-grade level to fifth-grade level, a student has to answer many more. The same thing happens at the other end. Both eleventh and twelfth graders probably can answer most of the items on a fifth grade test, so the difference between their average scores may only be a few points.

In other words, grade levels don't form an equal interval scale. If a student goes from fourth-grade level to sixth-grade level, all you can say is that the student was performing like the average fourth grader on the fifth-grade test, and now she/he is performing like the average sixth grader on the fifth-grade test. You cannot say how much she/he gained. And you certainly cannot say that a student who gained two years, by going from grade one to grade three, on a fifth-grade test, gained more than a student who gained one year, by going from grade five to grade six. To drive the point home, think about a student who scores at the tenth-grade level on a third grade test. Do you have any idea whether she/he would be able to answer the items on the tenth-grade test? Test authors are well aware of these problems, and they give appropriate warnings in the test manuals, but the warnings are easy to overlook.