steve's alternative recursion

the Quantitative GRE has a Quantitative failure:

The quantitative exam lumps a large portion of test takers with a significant variation in ability into a single score.

The verbal does not have a similar flaw.

the results from everyone who is trying to get into a graduate program:

clearly there is a problem here where the most common score of all test-takers is the maximum.

let’s take a closer look at how this makes the GRE Quantitative even less useful for specific graduate programs:

quantitative:	verbal:
notice in the students pursuing a PhD in math, computer science or electrical engineering, the quantitative exam is too easy, too many students are binned together at the highest score simply cause the test is too easy for them.	interestingly the above fields have 2 humps in verbal, i imagine the first hump is from non-native English speakers going into these fields of study, the 2nd hump is likely the native English speakers.
With philosophy PhD students here we see a more desirable distribution, but still, the perfect is almost as frequent as the mode.	A relatively normal distribution, only 1 hump and the perfect is far from being the mode.
in the 4 areas of study above, the quantitative GRE exam does a good job at differentiating the the people.  The distributions are very normal, with the max and min score occurring far less than the mode.	Even in the fields of study which attract the most verbally talented people, the verbal GRE test doesn’t break down, it still yields relatively normal distributions where the mode is far more common than the max or min.

lets do a real basic analysis of what’s going on:

 

the actual distribution of quantitative ability probably looks something like this:

 

note: normal curves have no maximum or minimum value of ability, just like there is no maximum IQ value, its just insanely unlikely to have an IQ of , say , 500… but still possible.

With practical limitations like a finite time limit, I can understand the need for bounds on the scores.

When you set a maximum, everyone with an ability higher than the maximum shall now be bundled at that maximum value, so lets look at what that will look like with a max=800 and min=200:

 

here you see unnaturally high probability to achieve a 200 and an 800,  the combined probability to score higher than the maximum will be lumped into that one score.  This cannot be avoided, but the number of people who would naturally score higher than the max can be minimized as to not be too significant, like the verbal GRE does.

So Why is the Quantitative so much worse than the Verbal GRE??

well the Verbal GRE has a mean of 477, almost equidistant to the max and min, while the quantitative has a literal (but misleading) mean of 579, so the entire distribution is nudged closer to the maximum boundary,  its also a bit wider than the verbal (higher std dev)

 

see the massive data point at an 800 score?  its value is equal to the accumulated area under entire tail end of the curve (above 800).

which of course looks very similar to the actual GRE Quantitative distribution:

so this is a fundamental issue with the test and should be corrected.

It can be fixed by lowering the mean to ~500 by making all questions more difficult across the board.

 

for those interested, the plots were done in maple of testdist2, where:

 

and

testdist2 := (min, max, x, mean, stdv) -> piecewise(x = min, int(distr(y, mean, stdv), y = -infinity .. min), min < x and x < max, int(distr(y, mean, stdv), y = 10*floor((1/10)*x) .. 10*floor((1/10)*x)+10), x = max, int(distr(y, mean, stdv), y = max .. infinity), 0) ;

 

using a plot command like this:

mean1 := 579; stdv1 := 130; gremax := 800; gremin := 200; plot(testdist2(gremin, gremax, x, mean1, stdv1), x = gremin .. gremax, numpoints = 100, style = point, symbol = cross, symbolsize = 8, labels = [ability, `~probability`], title = "Normal Curve with mean of 579, std deviation of 130, score increment = 10")

This entry was posted on Monday, May 11th, 2009 at 17:14 and is filed under Uncategorized. Find similar posts by selecting any of the following tags: education, gre. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.