Sorry, Brian--you can't measure student "quality" by using the LSAT

Paul Caron has linked us to Leiter's Top 40 Schools by Student Quality. (Thanks, Paul!)

I like Brian's stuff, and often I agree with him, but on this issue, I have to say that he's just plain wrong.

He's right that it's difficult to measure student quality by UGPAs, because (1) grade inflation and (2) difficulty of curriculum are hard things to factor into the mix. We could probably eyeball some UGPAs from some schools (e.g., Chemical Engineering at Rice University) and get some meaningful information from that, but we certainly couldn't compare every UGPA at every undergraduate institution to measure a student's "smarts." (Not to mention that some of these students may have responsibilities that affect their ability to do the traditional "just take classes and study and graduate in four years" path to their degrees.)

But he's wrong when he jumps to the use of LSAT scores as a measure of "smarts." LSAT scores are useful because they're a way of comparing people across institutions and majors. LSAT scores do a good job of predicting first-year grades, but even the LSAC says that LSAT scores aren't perfect predictors of first-year grades. They're just the best predictors we've found so far.

If you do a multiple regression analysis to try to predict first-year grades, you'll likely find that the combination of [(multiplier A) x UGPA and (multiplier B) x LSAT] can give you some of what goes into first-year grades (and by "some," I mean maybe 30-40%). The rest is hard to quantify, but it must certainly include things like studying the material.

I cannot imagine describing someone's "quality" using just the numbers. Quality includes a lot of nonquantifiable characteristics: maturity, honesty, dedication, empathy, etc. I'd like to think that evidence of leadership ability, ability to juggle multiple responsibilities, and willingness to think of others' needs would all factor into any judgment about "quality."

Brian, if you called your rankings "Top 40 Schools by LSAT scores and UGPAs," without any cognitive leap from those numbers to an overall judgment of "quality," I wouldn't say a peep.