It’s a popular meme, and it’s a good one for today.

I had said I was just going to point people who ask me about the latest SAT or ACT news to this blog post. It’s long and cumbersome, but it did sort of summarize all the points I talked about too often. Use the SAT if you like; don’t if you don’t.

I don’t even know why people think of me first when articles like today’s piece in the NY Times comes out, other than for some reason, I’m an opinionated loudmouth in a profession that values peace, harmony, and getting along. I had texts and emails before I got out of bed this morning. Here’s a link to the article, which should be free for some time.

The idea of predicting future human potential and behavior and potential using a standardized test has been around for a while. It’s been a popular idea among the less equity minded members of human society for sometime: Usually people who write the tests write them for people like themselves, and then are surprised when people not like themselves don’t do well on them. The idea of a single, multiple choice test to measure academic achievement sounds great, until you realize that the students being tested come from 38,000 high schools, following a non-standardized curriculum, taught by–how many?, 500,000?–teachers, and maybe then you realize what a ridiculous proposition that is on its face. But it is what we have.

University of Washington professor Jake Vigdor does a great job of taking apart the NYT article, in this thread here.

The Leonhardt article uses the California faculty senate article as a proof point (the one that mentions College Board over 50 times) but fails to cite the rebuttal by the person who has researched the topic more than anyone, and who found serious problems with the research study.

But let’s not argue the statistics; they never seem to persuade. Let me just make some random points, which is probably also likely to fail:

First, ask yourself how well any admissions office can predict college performance at the individual student level. It’s not hard in aggregate, of course (one of the mistakes amateur statisticians make all the time in deciding if two things are related). But at the individual level? When you look at the data you’ll be shocked. Slam dunks who flunk out and big risks who end up being superstars are more common than you might think. On average, all these anomalies even themselves out. 

The reason for the anomalies is simple: Way more inputs go into college success than we could ever hope to measure. And many things colleges never consider (how many hours will a student work during the school year; how much private tutoring can the parents afford; how many hours a week the student plans to play XBox; how far apart–or close together is the romantic interest; how much does the student have to help at home with elder or child care, to name a few) weigh heavily on academic performance in college.

Remember, Tom Brady was drafted 199th, Jerry Rice 16th, and Brock Purdy 262nd by people who do nothing but evaluate potential NFL players. And Ryan Leaf, JaMarcus Russell, and Tony Mandarich are among those who washed out.

Second, if it is really about predicting performance, ask your friendly admissions or IR staff member how the essay, letters of recommendation, supplemental questions, interviews, or demonstrated interest work. These things are all puzzling additions to the selection process. Many institutions used to require a photograph. Why, do you think? The answer is not a nice one. 

The Leonhardt article mentions that these factors may introduce greater bias than the SAT into the process (I wrote about it in 2016), yet curiously, not a single one of the institutions listed in the article is–as far as I know–contemplating abandoning those.

Third, I do not care what the Ivy Leagues (or the Ivy Plus, a moniker that certainly was not vetted with any branding agency, which is strange given that all of them are multi-billion dollar corporations) require for admission. It affects me not one bit. Use shoe size, or cranial measurements, or BMI, or make applicants take a DNA test to screen for the possibility that someone who is admitted might turn out to send bombs in the mail, or murder their parents, or invade a country on fake intelligence, or even try to overthrow the American government.

Fourth: Ask why any admission requirement exists. Hint: It’s for the good of the university. The SAT probably does help the Highly Rejective Colleges (hat tip to Akil Bello) but probably not in the way you think. Yale has memos in their archive from the mid-60s that recommended the SAT as the best way to reduce the expenditure on institutional financial aid, for instance, even while admitting it didn’t do much for predicting GPA.

Fifth, if the SAT does add incremental value to the prediction equation (and it almost always does), ask whether the juice is worth the squeeze. When you account for cash costs, stress, opportunity costs, and weigh that against the incremental value, I’m guessing you’ll say “no.” The first time I took a university test optional, the results were interesting: One-tenth of a GPA after the freshman year difference between test submitters and non-submitters. It was statistically significant, but meaningless practically, as both groups were above 3.0.

Finally, ask Leonhardt what the genesis of this article was. I can’t prove it, and of course no reporter would ever reveal their sources, but I’ve got $100 that says this was planted by the College Board, who saw a few cracks, looked at their floundering finances, and decided to take one swing for the fence. I’m sure it’s just coincidence that they found someone who went to Yale for undergrad, and that he included very little input from test critics (other than to label them as “liberals who try to wish away inconvenient facts,” which, if you don’t mind a little editorializing, is pretty lazy and pretty shitty journalism.) Spend three minutes with this video if you think such activity would be above College Board.

Not only is it not out of the question that they planted the whole package; it’s highly probable, I think. Not that I can prove it, but if you believe a magic test created by someone who’s probably never taught a high school class is predictive, I can believe what I want.

That’s it. I don’t care, but people asked. And the river of bullshit will start flowing again because of this article. I’m glad I’m not in a flood zone.