There’s a new study out, and it proclaims that Massachusetts is the smartest state, while New Mexico is the dumbest. (Cow Hampshire ranks 12, by the way, just making the first quartile.) That seemed a bit odd to me, so I thought I’d take a closer look at the study.
The Morgan Quitno Press took their numbers from public schools all across the nation and ran them through their magic formula to come up with their list. I’m no statistician, so I’m not going to even bother looking at that. But I will take a look at the 21 factors that they consider relevant in “smart” rankings.
The first three they mention are purely economic: Public Elementary and Secondary School Revenue per $1,000 Personal Income,Per Pupil Public Elementary and Secondary School Current Expenditures, and Percent of Public Elementary and Secondary School Current Expenditures used for Instruction.
In other words, how much of people’s money is taken by the government to spend on schools, how much per student that works out to, and how much of that actually gets spent on the kids. Because everyone knows that the more you spend on education, the better an education the kids get. That’s why Washington, DC students are consistently at the top of the rankings, and our kids here in New Hampshire do so abysmally in the SATs.
Whoops, no they don’t.
In fact, it seems to be an inverse relationship between money spent on education and actual results. But let’s not let uncomfortable facts get in the way of ideology, shall we?
Here’s another factor they took into account: #13, Average Teacher Salary as a Percent of Average Annual Pay of All Workers. I guess that means… oh, hell, just read above. The same reasoning applies.
And here’s #14, Percent of School-Age Population in Public Schools. After all, what’s the difference between dropouts, private school students, and home-schooled students? They all do SO much worse than public school students. That’s why we keep seeing the public school students winning spelling and geography bees, scoring perfect on the SATs, and… no, never mind.
Hey, let’s do some double-dipping! Let’s count the dropouts AGAIN, at #15: High School Drop Out Rate.
(Update: somehow this paragraph got dropped.) So, that’s 6 questions of 21 that really have very little to do with student performance. Without even trying, I’ve put a significant dent in nearly 30% of their source data. It makes me wonder what someone who actually knows what they’re doing could do with the rest of it.
Again, I’m no statistician, but I think I can see what guarantees high scores on this test. All a state has to do is spend big bushels of money on public schools. It’ll help if they also keep down private schools, and make it harder and harder to home-school. Just follow their magic formula, and pretty soon every child will be above average.
So did the teachers union help fund this study?
Honestly if we want to figure out the smartest students, why not just crunch the testing numbers in addition to how many kids go to college or technical schools and how many of those kids complete their post secondary educations.
Wow, they really do count some of the stuff twice.
We’ve got a few questions about fourth grade proficiency and then are the fourth grade parents strict about the homework. So that’s probably a double too.
They ask about the student-teacher ratio a few different ways too.
Only 6 of these are about student results (other than the graduation questions).
I personally don’t think of these results as a “smartest state”.
If it’s high pay / low student ration it might be “a good state to work in” report.
They don’t give much on their ratings (is it one point for each)…
http://www.morganquitno.com/edrank04.htm#METHODOLOGY
Remember even if all our kids are geniuses someone is going to be number one and someone is going to be number fifty…
These ACT results show how close some of the states really are for the kids that take the ACT. What’s interesting is that you can see what percentage of the kids take the ACT. Michigan has a slightly lower score than Massachusetts BUT 68% of MI takes the test but only 12% of MA does (on average 40% of each state takes the ACT). Oregon, Washington and Vermont all scored higher on the ACT and had exactly the same percentage of kids take the test (I like comparing grapes to grapes).
http://www.act.org/news/data/04/states.html
FYI Colorado has 100% of their students take it and they are only a little lower than the country average.
This is little more than advocacy masquerading as scientific analysis. If they were really concerned about which state is the “smartest”, they would look at outcome measures of performance alone. It is both interesting and quite revealing that the only measure of high school performance is graduation rate, and don’t include other measures of performance regarding high schools like exit exam performance, National Assessment of Educational Progress (which the US Dept of Ed characterizes as “the only nationally representative and continuing assessment of what America’s students know and can do in various subject areas”), Advanced Placment participation and test results, college entrance exam results, amount of scholarship money students receive, etc. The list goes on and on of things that could have legitimately been included but were not.
Instead they have chosen a lot of input measures, then double or triple counted them as they see fit. They measure how much money is spend on instruction, when the single biggest component of that in any school system is teacher salaries, which they then measure again. On the negative side they measure pupil-teacher ratio and then measure average class size. Gee, do you think those might be related?
While I have some background in statistics, if you really want someone to give this a going over you should contact Eric Hanushek at the Hoover Institute, or Caroline Hoxby at Harvard.
This is little more than advocacy masquerading as scientific analysis. If they were really concerned about which state is the “smartest”, they would look at outcome measures of performance alone. It is both interesting and quite revealing that the only measure of high school performance is graduation rate, and don’t include other measures of performance regarding high schools like exit exam performance, National Assessment of Educational Progress (which the US Dept of Ed characterizes as “the only nationally representative and continuing assessment of what America’s students know and can do in various subject areas”), Advanced Placment participation and test results, college entrance exam results, amount of scholarship money students receive, etc. The list goes on and on of things that could have legitimately been included but were not.
Instead they have chosen a lot of input measures, then double or triple counted them as they see fit. They measure how much money is spend on instruction, when the single biggest component of that in any school system is teacher salaries, which they then measure again. On the negative side they measure pupil-teacher ratio and then measure average class size. Gee, do you think those might be related?
While I have some background in statistics, if you really want someone to give this a going over you should contact Eric Hanushek at the Hoover Institute, or Caroline Hoxby at Harvard.
How many of the top ten voted blue?
Could there be a hidden motive for the story…
Did the person who assembled the study go to a public school?
What’s interesting is that you can see what percentage of the kids take the ACT. Michigan has a slightly lower score than Massachusetts BUT 68% of MI takes the test but only 12% of MA does.
That’s likely due, at least in part, to the perception that the ACT is the Western US test. The SAT dominates in the Northeast.
What would you get if you actually compared actual test results of actual students from actual schools?
Why, you’d have the Accountability part of the No Child Left Behind Act. Which act unfairly expects teachers to be held to standards and actually teach, so the Teacher’s Unions can’t have that. So they give us alternative studies like this.
The only wonder is that they actually had a ranking. Doesn’t that hurt the self-esteem of the students from low-ranking states? What if they held a Battle of the College Academics and nobody came?
Mass need to spend some of that education money on swimming lessons, I recommend the Ted Kennedy Aquatic Act of 2005…
At $10k per student, I’m not surprised Wisconsin is high on the list for something. The goofy reiteration of certain points like number of dropouts and teacher pay would lead rational persons to question the author of the report’s motives, IQ, or both. The only real value seems to be for teachers shopping for the good jobs, as Gary suggested, or perhaps to attempt to justify the bigger and bigger bucks being spent.
Jay Tea – Didn’t you blog about an identical study once before?
Massachusetts children are brought up eating lots of fish and chowder so it makes their brains less clogged!…