Over the last couple of days I have found myself pondering the US News & World Reports rankings (which confers a certain status, more on that later). How are these remotely valid? How can you rank heterogenous, dissimilar institutions? Each instructor has their own subjective evaluation of a student, course work is different at each institution, resources available to students are different, student-body diversity/dynamic is different, how can these be aggregated?
It literally is the same problem used to calculate numbers like GDP or trade balance/deficit.
Several years ago Washington Monthly published a study originally commissioned by US News & World Reports which criticizes the nebulous methodology used to create the ranking hierarchy. Here is a snippet:
Quality ratings of institutions are commonly performed in light of institutional goals. Compared with hospitals and graduate programs, undergraduate colleges are more heterogeneous in their goals. Goals may include: liberal arts education; vocational preparation; preprofessional and scholarly preparation; middleclass socialization; and service and leadership development. There are also varying mixes of goals across institutions. In large institutions there is considerable variance within the institutions, for example, among different subunits (“colleges”). The great variance both across and within institutions makes it very difficult to get consensus on quality criteria or on measures for undergraduate programs in general, or even for groups of colleges or universities that might appear similar. compared with hospitals there is also a paucity of publicly available comparable data on postsecondary institutions that are uptodate, this necessitating more reliance on data collected directly from the schools.
Several days ago, The New Yorker published an article which details “the social logic of Ivy League admissions.” An associate of mine pointed to a particular passage as evidence for his brand-equity hypothesis, which directly coorelates with status — that institutions such as Harvard operate using a Balanced Scorecard method to maximize the following:
Financial Perspective: Grow the endowment through alumni donations and grow the intangible asset known as the school brand
Customer Perspective: Select students with a high probability of being financially successful and who will contribute to brand equity
Operational Perspective: Offer a decent education but differentiate primarily on branding, admissions and fund raising
Learning Perspective: Invest in faculty and staff that contribute to brand equity over instruction
This answer a question Patri Friedman posed several months ago, What is the purpose of a university?
In addition, it also ties into a point that Arnold Kling brought up several days ago:
Colleges and universities use their ability to deny credit for work at other institutions in order to stifle competition and to maintain their own status. Instead of testing a student to see whether a subject has been learned satisfactorily, institutions simply refuse to accept credits from “inferior” sources of education. This status-oriented approach to accreditation pits traditional colleges and universities against the newer “for-profit” model of education, as described in a front-page story in the Wall Street Journal on September 30.
Note: this is just the tip of the iceberg, stay tuned for more.
(Speaking of brand equity, here is an amusing story from The Onion: Tiger Woods Signs $15 Million Deal To Endorse Alex Rodriguez)
{ 21 comments }
My school was ranked #2 party school in the nation by Playboy Magazine. We thought they must have been on crack, but hey, I figure Playboy knows more about what they’re ranking when focusing on that one segment, than USN&WR does when ranking the whole kit and kaboodle.
The biggest farce is the fact that teaching assistants are not taking into account when ranking. Add to that the fact that the quality of an education is priced as an input rather than as an output. That is to say we evaluate students in terms of SAT as they come in, charging them accordingly, and then do no evaluation at the end, but still graduate them with honors and encomium. Any ranking is questionable at best and probably intolerable.
Here is an enlighting account of the sociology of Ivy admissions from Malcolm Gladwell in the New Yorker:
http://www.newyorker.com/critics/atlarge/articles/051010crat_atlarge
The only major difference between universities in America, other than cost and the weather, is the contacts you make. The top schools might give you are wider range of connections you could use in your life as opposed to other schools.
Other than that, they are just about the same, overpriced and overrated.
As Gary North has suggested, a web based undergraduate program might be a better bet.
Should we be surprised that people think the US News rankings have value when they actually think the Nobel Prizes are meaningful? How about Academy Awards?
It suggests to me that there is something in human nature that makes some of us tune into these endorsements even if the decision makers or the criteria are unknown to us.
I would say that it is a great deal easier to rank colleges than to calculate “macroeconomic” figures. From my experience the rankings tend to be quite accurate and they serve their purpose quite well.
If private competitive institutions are unable to provide the market with useful information regarding products, who can?
If any of you wants to teach a bunch of idealist kids the virtues of the gold standard, please go here:
http://forums.jolt.co.uk/showthread.php?t=448545
An online undergraduate program would be nearly pointless. College is all about becoming socialized. Staying at home with mommy and daddy would defeat the purpose.
An online program might have some benefits, like more attention to academics. But that’s a double-edged sword: a lot of classes that could be taken online (i.e. no labs) are either useless or dangerous.
Ammonium,
The “socialization” argument is often used against homeschooling as well. But if this goal of socialization is really what college is about, why put up with the pretenses and the $40K per year tuition.
It suggest that we could have some very efficient new market entrants. Carnival cruise lines could dedicate a few ships to 4 years of around the world debauchery. Similarly, the city of Cancun could just extend their highly effective spring break socialization curriculum year round.
For those who actually are interested in education through distance learning at all levels here are some really excellent resources we have used:
http://epgy.stanford.edu
http://cty.jhu.edu
http://www.thinkwell.com
http://www.eimacs.com
http://www.teach12.com
If Ammonium is right, perhaps we should be paying more attention to the Playboy Party School rankings than US News
I did an article on the U.S. News Rankings years ago for my college paper. The thing that surprised me was that the bulk of the rating (at that time, maybe it’s changed) was based on “academic reputation,” which was nothing more than a survey of how schools rated their competitors.
Sure, “only officially centralized institutions with carefully screened experts can teach undergraduate programs to students.” That’s classic statist philosophy. Its peculiar why anyone considering the idea of web based college, would automatically associate the idea with the sole thought of a young student who never leaves his or her’s parent’s house?
Think of the all too familiar grad student lecturing biology to a hundred students with their outrageously expensive books, and costly auditorium. Now, think of a lifelong teacher and scholar who would compose reading lists, visual displays and recordings etc. at a fraction of the cost. I am not suggesting that web education is going to completely replace traditional college. However it will play a larger role, because the costs to go to a traditional college are rising every year, at the same time the value of the undergraduate degree from a top 50 college is falling because of a general over supply of undergraduate degrees.
One can point out flaws in the rankings, but the fact is, they reflect above all (and perpetuate) reputation. And when it comes to the opportunities that a degree will open up for you, it is the school’s reputation that counts. Sure, you might get a better education elsewhere–but you should be taking responsibility for your own education in any event (e.g., by reading sites like this one).
That’s why my advice to law students is to go to the best US News-ranked school you can, period. (With a few limited exceptions not necessary to discuss here.) You can go to a bottom-ranked school and learn everything you would need to know to open your own law firm, or you can go to Yale and learn none of that, but have many, many more opportunities in life thereafter. Except in limited circumstances, you’d be crazy not to do the latter, if you can.
J.H. Huebert,
It is difficult to argue with that advice. If the subjective value of a degree from a particular institution has value to employers then it is worth considering.
This seems to be in-line with the New Yorker article which suggested to me that it is not ultimately the “education” that employers value but the screening function of the admissions process.
Strange that you can’t just use your acceptance letters in your resume or CV. You are forced to pay an ungodly sum and spend 4 years to comply with the pretense that you are getting a better education.
I’m just starting to read Randall Collins’ “The Sociology of Philosophies.” His finding that dominant philosophers emerge from networks of prior generations’ dominant philosophers tends to support J.H. Huebert contention that one would be crazy to pass up Yale Law School if he could get in. The stimulus from personal interaction with eminent instructors and with other students who succeeded in getting into Yale Law School increases the likelihood that its graduates will become eminent lawyers themselves.
I’m suspect of these rankings. I think a lot of them have to do with the prestige of the faculty rather than the actual quality of instruction.
I’m a math-computer science major at UCSD (University of California, San Diego) and my first quarter here I found out that prestigious faculty has nothing to do with quality of instruction. In fact, in a lot of cases quite the opposite is true.
I was a transfer student and so I already had quite a few classes completed by the time I got here. My first quarter I took an upper division math class, and I can honestly say that the woman teaching it was the worst math teacher I have ever had. After spending all of my academic career in public institutions, that is saying a lot. This woman would lecture on topics way beyond what we were supposed to be learning (i.e. on the proof of Fermat’s Last Theorem), and her exams had almost nothing to do with the exercises she assigned from the textbook. A lot of people stopped going to lecture, and I personally dropped the class.
This woman happens to be a researcher here at UCSD, and a rather well known one at that. But this did not help one iota when it came to the actual quality of her instruction.
I ended up taking this class over the summer with a prof who is a ‘lecturer.’ A lecturer is someone who is ‘below’ that of assistant professor. But his lectures were actually on topic, much easier to follow and comprehensive. Additionally, his exams were actually sane. I learned far more from his class than I did from the researcher woman. In the world of academia the ‘lecturer’ prof I had over the summer probably adds little to the prestige of the faculty and the university. But it is him and others like him who actually produce quality instruction.
Another example is my prof for one of my programming classes. He too is a ‘lecturer,’ but among undergrads he is one of the most well known and liked profs on campus in the computer science department.
DJC, as someone who wants to homeschool I get a lot of the same “socialization” crap. It is, in fact, the only reason left and I hear it from *everybody* who doesn’t already support home schooling.
However, the entire “socialization” argument is fatally flawed, because kids “socialize” just fine at a playground. The only thing different that they learn in school is to *obey*.
Older kids “socialize” just fine in a part-time job, or gymnastics class at the “Y”, or any of the million other things that so-called “kids” do day in and day out.
The implicit message of the “socialization” meme is that homeschoolers (and distance-learning students too) are cloistered or imprisoned, separated from their “peers” and the community.
Anyone who has interacted with homeschoolers knows this is a blatant lie. While “normal” students are imprisoned 180+ days a year, surrounded only by others their own age regardless of achievement, in an artificial environment of coercion, homeschoolers are spending maybe 3 hours a day on exactly the same scholastic curriculum and then going out into the community to deal with other people instead of staying locked inside because someone ordered them to do so.
That is why, without exception, every homeschooler I have met has been far better “socialized” than their so-called “peers”. They do not bully, they do not cower. They deal with “adults” and younger “children” as people. They are individuals rather than following fads “just because everyone else is doing it”.
Tell the 14 year old business owner I did business with that they’re not “socialized”, that she’d be better off stuck in 8th grade trying to tell the guidance councillor what “career” she wants when she grows up.
Steven Kane: “I think a lot of [the rankings] have to do with the prestige of the faculty rather than the actual quality of instruction.”
I think this assessment is right on target. [Of course, there are exceptions.] The academic regime wants research and publications, so there’s no glamour in superior instruction. It’s the research that gets the acclaim, the prestige, and the government grants. Reputation first, customers second.
‘Quality of teaching’ is definitely a missing element in the USNWR survey. It doesn’t really have any measure for ‘student satisfaction’–retention and graduation rates as proxies are a bit crude. I’m not sure how you could measure this efficiently.
Curt,
You are absolutely correct, we have been homeschooling for several years and our own boys and other homeschoolers we have met are evidence to support your argument.
It is the idea of being sequestered in a room with just an authority figure and age peers that is strange when you think about it.
Gifted children, who are particularly ill-served by institution schools, have been bailing out for years with great results. I am hoping that the new Mises home study course proves to be applicable to this unique group.
I would highly recommend Rothbard’s book “Education, Free and Compulsory” if you haven’t already read it.
“The only major difference between universities in America, other than cost and the weather, is the contacts you make. The top schools might give you are wider range of connections you could use in your life as opposed to other schools.
Other than that, they are just about the same, overpriced and overrated.”
Connections are just as overrated.
DJC,
I was one of those gifted children trapped within a gub’ment school. Never would my parents have dreamed of homeschooling me, because that would have entailed taking an active interest in their son’s life.
Back then I “bailed out” the only way I could rationalize: by refusing to participate in class any more than the bare minimum needed to pass (and sometimes not even that much).
Lucky for me I discovered–gradually–that all humans are born to be artists. In this make-a-buck world too many of us try to deceive ourselves otherwise–another toxic by-product of the shameful school system.
“I think this assessment is right on target. [Of course, there are exceptions.] The academic regime wants research and publications, so there’s no glamour in superior instruction. It’s the research that gets the acclaim, the prestige, and the government grants. Reputation first, customers second.”
I think that research and publications could be great, IF you are a grad student. As an undergrad, still building the foundations needed before one can do research, the fact that my professors are prestigious reasearchers means little to me.
The problem is that the administration forces these researching professors to teach undergraduate classes that they don’t want to teach. My prof for that math class was a great example. At least two times she went off to some conference and left her T.A.s to do the lecture. It was apparent that she had very little interest in us the students or if we were actually learning anything (or even able to learn anything from her).
If these researching professors are going to teach at all, they shoud stay in the graduate departments. At least then they will be able to do what they want to do with their research papers, and not be wasting the time and resources of undergraduates.
Hello guys, I urgently need info on MLA format… I can’t death my essay. Does anyone know? Please help…
Comments on this entry are closed.