Skip to content

October 29, 2011

Let’s Dig Deeper to Understand School Rankings

by Anne Paddock

Ranking schools – high schools, colleges, business and law schools – is big business these days. US News & World Report, Newsweek, Business Week and countless other national and regional magazines and newspapers rank our schools annually.  Schools seem to be obsessed with their respective rankings proudly displaying them on their websites and in press releases. But, if the public dug deeper and understood how the rankings are computed, these publications would not be considered gospel.

Malcolm Gladwell, a writer with The New Yorker magazine wrote an article entitled “The Order of Things: What College Rankings Really Tell Us” in the Feb 14, 2011 magazine issue and every parent of a middle, high, or college age child should consider reading this article:

Gladwell points out the importance of understanding the methodology of how rankings are done and to question the value of these rankings.  He cites US News & World Report’s annual ranking of colleges and universities that is based on seven weighted variables:

  1. Undergraduate Academic Reputation:  22.5%
  2. Graduation and Freshman Retention Rates: 20%
  3. Faculty Resources: 20%
  4. Student Selectivity: 15%
  5. Financial Resources: 10%
  6. Graduation Rate Performance: 7.5%
  7. Alumni Giving: 5%

The importance of a variable is determined by who is compiling the list; in this case US News & World Report. Gladwell points out that “Undergraduate Academic Reputation” – the variable given the most weight – is determined by a survey sent to the country’s presidents, provosts, and admission deans asking them to grade all the schools in their category on a scale of 1-5.  Those at national universities are asked to rank all 261 other national universities. Certainly, these academicians are familiar with many schools but not all or even most schools so the question begs: how can a person know about the academic reputation of 261 universities? The answer: they can’t.  And, when asked to rate their competitors, how did they respond?

Gladwell almost gleefully points out that a cost variable is not included – as if cost is not important – maybe not to US News & World Report but for many people paying up to $50,000 per year per child, cost is a consideration.  Gladwell argues that many parents and students, in fact place considerable importance on this variable.

Parents love lists and nothing validates their decision more than to see their child’s school named to a top ten “best schools” list.  The school is probably a very good school but is it really one of the top ten schools in the country? Maybe yes, maybe no because:
  • The rankings are based on a set of variables that assumes everyone places the same weight or importance on the variables as the compiler and this is simply not the case; many variables also tend to be subjective and not easily measured and even those that can be measured have a relative importance;
  • School districts with more funds or private schools with large endowments are able to provide more resources and often times, a better education (but private high schools are not included on many high school ranking lists);
  • One school cannot be all things to all students and therefore, different schools work for different students;
  • A teacher or a professor can make a difference and no measurement of degrees will ever tell you how inspiring or how great a communicator a teacher is.
My brother turned down Yale because of the school’s location in downtown New Haven, CT and because of the wrestling program – and decided instead to attend William and Mary in Williamsburg, VA because the school had the right balance of location, wrestling coaches, and academics for him.  Most people thought he was crazy for turning down top ranked Yale for William and Mary and yet, I’ve never heard him say he regretted his decision. In short, variables are more applicable to an individual when the importance of the variable is matched to the individual. Large generalizations are just not always applicable when applied to the individual.

School rankings are not an objective measure of the quality of an education. At best, school rankings are a general guide based on the assumptions of the person or group compiling the list. At worst, the list can be misleading. US News & World Report publishes an annual list of “Best High Schools for Math and Science” and uses the following methodology to rank the public schools:
  1. The high school had to be one of their best high schools (each “best” list has a summary detailing the methodology, which should be read);
  2. A math achievement index was computed by determining the percentage of 12th graders who took at least one AP math course during high school and weighing this 25%; and the percentage of 12th graders who took and passed an AP test with a score of 3 or higher during high school – weighted 75%;
  3. A science achievement index was computed by determining the percentage of 12th graders who took at least one AP science course during high school, weighing this 25%; and the percentage of 12th graders who took and passed an AP test with a score of 3 or higher during high school, weighing this variable 75%;
  4. A math and science index was then computed with 50% given to the math achievement index and 50% to the science achievement index.
The obvious problem with the methodology relates to grouping every student that passes the AP exam as the same when in fact, there is a big difference in the knowledge of the material between a student that scored a 3 and a student that scored a 5 on an AP test. Theoretically, a high school that had 50% of its students take an AP course in Math with 90% scoring a 3 would rank the same as a school that also had 50% of its students take an AP course in Math with 90% scoring a 5. But, US News & World Report doesn’t consider this difference.

My daughter will be looking for a college that satisfies her rank of variables with the English and Language Departments of primary importance, followed by the location and the size of the school. And, then there is this variable she calls a “gut feeling” that she gets when she tours a school.  Totally unquantifiable but important, I suspect that gut feeling will play a big part in deciding where she will decide to spend the next four years of her life.

Comments are closed.