The number crunchers at the U.S. News & World Report consistently list Stanford among the top five American universities. As a result, our merit as students and individuals is invariably linked to the questionable calculus of these rankings. Some of us take the time to rail against the rankings. We may dive into the formulas and notice that perhaps factors such as alumni contributions are not the best indicators of education quality. We can single out heavily influential factors such the perceived relative prestige of a university as both subjective and problematically self-reinforcing. The New York Times reported that “the quantitative precision of the purely academic ratings, especially at the tippy top, can seem arbitrary — absurd, almost. When Caltech surged from No. 9 to No. 1 in the 2000 edition of U.S. News & World Report’s ‘’best national universities’’ list, it was thanks to a change in the scoring formula ... not to any meaningful improvement.” However, criticize as we may, the fact remains that our lives have been, and will be, partially shaped and influenced by the sometimes arbitrary, sometimes absurd scoring formulas of rating wizards like those at U.S. News & World Report.

Ratings wizards are merchants of prestige. By choosing the elements that factor into their formulas, they exercise profound influence on how we value educational organizations. This becomes dangerous when readers don’t actively seek to understand the concrete factors that determine our fuzzy notions of prestige. Because ratings can also affect the way that these organizations actually act, it is doubly important for individuals to be aware of the metrics by which things are judged. When ratings become powerful, institutions are stripped down to the skeleton of the statisticians’ score card instead of the flesh and nerves that experts in a given field know to be substance of that organization. Knotty and complex questions are reduced to combinations of numerically quantifiable factors.

Building a Better Legal Profession is a national grassroots movement of law students who hope to create market pressures that will stimulate workplace reform in the nation’s largest private law firms. Although the students of Stanford Law School who spearheaded this movement are a force to be reckoned with, in the litigious environment of modern America, anyone who takes on the task of reforming “big law” is taking on Goliath. So what was the weapon of choice for this organization’s struggle to improve diversity, increase pro bono work and restore a sliver of humanity to the lives of entry-level corporate lawyers? The group created a simple system of arranging available online data about firms. They ranked the firms against one another based on factors such as diversity, and then graded each firm with a letter A through F. Copy, paste and rank. It seems like a simple formula, and it is, yet it has won Building a Better Legal Profession press and praise across the nation. Creating a grading system that ranks law firms based on something other than profits is a neat way of capitalizing on America’s passion for rankings while changing traditional paradigms of prestige.

Ratings systems pressure the scrutinized organizations to focus on the factors that appear on the score card. The Bush administration’s No Child Left Behind Act has long been criticized for ranking schools based on standardized testing. The effect of this policy, many educators argued, was that as money was pumped into drilling students for tests, the depth and quality of education suffered. Organizations distribute their resources to improve the factors that they are rated on. This illustrated how ratings levy the prestige that they bestow to effectively influence organizational decision making.

In considering the prestige of universities, schools, corporations and the litany of other things that we like to put in numerical order and slap a grade onto, the ratings consumer should be aware of the assumptions that underlie the information they are consuming. If one doesn’t believe that high alumni giving rates or the size of a university’s endowment correlates with the quality of education at a given university, perhaps one should be cautious about accepting ratings calculated based on such factors. Similarly, if one does not believe that profit margins for partners are the best measures of a private law firm, then perhaps one should think twice about using profit-based metrics to choose a prestigious place of work.

In deciding to take ratings into their own hands, Building a Better Legal Profession has picked a powerful tool for reforming the legal profession. By building a report card for firms based on quality-of-life factors, this group puts quality of life back on the map of factors that should be accounted for when firms are compared against one another. Revamping ratings is a time-consuming project, and not something that we can all accomplish with ease. However, stopping short of implementing better and more relevant rating systems, the best that we can do as consumers is to be wary of the assumptions about quality and worth that underlie every rating system.