The Best Of The Best Of The Best: Understanding Surveys, Rankings, And Lists

Surveys, rankings, and lists can be very helpful. They give you a broad sense of how employers compare to each other. But they also have limitations, and they should not be the only criteria you use judge or choose employers. Here are variables to consider as you review the various rankings of law firms and other employers.

1. They’re Only As Good As The Information They Collect. If the underlying data is questionable or incomplete or subject to interpretation, then so are the rankings that are complied from that data. An employer that doesn’t provide all the information required by the surveyors might be ranked lower than a “better” employer that does.

2. The Difference Between Number One And Number Ten Might Be Miniscule. Again, check the underlying data. In the Olympics, the difference between the gold medal and the silver might be just a few hundreths of a point. Fourth place might be just one point difference. The same thing is true in these types of rankings. The difference between Number One and Number Ten (or even Number Twenty) might be a lot less than their numerical ranks suggest.

3. “Best” Can Be Relative. What’s most important to the people behind the rankings might not be most important to you. Check out the methodologies behind the numbers to be sure that the ranking system reflects your values and interests, rather than heavily weighs factors you don’t care about while ignoring factors you do care about.

4. Reasonable People Can Differ. Even when people agree on the factors that most important, they may still disagree as to how those specific factors apply to specific situations or employers. Some factors are objective and easily measurable, like number of tax attorneys or number of equity partners who are women. Some factors are not, like LGBT-friendly. An employer that one person finds oppressive might feel liberating to another. So if an employer that interests you does not rank well on these lists, don’t just give up on that employer. Instead, take it as an invitation to investigate further.

5. Consistency Is Key. Despite differences in methodologies, values, and other factors, certain trends do emerge. Those trends can be important. For example, an employer that appears on several different “top lists” over a period of years might indeed be “better” than an employer that appears on only one list. Remember too that ranking agencies might periodically change their criteria—with the result that the rankings change also (which takes us back to the first point in this article).