Linkedin’s Terrible, Horrible, No Good, Very Bad Rankings

 Just what the world needs…yet another business school ranking.

On October 1st, Linkedin published its first ranking of undergraduate programs. Based on Career Outcomes, the ranking leveraged data from over 300 million Linkedin members to identify the schools that place members in the most desirable jobs at the most desirable companies.

Not surprisingly, top business programs like the University of Pennsylvania and the University of Notre Dame scored high across the board. But you’ll also find plenty of discrepancies. For example, why is Villanova the #1 school for accounting when it doesn’t even rank among U.S. News’ top 30 schools in the field? Why is Harvard ranked in the top 10 for finance, investment banking, and marketing when it doesn’t even field an undergraduate business program? And how did the University of Phoenix – of all schools – rank #11 in marketing?

THE PROBLEMS START WITH THE METHODOLOGY

Those are among the questions that undermine a promising initiative from Linkedin. Their methodology is clever enough. And it starts with a logical three-step process. First, Linkedin identifies the “most desirable” companies in particular fields. In this case, Linkedin examined eight fields: accounting, finance, investment banking, marketing, design, media, software development, and startup software development. An employer’s desirability is based on two factors: The ability to attract and retain talent.

Here is Linkedin’s illustration of this approach: “Imagine there are two companies, A and B. If more finance professionals are choosing to leave company A to work at company B, the data indicates that getting a finance job at B is more desirable. This is based on the hypothesis that when a professional moves from one company to another, she gives the company she moves to a strong vote of confidence.”

From there, Linkedin identifies professionals on its network who work in that field. And Linkedin is careful only to include graduates – Bachelor’s degree holders, actually – who earned degrees within the past eight years to ensure past performance don’t weigh too heavily on the rankings.

Finally, Linkedin calculates the percentage of school graduates who work in these so-called “desirable jobs.” These percentages are the basis of the rankings.

METHODOLOGY UNDERVALUES GRADUATE SCHOOL

Follow so far? On its face, Linkedin’s process makes perfect sense. Poets & Quants has even relied on Linkedin profiles to better quantify which investment banks hire from which MBA programs. And that’s the first fatal flaw of Linkedin’s rankings. While they focus on undergraduate degrees, they don’t cross-reference that data against graduate degrees.

Here’s why this is important. Let’s say Student A collected an undergraduate degree at the University of Maryland, a prestigious program in its own right. Six years later, she earns an MBA from Harvard Business School. Afterwards, Student A lands a job at Goldman Sachs and quickly rises to senior analyst. Here’s the question: Which school is more responsible for this success: The undergraduate or graduate program? In Linkedin’s methodology, the University of Maryland would reap the reward – and rightfully so, to an extent. But would Student A have landed that “desirable” job if she wasn’t armed with a Harvard MBA? That’s a scenario that these rankings blithely ignore.

And there’s a big discrepancy between the undergraduate and graduate ranks. Let’s return to the investment banking example. At the MBA level, schools like Wharton, NYU, and Columbia dominate hiring at institutions like Morgan Stanley, J.P. Morgan, and Credit Suisse. At the undergrad level, Georgetown tops the list in this niche, with Yale, Duke, and Princeton ranking third, fourth, and sixth respectively (Wharton and Columbia rank second and fourth respectively). That’s a big diffence.

While Linkedin’s rudimentary methodology measures turnover and retention in gauging employer desirability, it doesn’t weigh other tangibles like salaries and capitalization or intangibles like prestige. For example, a company like Goldman Sachs may be penalized for higher turnover when, in fact, some employees use the brand name and experience to springboard into other companies at higher levels. The ranking also doesn’t add points for specific positions. For example, is being an associate just as desirable as being an analyst in those first eight years that Linkedin measures?

Maybe Linkedin does factor in these variables, but its bare bones explanation of its methodology sure doesn’t include it. What’s more, Linkedin doesn’t share the underlying data so readers to make up their own minds.