The debates happen every year around National Signing Day, as the inevitable backlash against recruiting rankings begins anew. It is not a revelation that evaluating recruits is an inexact science, but just because an overlooked recruit like Marcus Mariota or Johnny Manziel wins the Heisman Trophy does not mean that recruiting rankings are worthless.
They make mistakes, because of course they do. Nobody can ever perfectly evaluate 17- and 18-year-old prospects, and there are inherent biases toward players who do commit to the top programs, as offers from big-time schools are often taken as proof that a player deserves a high ranking.
Generally, though, recruiting rankings are a good guide to the best prospects and which teams landed the most talent. As SB Nation wrote last year, higher percentages of blue-chip prospects make it to the NFL. Of course there are exceptions, but recruiting rankings provide a decent snapshot of how much talent a given program brings in. The key word in that sentence is "talent." They are valuable only if you recognize them for what they are, which is not specific evaluations of how players fit programs. They measure accumulated talent and potential.
A coach can be a great recruiter but a poor developer of talent. Another coach can be a poor recruiter but a great developer of talent. Another coach can be great at identifying system fits and diamonds in the rough, and then developing them. This is where the gray area comes in when we look at production on the field. You can blame the recruiting services for improper evaluations of classes, or you can blame the coaches who don't develop talent.
So, let's look back at the last five years to find the best recruiting teams (including only the 120 FBS teams that have played at the FBS level since 2010), using 247Sports' composite rankings, and season-ending evaluations using Football Outsiders' F/+ rankings, which combine the S&P+ and FEI ratings to form a single objective, opponent-adjusted way of evaluating how good teams really are.
Top Five-Year Recruiting Teams
To start, let's look at the teams that have recruited the best talent over the last five years, using 247Sports' composite rankings from 2010-14. The teams are ranked simply by averaging the team ranking of those five years. Then, we do the same using Football Outsiders' F/+ ratings. Again, the number here is an average of the F/+ ranking from 2010-14 for each of those top-25 recruiting teams.
By no means is this an exact exercise, especially because we're using the same range of years. Teams in 2010, for example, used many recruits from the classes of 2006-10, and most recruits from last year's class have yet to make an impact, so there's no direct cause and effect at work here. Still, it's a snapshot of how teams have recruited over the last five years, and how well they've performed over that same time period.
To the surprise of nobody, Alabama has recruited the best over the last five years and it has played the best over the last five years too, matching its incredibly high expectations from recruiting on the field. Florida State, similarly, has effectively matched its recruiting output on the field. These teams recruit the best talent, and not surprisingly they have been the best teams on the field too. Ohio State is joining that conversation, just with 2011's 6-7 post-Jim Tressel/pre-Urban Meyer season skewing the data.
Ten of the top 25 recruiting teams over the last five years are from the SEC, with five from the Pac-12, four from the ACC, three from the Big Ten and two from the Big 12, plus Notre Dame.
So who has outperformed their recruiting rankings the most?
Power Five Overachievers
Many of the names on this list aren't surprising. Kansas State's Bill Snyder, TCU's Gary Patterson and Michigan State's Mark Dantonio are widely known as some of the sport's best coaches. You could say that recruiting services get these teams' classes wrong, or you could perhaps more accurately say that these are the teams that are the best at correctly identifying sleepers and system fits and also most effectively developing those players. So while Kansas State has finished an average of 61.8 in recruiting the last five years, its on-field performance, according to F/+, places it at 28.4 -- including ninth with a Big 12 championship in 2012.
Only Vanderbilt ranks lower than Missouri in the SEC in recruiting over the last five years, yet Missouri has won back-to-back SEC East titles, making a case for Gary Pinkel as one of the more underappreciated coaches in the country. At Georgia Tech, Paul Johnson has recruiting disadvantages because some top talent won't fit into the team's option scheme, but the Yellow Jackets just finished the 2014 season ranked sixth in F/+.
The above list includes only schools from the Power Five conferences. The actual biggest overachiever in the entire FBS? Utah State, which under Gary Andersen and Matt Wells has finished with an average F/+ ranking of 54.6 despite owning an average recruiting ranking of 108.4. Next on the list are Navy (+51.8), Northern Illinois (+50.8), Boise State (+50) and Air Force (+34.6).
Power Five Underachievers
Charlie Weis, self-proclaimed owner of a "decided schematic advantage," continued to prove at Kansas that he does not effectively develop the talent he recruits, just as Notre Dame didn't either (Turner Gill didn't do any better for the Jayhawks). It's not that Kansas has been bringing in great recruiting classes by any means, ranking an average of 56th over the last five years, but it has made things worse by performing as one of the worst teams in all of FBS, with an average F/+ranking of 108th.
Derek Dooley struggled to develop talent at Tennessee, something that Butch Jones is attempting to change. Texas, not surprisingly, has underachieved greatly compared to the talent it has signed as the Mack Brown era fizzled. Auburn's numbers, meanwhile, still suffer from the post-championship meltdown under Gene Chizik, so don't take this as a reflection on Gus Malzahn, who's one of the best coaches in the country. Of these teams, only Virginia has had the same coach for all five years, with Mike London starting in 2010.
The bottom of the list is mostly made up of Power Five teams, because they generally rank in the top half in recruiting, but the worst differential among Group of Five teams is New Mexico, which has an average recruiting rank of 89.8 and an average F/+ finish of 110.4.
Looking at all 120 FBS teams that have played the last five years, these are the programs that recruiting rankings predict rather well. We know about Alabama, which has no room to go up from No. 1 in recruiting rankings but also hasn't fallen either.
Strangely, Colorado State's average recruiting ranking and average F/+ ranking over the last five years are somehow both exactly 89.2, although the Rams have been on the rise lately, with Jim McElwain leading them to a 41st-place finish in F/+ in 2014 before taking the Florida job.
So what do we know about recruiting rankings? They can be remarkably accurate, and they can also be remarkably off-base. It all depends on which team you're looking at and, more importantly, who's coaching.