Sports

History tells us the NFL is terrible at evaluating quarterbacks

We could come close to an NFL record this year before any players take a snap. There are five quarterbacks who could come off the board on Day 1 of the draft, which would tie the 1999 draft for the second most since the merger. The only draft to post six first-rounders is the legendary Class of 1983, which delivered a trio of Hall of Famers in John Elway, Jim Kelly and Dan Marino.

As much as the league seems to be struggling to pick between the prospects in this year’s class, though, the coaches and executives of 1983 weren’t able to separate the wheat from the chaff until well after the fact. Elway was the first overall pick, but the Chiefs still managed to draft Todd Blackledge seven picks before Kelly. Blackledge threw 29 career touchdowns. Kelly topped 29 in 1991 alone. Tony Eason was taken one pick after Kelly and 12 picks before Marino, who would post the greatest passing season in league history to that point during his second campaign.

A league full of coaches and personnel executives who had spent and would go on to spend the majority of their lives working in the game of football were not able to pick between a trio of future Hall of Famers and two guys who would fail to make a single Pro Bowl. (Ken O’Brien, drafted after Eason and before Marino, at least made two Pro Bowls over his 10-year career.)

Thirty-five years later, I’m not entirely convinced we’ve gotten much better at evaluating quarterbacks. The league has access to more information than ever before, but the job has become tougher. A wider range of passing offenses at the collegiate level have made it more difficult for obstinate coaches to translate amateur success into bland professional schemes. Passers come better prepared for the pre-draft process than ever before and are far more selective about throwing at the combine.

As a result, the range of opinions — anonymous and otherwise — we hear about these players before they enter the league is truly remarkable. The error bars are impossibly large. Ask around about Wyoming quarterback Josh Allen and you’ll hear that he’ll turn into budding MVP candidate Carson Wentz or Titans washout Jake Locker. You’ll hear that Heisman Trophy winner Baker Mayfield will turn into either Johnny Manziel or Russell Wilson. This doesn’t happen in other sports. Jaylen Brown didn’t enter the NBA draft only to be compared to both Jimmy Butler and Bill Murray in “Space Jam.”

Picking the right quarterback is the most important thing an NFL organization can do. The Browns famously didn’t believe in Wentz and traded the second overall pick in 2016 to the Eagles, who did. The rest is history. You can basically get everything else wrong and still repeatedly make it to the playoffs with the right quarterback, as we saw in the first few years of Andrew Luck‘s career during Ryan Grigson’s reign as general manager in Indianapolis. It is not hyperbole to suggest getting this decision right is worth hundreds of millions of dollars.

And yet history tells us that the league will wrongly evaluate these prospects. Chances are that one or two of these five passers will turn into superstars, but it’s unlikely that those one or two will be the first quarterbacks taken on draft night. Some fans are about to buy authentic team jerseys they’ll quickly regret. Thousands of scouts spent tens of thousands of nights in hotels around America for decisions nobody will want to claim three years from now. We should be able to do better than this.

So, what has gone wrong? Why can’t we reliably figure out which quarterbacks will turn into superstars? And can we fix it in time to evaluate this year’s class?

Over the next two days, I’ll look into how and why we struggle with the most important part of the draft process. We’ll try to answer the “why” in Part 2 on Tuesday, as well as evaluate whether the problems are fixable and apply what we know (and don’t know) to the Class of 2018. But let’s start with what recent history can tell us.

How bad is the problem?

Pretty bad. We’re about to hit the 20-year anniversary of the 1998 draft, which famously saw Colts general manager Bill Polian agonize over whether to take Peyton Manning or Ryan Leaf, as Peter King documented for Sports Illustrated. Polian got his call right. The Colts grabbed Manning while the Chargers traded up to No. 2 and happily settled for Leaf, who threw 36 interceptions over 18 starts and was out of football by 2001.

Since then, it hasn’t been quite as easy. We’re still judging the most recent classes, and there are different ways to evaluate quarterbacks, but the first quarterback taken in the 19 drafts since 1998 has been the most productive and/or successful passer from that class only five times. It hasn’t happened since 2011, when the Panthers controversially chose Cam Newton over Blaine Gabbert and a bevy of dominant defenders with the first overall pick. (More on that later.) That was the fourth consecutive year in which the league correctly chose the most productive pro passer first, but the success rate wasn’t good before that and hasn’t been good since.

You can quibble with a couple of those choices. Maybe you prefer Luck over Wilson, even though Wilson has won a Super Bowl and has been far more productive in Seattle. Perhaps you think Jimmy Garoppolo has already proved to be better than Derek Carr. And there are some cases in which there really were no winners: The Bills would probably prefer to have Geno Smith over EJ Manuel, but you suspect they would rather have stayed out of the quarterback market altogether in 2013. You get the idea, though: The first guy often isn’t the best guy.

It’s fair to note that this doesn’t tell the whole story. Teams that pick first are often some of the worst organizations in football, and they’re among the worst because they’re bad at talent evaluation and subsequent development. It can become a self-fulfilling prophecy — in other words, the Browns stay the Browns.

From what we know, though, even some of the league’s most decorated talent evaluators struggle with evaluating quarterbacks. When he was deciding between Leaf and Manning in 1998, Polian paid the late Bill Walsh $5,000 to evaluate both passers, a price that I suspect most teams would love to pay for a quarterback evaluation today. If you were making a list of the best quarterback evaluators in league history, the architect of the 49ers dynasty might be a good place to start.

Quotes from the report made their way into a USA Today story at the time by Richard Weiner, and surprisingly, Walsh raved about Leaf. “He is gifted, in just a natural throwing motion that is so quick,” Walsh reportedly said in his evaluation. “With a flick of his wrist, he can get the ball just about anywhere he wants. He is a good competitor, amazingly agile and smooth and graceful in his movement as a big man can be. He handled the Washington State offense beautifully. In a sense, it was an aerial circus.” Mike Shanahan, then the coach of the Super Bowl-winning Denver Broncos, said the “excellent” Leaf was “a big, strong kid with unbelievable arm strength. He’s played in a system very similar to a lot of NFL teams. That cannot be underestimated.”

At the same time, we don’t know what Walsh’s report said about the future Hall of Famer on the other side of this comparison, and most evaluators preferred the Tennessee star. A poll of executives at the time found that 20 of 25 observers preferred Manning, while three preferred Leaf and two couldn’t decide. Then-Steelers general manager Tom Donahoe, a member of the Eagles’ front office staff when it chose Wentz in 2016, suggested that you couldn’t go wrong with either guy. Colts coach Jim Mora said that both Leaf and Manning would be good players in the NFL, while Seahawks coach Dennis Erickson believed that Leaf would turn into one of the “future stars of the league.”

We’re lucky to have that many quotes about the Leaf-Manning debate on the record. Usually, pre-draft scouting evaluations are the domain of the dreaded anonymous scout or personnel executive, whose comparisons are often facile and occasionally prejudicial. It’s only after the fact that those executives come out and reveal that they had a first-round grade on that franchise quarterback they passed on three times and let someone else take in the fourth round.

Fortunately, to get a better sense of what the league actually thought about quarterbacks, there’s Bob McGinn. The legendary Packers beat writer took an annual poll of anonymous evaluators before the draft during his time with the Milwaukee Journal-Sentinel, and asked them to rank the quarterbacks from each year’s draft class. McGinn then scored each of those ballots with five points for the top-ranked quarterback, four for the second-ranked quarterback and onward.

I was able to find polls going back to the 2005 draft, which was a fateful session for the team McGinn covered. Let’s run through the top five from those polls to see just how difficult it is to project quarterbacks.

2005

1. Alex Smith (63 points, 11 first-place votes)
2. Aaron Rodgers (53, two first-place)
3. Jason Campbell (25)
4. Charlie Frye (17)
5. David Greene (16)

While Rodgers challenged to be the first overall pick in the draft, the 49ers chose Smith, with Rodgers falling all the way to the Packers at 24. Many observers were concerned that Rodgers was a product of Cal coach Jeff Tedford, who previously sent Akili Smith, Kyle Boller and Joey Harrington to the pros with limited success. One NFL personnel director told McGinn that Tedford quarterbacks “all throw the ball the same way,” while another said Rodgers “is very rigid mechanically.” It’s difficult to think of a quarterback in NFL history who is more fluid in getting rid of the football off-schedule than Rodgers.

2006

1. Matt Leinart (71 points, 11 first-place votes)
2. Vince Young (61, six first-place votes)
3. Jay Cutler (55, four first-place votes)
4. Brodie Croyle (10)
5. Charlie Whitehurst (9)

Opinions were split on the first three quarterbacks, although the legendary Rose Bowl battle between Leinart and Young loomed heavily on every scout’s mind. Leinart lost his job to Kurt Warner in Arizona, while Young mixed game-changing plays, bad decisions and inconsistency during his run in Tennessee. Young made two Pro Bowls but lacked the longevity of Cutler. As then-Broncos general manager Ted Sundquist documented in 2015, Shanahan wanted to draft Leinart, only for the Cardinals to beat them to the punch. Cutler would be the last highly touted quarterback to hit the NFL with a significant losing record in college until the Titans drafted Locker in 2011.

2007

1. JaMarcus Russell (69 points, 15 first-place votes)
2. Brady Quinn (54, three first-place votes)
3. Trent Edwards (25)
4. Drew Stanton (15)
5. John Beck (8)

Kevin Kolb, who had the most success of any quarterback in this class, received only six points. Russell’s pro day was the stuff of legend, with both draftniks and ex-pros alike raving about his future. Bucs coach Jon Gruden compared it to “Star Wars,” while ESPN draft expert Mel Kiper Jr. compared Russell’s athleticism to Elway’s. Texans coach Gary Kubiak said he was sure Russell would turn into a great player, while Hall of Famer Terry Bradshaw said Russell was a “pretty easy” choice for the Raiders with the first overall pick.

Others preferred Quinn, although he fell to the bottom of the first round before the Browns traded up to snatch him away with the 22nd pick. Polian raved about Quinn, dismissing a 58.0 completion percentage while noting that Quinn had no protection during his time at Notre Dame. Others disagreed, but Quinn certainly had his backers. McGinn asked 18 scouts if they would prefer to have Quinn or Rodgers, who had thrown only 31 passes with limited success over his first two seasons backing up Brett Favre in Green Bay. Just four of those 18 scouts chose Rodgers while 12 chose Quinn and two said it was too close to call.

2008

1. Matt Ryan (71 points, 12 first-place votes)
2. Brian Brohm (53, one first-place vote)
3. Chad Henne (38, two first-place votes)
4. Joe Flacco (36)
5. John David Booty (11)

There were concerns at the time about Ryan’s accuracy, but after completing just 59.9 percent of his passes at Boston College, Ryan is up to 64.9 percent as a pro. Brohm, a quantitative darling who completed 65.8 percent of his passes at Louisville, failed to unseat Rodgers and never developed into a successful pro. Flacco rose late up draft boards by virtue of his arm strength, although McGinn astutely noted that the famously quiet Flacco could be characterized as an introvert.

2009

1. Matthew Stafford (46 points, 11 first-place votes)
2. Mark Sanchez (39, five first-place votes)
3. Josh Freeman (21, two first-place votes)
4. Rhett Bomar (1)
5. Pat White (1)

Stafford was another quarterback with a below-average 57.1 completion percentage in college, but he has improved his mechanics and accuracy as a pro, with the move to a short passing scheme under Jim Bob Cooter pushing him closer to 66 percent over the past several seasons in Detroit. Sanchez nearly parlayed his cool demeanor and lone year of college success into coming off the board second to the Rams, only to instead go to the Jets at No. 5 as part of a trade that the Browns somehow still lost.

2010

1. Sam Bradford (80 points, 20 first-place votes)
2. Jimmy Clausen (57)
3. Colt McCoy (30)
4. Tim Tebow (21)
5. Dan LeFevour (5)

The scouts got this one right, although Bradford never developed into the superstar some projected he would. Falcons general manager Thomas Dimitroff told McGinn that Bradford was “a tall-stature guy who has that prototypical stand in the pocket,” which seems like a roundabout way of saying that the 6-foot-4 Bradford was tall. The flashpoint was obviously Tebow. McGinn polled 21 scouts about his future, and the anonymous ones mostly got it right — 16 said he wouldn’t turn into an NFL starter, but eight of them suggested they would consider taking Tebow in one of the first two rounds of the draft.

2011

1. Blaine Gabbert (105 points, 16 first-place votes)
2. Cam Newton (100, 11 first-place votes)
3. Jake Locker (43)
4. Ryan Mallett (43)
5. Christian Ponder (37)

You’ll note that this list doesn’t include Andy Dalton (15 points) or Colin Kaepernick (11). Opinions were split on the class. One scout rather hilariously told McGinn that there were only second- and third-rounders at the position in the pool and bemoaned the fact that there was no Bradford or Freeman. McGinn polled the scouts on Newton’s future, and only two of the 24 said he would become a perennial Pro Bowler. Nine said he would be a solid starter, while another nine said he would play without ever becoming effective. Four said he would be a bust.

2012

1. Andrew Luck (95 points, 19 first-place votes)
2. Robert Griffin III (75)
3. Ryan Tannehill (57)
4. Brandon Weeden (18)
5. Brock Osweiler (14)

Luck was the highest-regarded prospect of this generation, so it’s no surprise he received every first-place vote from the queried scouts. Among the guys who didn’t make it to the top five were Kirk Cousins (13 points), Russell Wilson (6) and Nick Foles (1). McGinn himself ranked Osweiler over Wilson, with one scout saying Osweiler was “like the CEO of a company.” That’s another roundabout way of saying someone’s tall. Many were concerned about the 5-foot-11 Wilson’s ability to throw from the pocket, but the only thing that’s prevented him from doing so as a pro has been the Seahawks’ offensive line.

2013

1. Geno Smith (74 points, 11 first-place votes)
2. Matt Barkley (58, three first-place votes)
3. EJ Manuel (35, two first-place votes)
4. Mike Glennon (28)
5. Ryan Nassib (22)

The scouts rightly thought this was a dismal quarterback class, although it’s clear to see the difference between the consensus and what actually happened on draft day. While most of the league preferred Smith, the Bills were one of the exceptions and took Manuel with the 16th pick. The Florida State passer was the only quarterback taken in the first round.

2014

1. Johnny Manziel (68 points, eight first-place votes)
2. Blake Bortles (61, four first-place votes)
3. Derek Carr (49, two first-place votes)
4. Teddy Bridgewater (41, two first-place votes)
5. Jimmy Garoppolo (16)

It’s still unclear exactly who will end up as the best quarterback from this class, but it does seem safe to say it won’t be Johnny Football. Twelve of the 20 scouts McGinn polled suggested Manziel’s career would go down as a miss, with scouts expressing concerns about his off-field behavior even before hitting the pros. Zach Mettenberger received one first-place vote, with his six points placing him just ahead of AJ McCarron‘s five.

2015

1. Marcus Mariota (85 points, nine first-place votes)
2. Jameis Winston (84, 10 first-place votes)
3. Bryce Petty (45)
4. Brett Hundley (39)
5. Garrett Grayson (19.5)

Winston was seen as a far more polarizing prospect at the time of the draft, owing in part to his off-field troubles. One scout compared Winston to a less athletic version of JaMarcus Russell. Eight of the 19 executives on McGinn’s anonymous committee suggested Winston would be a disappointment or a bust, while just one said the same thing about Mariota.

2016

1. Jared Goff (73 points, eight first-place votes)
2. Carson Wentz (71.5, seven first-place votes)
3. Paxton Lynch (52, one first-place vote)
4. Connor Cook (29, one first-place vote)
5. Christian Hackenberg (11)

Dak Prescott finished behind Hackenberg with six points. While that seems comical now, remember that the Cowboys tried to trade up for Lynch and then were pipped to Cook by the Raiders before settling on the guy who would quickly become their franchise quarterback. One scout told McGinn that Prescott had “no accuracy and no vision,” while saying the Mississippi State product wasn’t an NFL quarterback. Another suggested that the Eagles would need to sit Wentz for a year or two before he could become a real starter, while others alternately compared him to both Bortles and Ben Roethlisberger.

2017

1. Mitchell Trubisky (61 points, four first-place votes)
2. Deshaun Watson (58, six first-place votes)
3. Patrick Mahomes (56, five first-place votes)
4. DeShone Kizer (32, one first-place vote)
5. Davis Webb (23)

It’s still far too early to draw any conclusions about this class, but it does seem likely that the person who submitted a first-place vote for Kizer probably won’t be bragging about it publicly anytime soon.

On Tuesday, we’ll try to figure out why teams struggle to evaluate quarterbacks, figure out whether the problems are fixable, and apply what we know (and don’t know) to the Class of 2018.

~Source reference~