Talent vs. Environment

[NOTE #2: the methodology used in the following study was, to an extent, inspired by message-board poster named "Arschloch" and his creative attempt to rank the all-time best RBs.]

Take a look at these the numbers of these two QBs over a 16-game season:

                 Pass Yards     TD     INT    Yards/Att    Team W/L
-------------------------------------------------------------------
Quarterback #1       4057       31      13       7.70        11-5
Quarterback #2       3207       17      18       7.37         8-8

At this point, you're saying, "alright, I've seen this tired old gag before. The top stat line obviously looks better than the other, so now Drinen is either going to (a), try to argue in some weird way that the bottom QB is actually better, or (b), reveal that the top stat line somehow belongs to someone who is universally thought to be worse than the owner of the bottom stat line. That surprising fact will then lead to some unexpected point. Yawn."

But that's not where I'm headed with this.

Quarterback #1 looks better, and I'm not going to try to convince you otherwise. In fact, I'd say there's no possible way to argue, just on the basis of what's shown here, that Quarterback #2 is better. There is some question about how much better QB #1 is than QB #2. But I'm not going to try to sway you on that either.

Personally, I like to look at yards per attempt first. QB #1 has only a relatively small lead in that category. But the much higher yardage total, combined with the higher yards per attempt, is a strong point in QB #1's favor. And I also think TD/INT ratio is important, and QB #1 looks much, much better there. While it's dangerous to attribute too much of the success or failure of an entire team to the QB, QB #1 grades out quite a bit better in that respect too, for whatever that's worth. All things considered, I'd say QB #2 looks substantially better than QB #1.

The above paragraph is purely my opinion. You don't have to agree with any of it in order to get something out of the rest of this article. You should, at this point, make your own judgement as to how big the gap is between these two.

Now I'll add one more piece of information: these two quarterbacks played with exactly the same supporting casts. Running backs? Same. Defenses? Same. Offensive lines? Same. Coaching staff? Same. They were in exactly the same environment.

How's that possible?

It's possible because QB #1 represents a composite of 8 of the best QB seasons of the past few years, and QB #2 represents a composite of their backups during the games in which they didn't play.

Here's the deal: I looked for all QBs during the years 1995-2001 who fit the following criteria:

1. Played at least 10 games in the given season, but
2. did not play a full 16 games. And
3. were among the top 5 QBs in the league for that season in terms of passing fantasy points per game (passing fantasy points = (pass yards) / 25 + (pass TDs)*6 - (INTs)*3).

That yielded 8 QBs. Quarterback #1 above is the combined stat line for those 8, scaled to a 16-game season. Quarterback #2 is the combined stat line, per 16 games, of their backups. Since there's no difference in the "system" or in the surrounding talent, the difference in stats must be the result of talent.

But that's probably stretching it a bit. It's not clear that the supporting casts were exactly the same. And maybe the backups played against tougher schedules. Or maybe this just happened to be a really crummy group of backups. To see, let's examine each of the 8 cases individually.

• Steve Young / Elvis Grbac - 1995: Young played 11 games. Grbac was, at the time, a 25-year-old second-year man. Grbac's 5 games were against teams with a combined record of 44-36. This was weeks 8-12, so all the games were meaningful in the sense that the 49ers had not clinched the division title or anything like that. The 49ers were 3-2 in the Grbac games.
• John Elway / Bill Musgrave - 1996: This one is kind of a cheapie. Elway played every game but one. Musgrave's game was in week 15 against a very tough Packer team. I believe the Broncos had already clinched home field throughout the playoffs. It was essentially a garbage game. Or at least Denver turned it into one, losing 41-6.
• Warren Moon / Jon Kitna - 1997: Kitna was a rookie. Moon missed one game, near the end of an uneventful 7-9 season for the Seahawks. Kitna's game was week 16, a meaningless game against a 4-12 Raider team. Seattle won it, 22-21.
• Steve Young / Ty Detmer - 1998: Young missed one game. Detmer's game was a week 10 win against the expansion Panthers.
• Vinny Testaverde / Glenn Foley - 1998: Foley actually started the year as the Jets' QB that year -- his two games were the first two, both losses against the 12-4 49ers and the 6-10 Ravens.
• Chris Chandler / Steve DeBerg / Tony Graziani - 1998: Chandler missed weeks 8 and 13 during the Falcons Super season. DeBerg was 44 and in his last season. Graziani was 25 and in his second. Both played in both games, as the Falcons beat the 4-12 Rams and lost to the 12-4 Jets.
• Brian Griese / Gus Frerotte - 2000: Griese played 10 games. Frerotte went 4-2 against a pretty weak group of teams (combined record: 37-59).
• Elvis Grbac / Warren Moon - 2000: It's kind of interesting that both these guys appear on both sides of the ledger in this study. This time, Elvis was the stud and Moon was the backup (age 44 at the time). Moon played in one game, beating the 1-15 Chargers by a point in week 13.

You are encouraged to draw your own conclusions here. To me, this looks like a very standard set of NFL backups playing a pretty standard collection of games. I don't believe that this particular sample is skewing things one way or the other. If anything, the two backups who played the most games (and hence contributed the most to the composite stat line of QB #2) were Grbac and Frerotte, two pretty respectable QBs. So I think it's reasonable to estimate the difference between a stud QB and a typical backup as the difference between QB #1 and QB #2 above.

Talent vs. System is the grand-daddy of all football debates, and I won't even pretend that I've come anywhere close to settling it definitively or conclusively with this piddly little study. But, as one who has always sided with the talent side in this debate, I don't see anything in this study that leads me to change my mind. Great QBs are great, for the most part, because they're great. Not because they were in the right place at the right time.

Unless otherwise noted, all stats come from football-reference.com and the disclaimer applies