Every time a team goes for it on 4th down, the same things happen.
The crowd goes wild.
Phil Simms becomes deeply sad.
The NFL statistical intelligentsia (hi guys!) nod in approval, but hold their collective breath.
Every time a team fails to convert on 4th down, the same things happen.
The same crowd that cheered the decision speed-dials the sports radio show to demand the coach's immediate beheading.
Phil Simms fractures his tongue in the art of clucking.
The stat-minded 4th-and-1 apologists either launch into detailed sermons on the nature of probability or declare that while the decision was good, the specific play call was not.
4th down is one of the last great frontiers of football strategy. Traditionalists insist that punting is the proper decision in all but the most dire (down by six, 30 seconds left) or least risky (inches to go, ball at opponent's 39) circumstances. Coaches range from extremely aggressive (Sean Payton, Bill Belichick) to Simms-conservative (John Fox). No head coach is as aggressive on 4th downs as the hardcore number crunchers would like: nearly every statistical study suggests that in most 4th-and-short situations, going for it is the proper percentage play.
But just what are the percentages? And if going for it is the right decision, then are there ideal 4th down plays? Plays that should be avoided at all costs? To find out, I combed the Football Outsiders database and examined every 4th-and-short conversion attempt from 2010 to 2012.
What follows is the ultimate clip-and-save guide to 4th down strategy: when to run and when to pass, when to sneak and when to take a chance, and of course: when to hold 'em and when to fold 'em. You don't have to agree with Simms, nor the "never punt" brigade, but you should at least be armed with the data.
We are examining voluntary 4th down conversion attempts in non-goal-to-go settings; while 4th-and-1 at the one-yard line is an interesting situation, it is also its own beast when compared to, say, 4th-and-2 at midfield. So the following data set was assembled by:
- Starting with all 4th-and-1 or 4th-and-2 plays from 2010 through 2012.
- Eliminating any goal-to-go situations. Keep in mind that this does not eliminate 4th-and-1 from the 10-yard line, or similar situations.
- Eliminating all plays that occurred when a team was down by more than 14 points. While this does not take all of the desperate, "no choice" attempts out of the data, it removes many of them.
- Eliminating fake punts and field goals
After the scrubbing, we are left with 466 conversion attempts: about 155 per year, 4.8 per team per year. This sample is robust enough to draw conclusions from, though at the same time it is small enough that a few unusual events can cause distortions, so we must be careful about slicing and dicing the data too much.
The Raw Data
Going for it on 4th-and-short has had a 62.7% success rate over the last three years. Table 1 shows the year-by-year success rates for running and passing plays.
There is a noticeable dip in the 2011 data: 2010 and 2012 look alike and suggest an over 66% (2 out of 3) conversion rate, but 2011 calls the cops on the go-for-it party. A quick scan of the data shows no obviously anomalies, like Blaine Gabbert throwing 20 4th-and-2 passes or anything. The year-to-year data sets are small enough to allow for this kind of fluctuation. It is best to assume that the "real" success rate lies between the low of 2011 and the high of the other two years, though we will see later that there are compelling reasons to lean toward the higher rate.
Coaches run the ball on just 63.7% of conversion attempts, which sounds low. There are still some desperate pass attempts in our data -- 44% of the fourth down pass attempts occur in the fourth quarter -- but we are probably looking at a stabilized strategic decision by NFL coaches. All other things being equal, they will run about 2/3rds of the time when converting 4th-and-short.
Is that ratio optimal? Obviously, teams must pass once in a while in short-yardage situations or else the rushing success rate would decline. Pass plays have a higher reward factor then runs, outgaining runs 1032 to 831 in the study, despite half the attempts and a lower conversion rate. And passing success rates are high enough to justify their semi-regular use as a tendency breaker. A team with Peyton Manning should pass more than a team with Adrian Peterson, 4th-and-inches is different than 4th-and-a-long-2, but in general 2-to-1 run-pass ratio appears to work well for 4th down conversions.
The raw data tells us that 4th-and-short conversion attempts are not a high-risk gamble. A 63% success rate represents good odds, and those odds only go up when Drew Brees or Tom Brady is your quarterback or Peterson your running back. Running plays are more likely to succeed than pass plays, but the difference is not great enough to justify "NEVER EVER PASS IN THAT SITUATION" foot-stomping. If your coach passes 75% of the time on 4th-and-1, there may be a problem, but the occasional play-action rollout has a good chance of succeeding in its own right AND keeping defenses off balance for future handoffs.
What to Call
Okay, so you are on board with going for it on 4th-and-short. But why does your coach always run the same stupid play? He always feeds the running back right into the line of scrimmage, where the defense is waiting for him. That never works! Or he calls a run to the outside, which takes forever to develop. That never works! Or he calls the sneak, with two defensive tackles waiting on each side of the center for the quarterback. That will get your quarterback killed, and it NEVER WORKS! Or he spreads the field -- spreads the freakin' field -- and passes. THAT NEV … oh you get the idea.
Before we break down success rates by play type, let's run down a list of plays that are likely to be called on 4th-and-short:
The quarterback sneak. It's a better idea on 4th-and-inches than 4th-and-2, but it's hard to stop someone who can gain two yards by falling forward from gaining one.
An I-formation power handoff. Predictable? Sure is. YOU try stopping Marshawn Lynch when he only needs to go three feet and has a fullback in front of him. (Plays categorized in this group and the next must include two running backs and one or zero wide receivers in the formation.)
Play-action from a power formation. Well, if that handoff is so predictable, then why not fake it, maybe roll out, and throw the ball? A goal-line staple that has given third-string tight ends hope for over 50 years, this play can be effective anywhere on the field.
Pitch/outside run. Hated by most fans and announcers because it traps the running back between the sideline and 11 angry defenders, the pitch works for fast running backs and against defenses that stack the middle to stop inside runs.
Spread the field and run. The state of the strategic art, circa 1992: instead of cramming big guys in the middle of the field and running straight into the logjam, why not spread the field with three or four receivers and let the back run toward open space? (Plays categorized in this group or the next include three or more wide receivers in the formation).
Spread the field and pass. Come to think of it, if a team uses a spread formation and passes on regular downs, why stop on fourth-and-short? This tactic is more appealing late in the 4th quarter than midway through the first.
The fullback give. An old-school choice: Line up in the I-formation but give the ball to that big blocking back. A 1970s-style key breaker that works best when the tailback pretends to take a pitch or run to a different gap.
Read-option stuff. The new-school choice. It does about the same thing the old fullback give did to the defense, except that now the tailback is the fullback and the quarterback is the tailback.
We can get more technical and specific, but those eight options cover most of the play-types a team can realistically run on fourth-and-short. All of them have advantages and disadvantages. Are any of them the one true call? Should any of them be tossed away forever?
Here are the conversion rates for our eight play-types. There is some overlap in the data. The quarterback sneaks were removed from the other running plays, but other plays that could fall into two categories (like a spread-formation run and a pitch play) were included in both categories. Also, a bunch of plays fall into no particular category: the team lined up in a base formation with no bells and whistles and executed an ordinary play. These attempts are lumped in the "uncategorized" category.
You can hate the quarterback sneak all you want. It works, and it is battle-tested enough that the 80% success rate is no data blip. Assuming that it is 4th-and-inches, the sneak is such a good percentage play that it should be used far more often: if a coach calls a different play, he should ask himself why he's calling it.
Frustration at the I-formation, multiple-blocker, up-the-middle run is justified: both the pitch to the outside and a run from a spread formation have higher success rates. Outside runs also have greater big-play potential: most of the long runs in the study, including 20+ yarders by LeSean McCoy, Ray Rice and Ryan Mathews, were to the outside. The only thing better than converting on 4th down is converting with a long gain. The fact that power-formation runs between the tackles are more than twice as likely to occur as pitches, sweeps, or spread the field tactics suggests that coaches are missing an opportunity to increase their 4th-down conversion rates by diversifying their play calling.
Passing on 4th down remains a bad gamble, though not so bad that it is not worth mixing into the 4th down package. More than half of those 97 spread-formation passing plays occurred in the 4th quarter, when time and urgency were often a factor. But with spread-run conversions about 50% more likely than spread pass conversions, coaches may want to reconsider their decision to pass five times more often than they run from a spread on 4th-and-short.
Note the rarity of fullback gives and "read-option stuff," which in this study collects quarterback bootlegs or any other designed quarterback run. The asterisks are a reminder that you can't draw conclusions from plays that happen less than four times per season across the whole league. The rarity of these plays, particularly options, looks like another missed opportunity for coaches to be less predictable on 4th-and-short while getting the ball to the players most like to bruise out three or four tough feet.
Have you ever thought about the quarterback sneak? I mean, really thought about it? On the surface, it is pretty simple: if the quarterback is just going to take a snap and plow into a blocker's back, there is not much to do from a play design standpoint. But there are things the offense can do to make the sneak sneakier:
Motion. By moving some running backs or receivers around, the offense can make it look like they are planning something more complicated than a sneak. The defensive line might even shift in response to the motion, opening a little bubble for the quarterback to aim for.
Quick Count. Get to the line, and BAM: snap it as soon as everyone is set.
Hurry-Up Offense. If you are going to hurry to the line, why even huddle? This strategy is not as useful as you might think, because so many 4th-and-short situations are preceded by measurements, which slow everything down for the defense.
Hard Count. The opposite of a quick count. A varying cadence could get the defense to jump offsides. It could also fool the defense into thinking that the offense is just trying to lure them offsides, causing defenders to slow their reactions to the snap.
How common are these strategies? I went back and watched film of all of the fourth-down quarterback sneaks of the 2012 season. It turns out that few teams add any window dressing to their sneaks:
Total Sneaks: 22
Pre-snap Motion: 5
Quick Count: 12
Hard Count: 1
The lone hard count was executed by Mark Sanchez, who converted the first down. The quick counts were 9-of-12 on conversions, the regular counts 9-of-10. Motion was 5-for-5. These numbers are too small to draw conclusions from, except for the conclusion that teams could probably make the sneak more effective by adding a little engineering.
Only one sneak took place from a full-house backfield last year, for example: a failed effort by the Jaguars. There were no pistol diamond or kings-queens formations, the kind that force defenders to worry about four potential ball carriers. Nearly every sneak is run from a single-setback formation, even though the teams running them may use the shotgun-pistol and multiple backs in most of their offense. If a tiny wrinkle can increase the sneak's effectiveness from 80 to 85%, it would be worth the effort for the team that earns one or two extra first downs.
Drawing the Defense Off
Drew Brees came to the line on 4th-and-1 on Sunday against the Bears looking like a man ready to run a sneak. But no one moved when he made a hard count … except half of the Bears defense. The penalty netted a first down for the Saints, and made the Bears look pretty silly. That "draw them off" tactic should never work, because defenses know it is coming.
But the hard count does work. Defenses jumped offsides, encroached, committed a neutral zone infraction or had 12 men on the field (a sign of defensive disorganization caused by the unusual circumstances) 21 times on 4th-and-short from 2010 to 2012. A play that works seven times per year is worth attempting, especially when the offense has little to lose. Typically, the offense has great field position and plans to punt anyway if the gambit fails, so all they will sacrifice is a timeout. (If that. Many teams just take a delay of game penalty to give the punter room.)
And of course, with offenses growing more bold on 4th down and sneaks working 80% of the time, the defense cannot really afford to lean back and giggle when someone like Brees starts barking at the line. The hard count lurks in the back of fourth down strategies, an extra tool in the offensive toolkit.
A large percentage of fourth-down conversions take place in what Gregg Easterbrook used to call "The Maroon Zone": the region between the opponent's 40 and 20 yard lines. This makes sense, of course: that is the region of unappealing 55-yard field goal attempts and punts which will only net a few yards of field position. Here is a breakdown of the conversion attempt rates and success rates by field position:
At first glance, this data appears to have an "a-ha" revelation. Success rates plummet in the red zone, making 4th-and-1 gambles from the 12-yard line, like the ones the Falcons and Bengals failed to convert two weeks ago, an apparently bad percentage play. If the chance of success plunges to 50-50 (due to the safeties' ability to play closer to the line, for example), why not just take the easy field goal?
A deeper look reveals a distortion that lurks within all of this data: the desperate 4th quarter conversion attempt. We cleaned many of these out of the data by eliminating two-touchdown deficits, and I did not want to scrub too heavily by eliminating all 4th quarter attempts, because there are still plenty of late-and-trailing situations when the coach has freedom of choice. (Think of the Bills-Ravens game in Week 4 for example: John Harbaugh kept kicking field goals in the 4th quarter, down by two scores, because there was time left for another possession.) But when you are going for it in the red zone in the 4th quarter, chances are you have no other choice. That brings several variables to the party: the team making the attempt is trailing, which means they are probably weaker than their opponent, and the clock may be a factor. Long story short: there are many reasons to expect these plays to have a lower success rate than plays in which the coach's hand is not forced to make an attempt.
In the 4th quarter, teams are 20-of-50 on red zone conversion attempts, or 40%. They are 43-of-65 on red zone conversions in other quarters, or 66%. Red zone success rates are not much different from overall success rates; they are just more likely to include unique, unfavorable circumstances.
- There is no one "bad play" to attempt a 4th down conversion, at least among the plays a team might realistically attempt. (The double-reverse flea-flicker would be pushing luck.) The worst percentage play is a pass from a three-plus receiver formation, but the success rate is still encouraging enough (and likely tamped by desperate 4th-quarter plays) that it is still worth keeping the ball in Brady or Peyton's hands.
- The closest thing to a "money play" is the quarterback sneak, which could be made more effective if coaches applied a little more ingenuity to the play.
- Options and handoffs to fullbacks are used so infrequently that they may be an unexplored mine of successful 4th-and-short conversions. If the 49ers are going to surround Colin Kaepernick with Frank Gore and two blocking backs on first down, making everyone in the backfield a likely ballcarrier, it only makes sense that they will try it on 4th down.
- Outside runs and runs from spread formations are underused, based on their success rates.
- The overall 4th-and-short success rates are highly stable, from a data standpoint. Despite the decrease in success rate in 2011, there are no anomalies in the data. Break down the success rate by play type or field zone, and you get consistent results. There are no meaningful situations in which teams are suddenly 2-of-18 or anything; there's an understandable success rate decrease in the 4th quarter, but otherwise rates hover in the 55-80% range, however you slice them.
That stability is the most interesting finding of this study. The 62.7% success rate we started with is much closer to 66.7%, or two-thirds, if a few more dire circumstance attempts are teased out. That two-thirds certainly goes down when the Jaguars offense is facing the Seahawks defense, though it goes way up when the Broncos offense faces the Raiders defense. It probably rises when a team has been marching down the field and falls when a team has been sputtering, because football players are not robots, and confidence does play a role when we start shaving tenths of a percent. It probably goes up in a dome and down in the snow: there is not enough data to draw those conclusions. But it clearly goes up and down from a 2-out-of-3 chance of converting a first down and retaining possession, meaning the base probability is really, really good.
So when the guys at the water cooler are debating the latest 4th down gamble, bring this up: if a team goes for it on 4th-and-short three times in a game, they are statistically likely to gain two 1st downs and turn the ball over on downs once.
If the team makes those three attempts at, say, the opponent's 40-yard line, that means they will get the ball in scoring position twice but give the opponent good field position once. Which would you rather have: the ball twice in scoring position (with an opponent around its own 40 once), or three punts? Unless you are talking about the 2000 Ravens, it's really not a choice at all.
The problem is that teams rarely attempt three 4th down conversions in a game. Many only attempt four or five per season, so those failures sit in our minds with bullseyes attached to them. Critical 4th-down failures can be talked about for years. Does anyone remember the Saints' successful conversion on Sunday? The Seahawks conversion? The bumbling, faltering Texans gained 19 yards on 4th-and-1 against the 49ers before things got totally out of hand: it was a smart, successful decision on a night of failure. But successful attempts get buried in the middle of scoring drives. The touchdown (or drive-killing interception) five plays later makes the highlight real. The failed conversions affect a coach's reputation and define his entire tenure.
No wonder coaches say "damn the percentages" and punt. Even a 2-to-1 success ratio could place him on the hot seat if a few of those "1's" line up at the wrong time. The solution for the nervous coach is not to go for it less, but go for it more. Three years of data shows that the successes will overwhelm the failures.
It takes counterintuitive courage to steer directly into those waves, but coaches are gradually learning what to do on 4th-and-short, and how to do it.