Luck and Competitive Balance, Part 1: The Post-Season

The more I look at the numbers being crunched by the Berris, Schmidts and Brooks of this world, the more convinced I become that getting lucky is the single most important factor determining the outcome of baseball games, and baseball seasons.

I’ll admit, it’s not easy for me to acknowledge the role played by luck in baseball.  I want to believe that my sports teams succeed because they achieve a standard of excellence, or because their desire to win is higher than the other team’s, or because they possess the “right stuff” to succeed under pressure.

The numbers say otherwise.  The numbers say that the Yankees got lucky in 2009, when they won the World Series, and that they got really lucky in the late 1990s when they won three World Series in a row.  But the Yankees’ luck wasn’t there from 2001 to 2007, when they had consistently excellent teams but could not “buy” a world championship.

You’ve heard the expression that it’s better to be lucky than good?  Well, maybe some of the 13 championship teams since 1989 were very lucky and not all that good.   This would affect our understanding of competitive balance.  It might be that the gap between the best and worst teams in baseball is wider than it appears, but that the luck factor frustrates the ability of the best teams to dominate the sport.

In this post and the posts that follow, we’ll look hard at the luck factor in baseball.  Let’s start the analysis with an examination of the process that decides baseball’s world championship: the post-season.

Luck In The Post-Season

Is there luck in the post-season?  To answer this question, let’s start with the 2009 post-season.  The Yankees had the best regular-season record in 2009, and they were favored by many experts to win it all (which is exactly what they did).  Were the Yankees the true favorites to win the 2009 post-season?  What were the odds that the Yanks would make to a late October or early November tickertape parade?

To figure out the odds of winning a playoff season, the sabermatricians use the “Log5” method derived by Bill James in the 1980s.  This formula uses the season-ending win percentages for any two teams meeting in the playoffs, and factors in the home-field advantage, to determine the chance each team has to win the playoff matchup.  Below are the Log5 charts for a 5-game and 7-game series, borrowed from the book “Baseball Between The Numbers: Why Everything You Know About The Game Is Wrong”, by Jonah Keri and a team of experts from Baseball Abstracts:

5-Game Series        
  Away Team      
  .700 .650 .600 .550 .500
.700 51.5 62.1 71.2 78.7 84.8
.650 40.9 51.5 61.4 70.2 77.8
.600 31.6 41.5 51.5 61 69.7
.550 23.7 32.5 42 51.5 60.8
.500 17.3 24.7 33.1 42.1 51.5
           
           
7-Game Series        
  Away Team      
  .700 .650 .600 .550 .500
.700 51.3 63.5 73.8 82 88.2
.650 38.9 51.3 62.7 72.8 81
.600 28.3 39.7 51.3 62.3 72.2
.550 19.8 29.4 40.1 51.3 62.1
.500 13.3 20.9 30 40.4 51.3

Let’s use these charts to figure out the Yankees’ odds of winning the 2009 World Series.  In 2009, the Yanks finished with a .636 winning percentage, and their playoff opponents had winning percentages of .533 (Twins), .599 (Angels) and .574 (Phils).  So we can use the tables above to make the rough estimate that the Yanks’ chances of beating the Twins in a 5-game series  were about 70%, and that the Yanks had about a 60% chance of beating the Angels and about a 65% chance of beating the Phillies.  So, great!  The Yankees were favored in each series, no wonder they won it all!

But wait.  The fact that the Yanks were favored in each series does not mean that the Yanks were favored to win three series in a row.  To figure out the odds of winning more than one series, you need to multiply together the odds of winning each series: 70% x 60% x 65%.  If you do this, then at the outset of the 2009 offseason the Yankees had about a 27% chance of emerging as World Champion.  In horse racing parlance, the odds against them were about 3-1.

This is typical.  According to Keri and the “team of experts” who wrote “Baseball Between The Numbers”, it is rare for any team to have better than a 25% – 30% chance to win the World Series.  That’s not much of a chance.  It means that a team can be the best in baseball for eight years, and (if luck holds) maybe win the World Series two of those years.  This roughly parallels the Yankees’ record in the 2000s: the Yanks made the playoffs in 9 of those 10 years, they had the best record (or tied for best) in 4 of those 10 years, and they took home two World Series championships.  The Red Sox were luckier in the 2000s: their record was 6 playoff appearances, tied for the best record in baseball in 1 of those years, and two World Series championships won.

Berri, Schmidt and Brook provide examples of teams that were not as lucky as the Yanks and Red Sox.  Remember the 2001 Seattle Mariners?  They won 116 games in 2001, compared to the Yankees’ 95 games.  The Mariners were about an 80% favorite to beat the Yanks that year based on the Log5 calculation, but the Yanks knocked out the M’s four games out of five.  Or how about the poor Oakland A’s – between 1999 and 2004, the A’s won more games than any team except the Braves and the Yankees.  But they never made it out of the first round of the playoffs in these years.  Or the Braves: consecutive division championships from 1991 to 2004, but only one World Series championship.

Some of you might object at this point.  Sure, the odds say that a team like the Yankees might win about one World Series in every four tries.  But the odds don’t know Derek Jeter, and Andy Pettitte, and Mo Rivera; the odds can’t factor clutch into their calculations.  In response, I’ll simply point to my earlier post on clutch, and to the many analysts who have studied the phenomenon of clutch.  Most of these analysts doubt that there’s any such thing as a clutch performer, and the few who believe in “clutch” see its effects as rather small.  So, let’s make the best possible case for clutch: a hitter like Derek Jeter in 2009 might be able to raise his .330 regular-season batting average to.360 in clutch post-season situations.  That still means that Jeter (or any other clutch hitter) is going to fail more often than not to deliver a clutch hit in a clutch situation.

What about Mariano Rivera?  Possibly no baseball player has been celebrated more for his post-season performances than Mo Rivera.  Here’s Mo’s post-season breakdown from Wikipedia: a postseason won–loss record of 8–1, a WHIP of 0.77, an ERA of 0.74, 39 saves, and 14 two-inning saves.  Does Mo pitch better in the post-season than during the regular season?  Well, based on some of these statistics, Mo does appear to raise his game in the post-season.

But a closer look at Mo’s statistics on FanGraphs suggest otherwise.  Mo’s lifetime xFIP (expected fielding-independent pitching) is actually higher during the post-season, 3.14 compared to his regular season 2.98 (this is only measured since 2002 on FanGraphs).  Mo also appears to be the beneficiary of some post-season luck: the BABIP (batting average on balls in play) against Mo during the post-season is .229, compared to a regular season average of .274, and his LOB% (left on base percentage) is 90.1% in the post-season, compared to 80.1% in the regular season.  Please understand, Mo’s regular season BABIP numbers and LOB% already push the limits of what we think pitchers can achieve without getting lucky.  Mo’s post-season numbers suggest that come October, Mo has been both lucky and good.

Let’s be clear on this.  Derek Jeter and Mariano Rivera are great players.  They are future first-ballot hall of fame inductees.  With great players like Mo and Jeter on their roster, the Yanks enter any post-season with at best about a 25%-30% chance of winning the World Series.  But the chances are always going to be better that some team not named the New York Yankees will win the last game of the World Series. There is no way around this, no matter how “clutch” any Yankee or Yankee team might be.  The greatness of any Yankee team cannot overcome the luck factor built into baseball’s playoff system.

If we cannot rely on clutch to overcome the Log5 odds, is there anything a team can do to improve its chances in the post-season?  Perhaps.  In “Baseball Between The Numbers”, Nate Silver and Dayn Perry argue that teams built around run prevention (i.e., defense and pitching) perform better in the post-season than teams built around great hitting.  In particular, having a great ace relief pitcher seems to be important in the post-season.  However, I have not seen this analysis confirmed elsewhere, and even Silver and Perry think that run prevention accounts for only about 11% of playoff success.

Even Silver and Perry conclude that when it comes to the baseball playoffs, “it’s plain old luck that prevails.”

I doubt that any of this comes as a great surprise to knowledgeable baseball fans.  It’s been stated, often and elsewhere, that winning the post-season is a matter of luck.  So I’m not breaking any new ground here.   In fact, the luck factor built into the post-season is often cited by those who argue that baseball has good competitive balance.  The thinking here seems to be that balance exists because the odds are always going to be 3-1 or greater against the Yankees, and because the other 7 teams in the post-season all have a chance to win it all, no matter how weak those teams might be.

This argument is so commonly made, it is strange that the obvious weaknesses in the argument are rarely if ever pointed out.

  1. The baseball playoffs are not entirely a crapshoot.  Go back to Bill James Log5 calculation.  Sure, no team will be a favorite against the field to win the World Series.  But just as sure, the stronger teams have a distinct advantage over the weaker teams.  Yes, the Yanks had only a 27% chance of winning the World Series last year, but that’s a lot better than the odds against the Minnesota Twins, who had only about a 4% chance of winning the World Series (assuming they would have had to beat the Yankees, Angels and Phillies to do so).   Unless Minnesota can improve on those 4% odds, they might have to play in 25 World Series to win one … because 4% does not balance against 27%.
  2. In order to win a World Series, all but the luckiest teams have to make frequent visits to the post-season.  In the parlance of the lottery, you’ve got to be in it to win it; it’s also true that when it comes to the baseball playoffs, you have to be “in it” consistently to have a reasonable chance to win it all.

It is highly misleading to find competitive balance based on the number of teams that have made it once or twice to the playoffs.  A truer measure of competitive balance is to look at how often a team makes the playoffs.  The following chart shows the number of times each baseball team has made it to the post-season since 1995:

NYY 14
ATL 11
BOS 9
STL 8
CLE 7
LAD 6
HOU 6
LAA 6
OAK 5
MIN 5
SEA 4
SDP 4
SFG 4
CHC 4
ARI 4
COL 3
TEX 3
NYM 3
CHW 3
PHL 3
BAL 2
FLA 2
CIN 1
MIL 1
TAM 1
DET 1
WAS 0
KCR 0
PIT 0
TOR 0

From this list, we can see that the Minnesota Twins (the 25-1 longshot in the 2009 playoffs) actually appear to be one of baseball’s elite teams, with five playoff appearances in the last 15 years (tied for 9th best on the list).  The Twins’ chances of taking home a World Series championship look a whole lot better than those of the Detroit Tigers, the Milwaukee Brewers, or even the Florida Marlins.

Funny thing about the Florida Marlins.  Lots of analysts point to the Marlins as an example of how a small-budget team can achieve post-season success (that is, when the Marlins aren’t selling off their best players and slashing their payroll).  The Marlins have made the post-season twice in the last 15 years, and they won the World Series both times.  What were the odds?  According to my rough Log5 calculations, the Marlins had about a 7.5% chance of winning the World Series in 1997, and again in 2003.  The odds against the Marlins winning these two World Series in two tries: about 175 to 1.

It may be telling if the argument in favor of baseball’s competitive balance relies, even in a small way, on the success of a 175-1 longshot.  Over the last 15 years, a couple of other longshots (based on their regular-season performances) have won the World Series: the 92-win 2001 Diamondbacks and the 83-win 2006 Cardinals.  Any World Series winner is lucky, but these teams got very lucky.

There can also be an element of luck involved when someone looks at post-season results to assess baseball’s competitive balance.  A study of competitive balance in baseball performed in, say, 2000 (when Bud Selig’s Blue Ribbon Panel determined that baseball had a competitive balance problem), would look with concern at the Yankees’ four World Series championships from 1996-2000.  But a similar study performed in, say, 2008 (after recent World Championships by the Diamondbacks, Cardinals, Rays, Marlins and Red Sox) might conclude that the competitive balance problem had been solved .  Of course, it’s unlikely that baseball’s competitive balance was any different in 2008 than it was in 2000.

What’s more likely is that baseball’s post-season tells us little about competitive balance.  In the post season, a good team, even an OK team, often beats a great team.  That does not narrow the gap in the ability of great, good and OK teams — it only points out that even a great team cannot overcome the luck factor built into the post-season.

So, here’s one take-away lesson: the fact that lots of teams have a chance to win the World Series does not mean that baseball has good competitive balance.  It simply means that baseball does not need competitive balance in order to give lots of teams the chance to win the World Series.  If you’re a dispassionate analyst like Berri/Schmidt/Brook, perhaps this state of affairs might appear to be satisfactory to you.

But if you’re a baseball fan, if your team is the Reds or the Orioles or the Royals, then of course baseball’s competitive balance might not seem good to you.  You may have noticed that while the World Series odds against the 2009 Yankees were around 3-1, those odds were more favorable than the odds in favor of your team (which were 0).  Moreover, you’ve noticed that year after year, the Yankees retain roughly those 3-1 odds, while the odds in favor of your team usually remain 0 and are rarely as good as the odds favoring the Yankees.

Luck is a funny thing.  If an activity like the baseball playoffs is controlled by luck and a combination of other factors, then those other factors become magnified in importance.  If you run a poor baseball team, you can only hope to get lucky.  If you run the Yankees, you can hope to get lucky, and you can also grab an added edge (all 17.6% of it) by spending on payroll. Unless or until we can identify something other than luck and payroll that leads to post-season success, the advantage here goes to the  Yankees.

And if you root for a team other than the Yankees, you don’t have to like this state of affairs.  Not one little bit.

[Please note, I am travelling August 23 and 24, so it may take a while for me to respond to any comments.  Please be patient -- I do try to respond to any comments I receive.]

14 thoughts on “Luck and Competitive Balance, Part 1: The Post-Season

  1. Good stuff, Larry. But now you guys have to change the name of your site. :wink:

  2. Larry@IIATMS

    Steve, thanks, and very funny!  Actually, if luck is a big factor in baseball success, this makes having money more important.  Let's say that money accounts for 20% of wins.  But if luck accounts for 50% of wins (and I don't know that it does), then money would account for 40% of wins not accounted for by luck, 40% of the wins within the power of an organization (at least, a rich organization) to control.

     

    Brien, I want to dive deeper into the advanced statistics in a later post.  Sure, it's possible that Mo is SO good that he can induce a .230 BABIP in the post-season. This is why I compared regular season to post-season numbers; the regular season is based on a larger sample size.  Mo's regular-season BABIP against already indicates extreme skill.  The difference between Mo's outstanding regular-season numbers and his other-worldly post-season numbers is PROBABLY a matter of luck.

     

    You're right, it's possible that someone is SO good that he produces numbers that LOOK lucky.  I'll address this possibility in later posts.

  3. If you didn't see the New York Times interactive thing on Rivera, go check it out, especially the plot that tracks all of his pitches from last year. It's unbelievable, but there's almost no pitches put in the middle of the plate. ~95% of Mo's pitches on on or around the corner. Hard to make good contact on that if you're a hitter.

  4. Mark Smith

    Well put. I can't remember who said it, I think Peter Hjort at Capitol Avenue Club (at least that's where I first heard it), but the playoffs are a weighted crapshoot, much as you described. And I have slowly been coming toward your conclusion that the playoffs don't mean much in terms of competitive balance. However, one question still bothers me. If so many NBA teams haven't won and so many MLB teams have won their championships, how do you explain this when the NBA has so many other possible champions in their playoffs?

  5. Jason@IIATMS

    Mark: I think it's different in the NBA because if a team has two uber stars, that's 40% of the playing roster.  The MLB dynamic is so much different with 8 starters including a rotating starting pitcher. What if Michael Jordan could only play once every few games?

  6. The NBA is a league that will always be given to producing dynasties, even with perfect economic balance, for the reasons Jason states.  As Bill Simmons once said, it's a very easy league to "figure out."

    But in MLB, the number of parts you need to put together to "build" a champion is greater and more fragile, so dynasties would be very rare in MLB if the economics were equal.  And Larry nails it.  Because of the crapshoot that is the playoffs, particularly the best-of-five first round (quoth Billy Beane: "My (expletive) doesn't work in the playoffs"), the competitive balance situation is not nearly as bad as the most fervent anti-Yankees crowd would have you believe, but it is also not remotely as fair as the most rosy-eyed of our fellow Pinstripers manage to convince themselves to believe.

    II(still)ATMS.  We root for the team that will always have the best chance to win under the current state of affairs, even if no one team can really push its odds above, say, 30%, at the start of any given year.  Accept it, or don't.

  7. Mark Smith

    That's fair. Playing the odds, it would appear as though the NBA should see more different champions, but I can see your argument.

  8. Good read Larry, but again I think you’re kind of overstating what counts as “luck.” For example, FIP has always underrated Mo, and that’s always been known, because it doesn’t account for the fact that he’s so good that he can hit the corners consistently with his cutter and induce weak contact. So, in a sense, it’s hard to tell the difference between luck and extreme skill at times.

  9. MikeM

    Apparently I got lucky too. My uncle’s best friend played in the minors for the Yankees so I was raised a fan although I grew up in Pittsburgh. My less fortunate best friend is a Pirates fan and I love to link him articles like this because anytime we have a long baseball conversation it inevitalby shifts to competetive balance and payroll. My thoughts on this article however is that that 17.6% more wins due to payroll is really under played here. Yes its a relatively small number but it seems like its a number you can count on year in and year out to help you make the playoffs unlike luck. In baseball you don’t need to win 20% more games than the next guy to make the playoffs, just one so starting the season with basically a percentage of games in hand over a team in your division really shifts things more in the team with the significantly higher pay roll’s favor. In my opinion based on this research if the numbers hold true its like being 10 or 12 games back from a team on opening day. That kind of handicap just seems damning to me. I don’t have any stats to back this up but gut feeling should be more than enough; 95 vs say 83 wins makes all the difference in making the playoffs and it seems those extra 12 wins can be attributed to payrolls. The actual quantity of wins attributed to payroll isn’t all that eyepopping but in context that doesn’t make them any less valuable.

  10. Moose

    Talking about “luck” sounds much more ridiculous than it should, even with these compelling numbers.  Let’s remember what we’re talking about.  Injuries are mostly luck (with respect to timing at least), weather (which affects pitching rotations big time), and all the obvious in-game factors.  When you decompose luck, it doesn’t seem so crazy after all.
     
    My question is, “how much does luck affect a typical regular season game?”

  11. Larry@IIATMS

    Ty, I don't think you're reading me carefully enough.  I've been arguing here all year that the Yanks' money provides them with a substantial advantage.  The 17.6% figure is not mine, but it comes from some very smart people and it has to be addressed.  Hold on and wait for parts 2, 3 … well, however many parts I write in this series.  In the meantime, consider how the 17.6% figure might be affected by teams like the Cubs and Mets.

     

    Mark, take a look at http://dberri.wordpress.com/2006/11/20/the-short-….  The argument is that the NBA has a built-in problem with competitive balance, because the league will always be dominated by a few players who are much better than league average.  I'm not endorsing the argument, but it IS very interesting.

     

    Matthew, great post.  I like people who search for the middle path.

     

    Moose, I plan to address luck in the regular season in my next post in this series.  (or maybe the one after that)

     

    MikeM, we very much need to unpack that 17.6% statistic.  I suspect that it is highly misleading, and is based heavily on a too-small sample size, but I have more analysis to do.  My suspicion is that 17.6% means that the Yankees and the Red Sox don't win every World Series, and that the Cubs and Mets are inexplicable underachievers.  I need to do the analysis to confirm, but my guess is that the effect of payroll on team success can be described in a more complete way than by a single percentage figure.

     

  12. Ty

    NYY
    14

    ATL
    11

    BOS
    9

    STL
    8

    CLE
    7

    LAD
    6

    HOU
    6

    LAA
    6

    OAK
    5

    MIN
    5

    SEA
    4

    SDP
    4

    SFG
    4

    CHC
    4

    ARI
    4

    COL
    3

    TEX
    3

    NYM
    3

    CHW
    3

    PHL
    3

    BAL
    2

    FLA
    2

    CIN
    1

    MIL
    1

    TAM
    1

    DET
    1

    WAS
    0

    KCR
    0

    PIT
    0

    TOR
    0

    And your arguement is that payroll is a relatively small factor

    (This arguement is even more ridiculous seeing as boston and NYY are in the same division

  13. TJ Milo

    Luck was integral in the Yankees World Series win last year. Remember when Mauer ripped a ball down the left field foul line in extra innings in game 1 of the ALDS. The ump ruled it a foul ball when it was in fact fair.
    In the first 2 games of the ALCS, the Angels had 5 errors. And then they committed 2 additional GIGANTIC throwing errors in the bottom of the 8th at Yankee stadium. Going into the inning the Yanks were up 3-2. I think by the end of the inning the Yanks were up 5-2. If the Yanks had been up 3-2 against the Angels in the top of the 9th of game 6, I give the Angels a 50/50 chance of tying the game.

  14. Frank

    I think the whole advantage of money is to make the playoffs. That’s where the Yanks and Sox have a major competetive advantage. Only by making the playoffs consistently can you have a chance of winning it all.

Comments are closed.