Curtis Granderson & The Vagaries of Defense Sabremetrics

The metric Berthiaume uses is Defensive Runs Saved (DRS), but he could’ve made the same argument with Ultimate Zone Rating (UZR), Relative Zone Rating (RZR), Range Factor (RF), or Fielding Runs Above Replacement (fRAR).  Among centerfielders, Granderson is at or near the bottom in all the above metrics in 2011.  However, this is the first time in three years the metrics have been anywhere close to united in their evaluation.  In 2009, Granderson’s last season as a Tiger, he ranked 2nd in Defensive Runs Saved (+14), but middle-of-the-pack in all the other metrics.  In his first year in New York, the evaluations were even more diverse:

DRS: +4 (12th), UZR:+6.4 (4th), RZR: .909 (12th), RF: 2.58 (15th), fRAR: -1 (11th)

It’s worth pointing out here that UZR, which most optimistically rates Granderson’s 2010 defense, is factored into FanGraphs WAR, while his more humble score in fRAR effects Baseball Reference’s WAR calculations.  Thus, FanGraph’s WAR for Granderson is a full win better than Baseball Reference’s (3.6 to 2.5).  The picture grows even more complicated when you go back to 2008, which FanGraphs suggests is Granderson’s worst defensive season ever (-11.1 UZR), while Baseball Reference thinks it’s his best (+6 fRAR).  Meanwhile, another popular metric, Total Zone (TZL), puts it somewhere between (+1.3).

Seriously?!?

The creators of all the above statistics would be united in this chorus: “Beware of sample size.”  The stats which Berthiaume cites take into consideration only four months of data, while many find comprehensive defensive metrics to be unreliable even for entire 162-game seasons.  John Dewan, creator of the Fielding Bible and his own Plus/Minus system for evaluating fielding, doesn’t use anything less than three-season samples (in the last Fielding Bible, which rated the ’06-’08 campaigns, Granderson ranked a respectable eighth among MLB center-fielders at +20).  But, if we take Granderson’s data from 2008 to the present, we only get a slightly more even picture of his defensive value:

DRS: -2 (9th), UZR: -14.0 (11th), RZR: .916 (10th), fRAR: +2 (8th), TZL: 4.2 (3rd)

TZL, arguably the most evolved metric (or, perhaps merely the hottest because it is the newest, sabremetricians being as fashion-conscious as the rest of us), remains a serious outlier.  But it is also the only of these metrics which does not account for the 2011 season, yet.

The sample size problem is further exasperated by the fact that Yankees pitchers aren’t creating flyballs at a particularly high rate (which is, of course, a good thing).  New York’s 35.3% flyball rate ranks 24th in the majors.  Because many defensive metrics assume a relatively consistent rate of opportunities, players that get more opportunities over shortened intervals can see their stats inflated, or vice versa.  It seems relevant to point out that the #1 team in the AL in flyball rate (Boston) is also the #1 team in outfield UZR.  The same is true in the NL (Arizona).

Unfortunately, sample sizes aren’t even the last of our problems.  Berthiaume points out that Brett Gardner and Nick Swisher are both having excellent defensive seasons according to DRS (as well as dRAR and UZR) playing on either side of Granderson.  What he doesn’t acknowledge is that there is considerable evidence that good outfielders can effect each other’s statistical performances adversely.  It’s pretty easy to understand how.  If you have two rangy outfielders like Granderson and Gardner, they are occasionally going to be chasing the same balls in the outfield gaps.  Both cannot get credit for the putout, even if both could have made the catch.  Moreover, acknowledging this, the Yankees will more frequently position their outfielders against the logic of the “typical zone” which is used to calculate many of these metrics.  For instance, shading Granderson strongly to right-field, which means 1.) Gardner covers portions of left-center which the center-fielder might normally take responsibility for and 2.) Swisher has more luxury to guard the line and/or play shallower or deeper than he otherwise might, thus allowing him to make plays he would not have if, say, Alexis Rios were his centerfielder.  This could also limit Granderson’s opportunities compared to average positioning, making it appear as though his range is shrinking, even if he is merely leaving more plays to his teammates.

The converse can also be true.  In 2007, for instance, Granderson posted career highs in DRS and UZR, trailing only Coco Crisp and Andruw Jones among MLB center-fielders.  Certainly, it is possible that 26-year-old Grandy was a step or two faster than 30-year-old Grandy.  But, he was also flanked on both sides by rickety future DHs like Magglio Ordonez, Gary Sheffield, Craig Monroe, and Marcus Thames.  One can probably assume that Granderson didn’t have to worry about “calling them off” quite as often as he does the fleet-footed Gardner.

Although most defensive metrics are “park-adjusted,” weighted to account for environments, there is considerable circumstantial evidence that the park factors used for these calculations are still seriously imperfect, especially for outfielders.  The most obvious problem is Fenway Park’s strange dimensions.  Consider this: since 2008, J. D. Drew ranks 2nd among outfielders in UZR, suggesting he’s better than Ichiro, but close to the bottom in DRS, suggesting he’s as bad as Corey Hart.  Jacoby Ellsbury has also suffered from seriously fluctuating defensive performances, while Carl Crawford, in his first year with the Red Sox has gone from outfield divinity (the top UZR in all of baseball, at any position, from ’08 to ’10) to Jason Bay -level mediocrity.

We can’t glean much from three years of data for Yankee Stadium, the dimensions of which would not seem to be all that extraordinary, but it’s worth pointing out that since 2009, Yankees outfielders have frequently landed at the extremities of the fielding scales.  Gardner’s 36.8 UZR/150 is double that of the next most productive left-fielder over that time (Crawford).  Swisher’s defensive performance went through the roof upon arriving in New York, while Melky Cabrera has been unable to achieve anything remotely resembling his defensive value with the Yankees since moving to other ballparks.

All of this leads me to believe that outfield metrics must be taken with a grain of salt, especially for single seasons, and especially when those seasonal fluctuations aren’t confirmed by scouting reports.  Most evidence, including such observations, would seem to suggest Curtis Granderson is neither a superlative center-fielder, nor a particularly weak one.  To return to Berthaiume’s argument, it strikes me that the evidence suggesting Granderson does his team a disservice defensively is circumspect, while there is no denying he has been outstanding offensively, and from a position not renowned for offense.  Granderson leads all AL centerfielders in wOBA and wRC+, as well as the traditional HR and RBI.  Getting that type of production from an “up the middle” player is increasingly rare in this new era of focus on pitching and defense.  Granderson would need to give up a whole lot of runs with his glove in order to even begin to offset what he does with his bat and his wheels.

UPDATE: Shortly after posting I went for a swim and listened to a Baseball Today interview with Baseball Reference founder, Sean Forman, who, among other things, compares the methods used to calculate WAR on his site to those used by FanGraphs.  I was, apparently, incorrect in suggesting that BB-Ref. uses fRAR for WAR (although you can find it on their site).  They now use a version of Total Zone (although a different version of Total Zone from that provided by FanGraphs).  Ergh!  Anyway, it doesn’t change the gist of the post in any way, but I wanted to make the correction and point out the interview for those who are curious.

Matt teaches at The University of Alabama. Roll Tide. He specializes in American Literature and Rhetorical Economics. Fate chose for him the peculiar perdition of rooting for the Chicago Cubs and the Los Angeles Clippers.

11 thoughts on “Curtis Granderson & The Vagaries of Defense Sabremetrics

  1. williamjtasker

    Holy crap! I just wrote about this today too. And posed the same questions. I love when that happens and it happens a lot here. We both agree that the range and usage of Gardner definitely impedes Granderson's defensive metrics. Right on.

  2. Dave

    All defensive metrics need to be taken with a grain of salt. And that goes for statistics that rely on them like WAR.

    • On the one hand, I agree with you. Although I find WAR very interesting and useful in certain situations, I think it's unrealistic to believe that any single measure can consistently and accurately measure the worth of a baseball player. I think it's very unrealistic to say that Jacoby Ellsbury is a better MVP candidate than Curtis Granderson just because he is 0.3 better in WAR (that difference being entirely due to a suspect evaluation of both their defensive contributions).

      I do think, however, that defensive metrics are worth using in many situations, especially for infielders, so long as you have large data sets and account for margins of error.

      • Hank

        Completely agree the OF metrics are pretty sketchy…. if you take UZR there just isn't enough zones to accurately measure over a small sample size (i.e at least multiple years needed). While UZR assigns a certain probability of an out to a given zone, that does not mean every ball hit in the zone has an equal probability of an out. This only works over large samples where an OF will see a fairly wide distribution of balls in any given zone) And of course since it ignores the starting position of the fielder and has pretty coarse control over flyball type, you can't assume these things even out over the time period people usually use these models for..

        And the park adjustments need some significant work… Carl Crawford is not a 22.5 UZR/150 at the Trop and a 7.5 UZR/150 away from it (and folks wonder why his defensive #'s appear to be "down"… maybe without a half season of the Trop baked into them they aren't that far off). This is over a 6 year sample so it's hard to chalk it up to noise/luck/variation. This seems like either input bias or a park factor that has not been accounted for. He's apparently an above average arm at home and a below average arm on the road? (I'm going to go with the latter)

        And don't even get me started with some of the non-range components like armR (this is an even smaller sample size and noisy as base advancement is highly contextually related) and errR (subjective).

  3. Andrew518

    Statistics and comparison is a fun part of baseball but I've said it once, I'll say it again, and I'm sure I'll say it again:

    Defensive Sabermetrics mean absolutely nothing.

    Call me old school or out of date, but you can't quantify everything. Somethings are and always be subjective. The more complicated a formula required to create a satistic the less it means.

    "Old" statistics, AVG, RBI's etc don't completely tell the story of a player, neither do WAR or UZR.

    • BrienJackson

      "Call me old school or out of date, but you can't quantify everything."

      Of course you can. There's 162 games in a baseball season. That's a lot, and it's even more over 5-10 years. If something has an effect on a baseball game, given enough games it has to be quantifiable somehow. It would just be a matter of figuring out how to do it.

      Defensive metrics are fine, they're just not stand alones. You need the eyeball test to help check apparent outliers in the data. On the other hand, you need the numbers to make sure your eyes aren't lying to you. Defense is one of the areas where scouting and statistics interact in the best way.

  4. ""Old" statistics, AVG, RBI's etc don't completely tell the story of a player, neither do WAR or UZR."

    Right on.

    There is no one, sure-fire, foolproof, bullet-resistant stat that accurately captures everything. They keep looking for it and honing those stats in existence. But total reliance on any one stat is a fool's errand.

    And no matter what any metric says, some will flatly declare: "I only believe what my eyes see!" and that's that. Me, I put Granderson In the top half of the CF's today. not the worst, not the best. Better than average. And that's just fine for me.

  5. Frank S.

    Although Grandy is the second best CF on his own team, he's the best defensive every day CF the Yankees have had since maybe as far back as Rickey Henderson (who was an underrated CF who switched positions when he signed with the Yanks). That's what my eyes, and not the numbers tell me. Of course, that's not saying much, since Bernie and Damon were, from my perspective, below-average CFs, Melky had a good arm but not much else, and Roberto Kelly wasn't very good defensively at all. Before Rickey, Rupert Jones was pretty darned good, I thought.

  6. chad

    having a defensive"stat" that shows swisher being a better outfielder than granderson just goes to show how you can't trust them outright.

  7. JayJay

    Thsi all constitutes a pretty serious and legitimate kink in Gardner's overall value. A lot of his value has been in his perceived defensive value. He's good, probbaly very good, but probbaly not the top-10 (or whatever) in the AL player his defensive numbers would have some of us believe

    • Agreed. Gardner has performed consistently well in all defensive valuations (including the eye test), but his numbers are probably slightly inflated this season, as are Jacoby Ellsbury's, Carlos Lee's, Matt Holliday's, etc. I'm very suspcious of all the OF on the WAR leaderboards who are getting big spikes from UZR (or vice versa).

Comments are closed.