PDA

View Full Version : Criminals of WAR



YankeeFan28
11-16-2009, 10:39 AM
Iím not going to single anyone out, since weíre all guilty of abusing FanGraphsí Wins Above Replacement metric. But Iíve been seeing cases pop up where itís getting out of hand. So Iíve set up a few guidelines for how to go about using WAR responsibly. Do not break these rules, or I may call you out.

1. Do not exclude baserunning from a position playerís WAR. Iím sure David Appelman will include baserunning in the next edition of WAR, since itís so easy to calculate, but the numbers are already out there, so please take the time to go to BP, B-Ref, or BJOL to look up the numbers and tack them on.

2. Do not place undue trust in WAR for catchers. How much of a catcherís value do you think is in his defense? Iíll give you a hint: itís a lot. FanGraphs has unfortunately yet to give an effort to quantifying this vital aspect of the game, other than with the positional adjustment. In fact, catchers should possibly be considered a separate group of players with a separate replacement level and therefore be treated as different from all other position players.

3. Do not place undue trust in WAR for pitchers. First off, pitcher defense and hitting arenít included. This should be righted ASAP. Then there are the more nuanced issues like how leverage is accounted for and the conversion of FIP to runs. Personally, Iíd trust the calculations of David Gasskoís pitching runs created or StatCornerís WAR well before I would FanGraphsí WAR.

4. Do not cite WAR as a measure of skill. WAR measures production. FanGraphs has a lot more granular data if youíre trying to assess skill. And if youíre going to try to make a projection of WAR, regress each component individually. Also, players with negative WAR still may have value if they excel at a certain skill that can be leveraged.

5. Do not use the linear conversion of WAR to salary to determine what a team should be willing to pay a free agent. Every team has a different scale, depending on that teamís market and where the team is on the win curve. Few teams should pay $5 million for a single win.http://www.hardballtimes.com/main/blog_article/criminals-of-war/

Toirtap
11-16-2009, 02:41 PM
My opinion on his points (as if you cared):

1. True enough, but not a huge deal in most cases.
2. True enough, but a tad melodramatic. Plus as has been discussed in other threads here, people like Rally, Jin, and C1B have put together some solid estimates.
3. Very true, and the most salient point IMO (although Gassko's PRC is a zero-level method and isn't really comparable to WAR). There's a perfectly good case for basing WAR on FIP, but there's also a perfectly good case for using actual runs allowed, component ERA, etc. The ubiquitous nature of Fangraphs WAR in certain parts of the sabermetric community risks marginalizing other points of view on pitcher evaluation.
4. True.
5. Yes, but average value is still an important guidepost.

weebs
11-16-2009, 02:47 PM
I've found The Hardball Times to be very unprofessional and scathing when it comes to stats they don't carry or didn't develop.

Toirtap
11-16-2009, 03:16 PM
I have never gotten that impression at all. The Hardball Times is a class operation. And I don't think there was anything remotely scathing about that post, with the possible exception of the title which is a pretty obvious pun.

I could name a certain blogger at Fangraphs who has a much nastier tone than anyone at THT has ever come close to...

weebs
11-16-2009, 03:18 PM
Goes back to this one too.

http://www.hardballtimes.com/main/blog_article/war-vs.-wins/

From the comments you can tell that there's a bit of a war going on between THT and Fangraphs.

Toirtap
11-16-2009, 04:12 PM
Again, I don't see anything unprofessional or scathing about that post. One can certainly reasonably agree or disagree with Steiner's point, but I don't see how it's unprofessional. I think Appelman's response was understandable as it was his site's metric being critiqued, but I just don't see anything unfair there.

Peer review and critical appraisals of metrics is an important element of any quantitative field, and sabermetrics should be no exception. It should be done respectfully, of course, with open-mindedness to alternative ways of thinking about the problem. If I was looking for examples of criticisms that weren't in that spirit, there would be a lot of other places I'd look first.

weebs
11-16-2009, 04:19 PM
That's fair. It's just a personal opinion of tact that I see from THT.

C1Bman88
11-16-2009, 04:25 PM
1. "Baserunning" is implemented in FanGraphs' WAR, but it's limited- all they use are SB and CS. Jeremy is correct in that BP carries a great baserunning statistic in EqBRR, but I wouldn't bother mentioning BJOL or B-Ref. BJOL requires a subscription and Bill hasn't made his methodology clear at all (at least to the best of my knowledge), so I don't really trust it. With B-Ref, you have to calculate the numbers on your own...and not everyone wants to do that.

2. FanGraphs doesn't implement catcher defense because they're not confident in the basic estimates we have. I can't really fault them for that, but the very least they could do is what JinAZ does. They work with BIS; the data is most definitely there.

3. I think Jeremy's wrong about "pitcher defense" and "pitcher hitting." A pitching staff will be involved in about 6% of a team's total plays. There are very little plays in which a pitcher does anything, to the point where measuring it is practically useless IMO. But, there are a few comprehensive systems that quantify pitcher defense- PMR, Plus/Minus, and Context-Adjusted Defense. As for hitting, you'd have to make some sort of adjustment for the league. Obviously, NL pitchers get more AB than AL pitchers do. I think Jeremy's nitpicking here, but I do agree that incorporating leverage and using tRA, rather than FIP, would be ideal for determining a pitcher's "true" value.

4. Very true. WAR doesn't measure skill or ability; it measures value provided on the field relative to a replacement player.

5. Sure, but I see no problem with using it.

I'm surprised that Jeremy didn't comment on the concept of "replacement level," since there's no static definition. Personally, I'd rather FanGraphs use a baseline of "average" rather than 20 runs per 600 PA. Everyone knows what average stands for; not everyone will understand or agree with a replacement level.

Toirtap
11-16-2009, 04:38 PM
I wouldn't do away with WAR, but I agree with C1B that it would be great if they listed WAA as well. I always include both average and replacement baselines so the reader can make his own choice (of course, there are intermediate choices as well, plus multiple reasonable replacement levels, so the options are hardly comprehensive). Fangraphs does explicitly tell you how many runs the player was credited for relative to a replacement, so you can back it out, but of course you have to it by hand.

I was disappointed that Matt Klassen (whose work I enjoy) called RAA "just a toy stat, but a fun one" on Fangraphs last week (http://www.fangraphs.com/blogs/index.php/the-5-or-so-average-est-position-players-of-2009/) (in fairness, it's possible he was referring to the absolute value of RAA as the "toy"). I agree that it's a meaningful baseline.

Personally, I don't think you need a league adjustment for PA if you include pitcher hitting, since it is a value metric. NL pitchers have a larger impact (for better or worse) with the bat than do their AL counterparts. As long as you set it up so that the net contribution for all pitchers is zero, I'm fine with it.

poodski
11-17-2009, 01:06 PM
3. I think Jeremy's wrong about "pitcher defense" and "pitcher hitting." A pitching staff will be involved in about 6% of a team's total plays. There are very little plays in which a pitcher does anything, to the point where measuring it is practically useless IMO. But, there are a few comprehensive systems that quantify pitcher defense- PMR, Plus/Minus, and Context-Adjusted Defense. As for hitting, you'd have to make some sort of adjustment for the league. Obviously, NL pitchers get more AB than AL pitchers do. I think Jeremy's nitpicking here, but I do agree that incorporating leverage and using tRA, rather than FIP, would be ideal for determining a pitcher's "true" value.

I dont know. I mean this article http://www.fangraphs.com/blogs/index.php/spreads-in-pitcher-hitting-and-dh-hitting showed that hitting for pitchers can be fairly large. I mean the difference between the Cubs and Pirates is two win shares.

In comparison the difference between Zambrano and Ohlendorf can be two win shares. Just with the bat.

This little piece kinda sums it up:


This is what makes Zambrano so much more valuable in the NL Ė his 3.95 career FIP isnít ace quality, but with a .305 in 65 PAs as a pitcher, he contributed eight runs with the bat, nearly a win worth of production.

C1Bman88
11-17-2009, 02:17 PM
I dont know. I mean this article http://www.fangraphs.com/blogs/index.php/spreads-in-pitcher-hitting-and-dh-hitting showed that hitting for pitchers can be fairly large. I mean the difference between the Cubs and Pirates is two win shares.

In comparison the difference between Zambrano and Ohlendorf can be two win shares. Just with the bat.

This little piece kinda sums it up:

I think they do need to be careful with it though, since AL pitchers aren't going to see nearly as many PA as NL pitchers do. The problem I have with it is that it could very well underrate the value of good hitting pitchers in the AL that never get to hit. I personally think an adjustment of some sort would have to be made.

Toirtap
11-17-2009, 03:01 PM
If WAR is being used as a tool to estimate actual value contributed, then I fail to see why there should be an adjustment for AL pitchers. A skill not used in the game does not create runs or wins.

On the other hand, if you are using WAR to estimate what a player's value would be in a different context, then I agree that you would want to make an adjustment for the offensive contributions of AL pitchers in a NL context.