Introducing SIERA

Postby VoxOrion » Mon Feb 08, 2010 17:19:46

Do I have to say it like an acronym or can I call it "See-air-ah"? You know, for when I'm talking baseball with the chicks.
“There are no cool kids. Just people who have good self-esteem and people who blame those people for their own bad self-esteem. “

VoxOrion
Site Admin
 
Posts: 12963
Joined: Thu Dec 28, 2006 09:15:33
Location: HANLEY POTTER N TEH MAGICALASS LION

Postby MattS » Mon Feb 08, 2010 17:21:29

VoxOrion wrote:Do I have to say it like an acronym or can I call it "See-air-ah"? You know, for when I'm talking baseball with the chicks.


I've been pronouncing it See-air-ah, so I think you're good :)

MattS
There's Our Old Friend
There's Our Old Friend
 
Posts: 3580
Joined: Thu Dec 28, 2006 21:17:00

Postby Wheels Tupay » Mon Feb 08, 2010 17:31:21

Yea, when I clicked on this I was definitely thinking that you had a baby and that there was going to be pictures.
"That’s the Southwest Philly floater, man."
Now imagine that everything you ever imagined... is possible. - Hinkieology
EDP 2020

Wheels Tupay
Plays the Game the Right Way
Plays the Game the Right Way
 
Posts: 30614
Joined: Wed Dec 16, 2009 16:35:17
Location: Keepin' it Gritty.

Postby TenuredVulture » Mon Feb 08, 2010 17:45:45

MattS wrote:
bleh wrote:SIERA = 6.262 – 18.055*(SO/PA) + 11.292*(BB/PA) – 1.721*((GB-FB-PU)/PA) +10.169*((SO/PA)^2) – 7.069*(((GB-FB-PU)/PA)^2) + 9.561*(SO/PA)*((GB-FB-PU)/PA) – 4.027*(BB/PA)*((GB-FB-PU)/PA)

The beauty of it is its simplicity


I mean, it's something that you need to put into Excel or go to BP to get like everything else. How many people calculate xFIP in their head or by hand? The best thing about it, in reality, is that it gets rid of a lot of the nonsense simplification in other estimators. What good is an ERA estimator that misses badly for anything but reasonably average or somewhat above average pitchers? I think it's more useful to be able to look at elite pitchers and figure out how good they really might be.

It'll all be on the stats page very soon, so it's not like anyone will need to calculate by hand.


It sorta looks like a regression equation, but it really isn't, is it? Cause you've got interaction terms and squared terms and a bunch of other fun stuff.
Be Bold!

TenuredVulture
You've Got to Be Kidding Me!
You've Got to Be Kidding Me!
 
Posts: 53243
Joined: Thu Jan 04, 2007 00:16:10
Location: Magnolia, AR

Postby MattS » Mon Feb 08, 2010 17:49:39

TenuredVulture wrote:
MattS wrote:
bleh wrote:SIERA = 6.262 – 18.055*(SO/PA) + 11.292*(BB/PA) – 1.721*((GB-FB-PU)/PA) +10.169*((SO/PA)^2) – 7.069*(((GB-FB-PU)/PA)^2) + 9.561*(SO/PA)*((GB-FB-PU)/PA) – 4.027*(BB/PA)*((GB-FB-PU)/PA)

The beauty of it is its simplicity


I mean, it's something that you need to put into Excel or go to BP to get like everything else. How many people calculate xFIP in their head or by hand? The best thing about it, in reality, is that it gets rid of a lot of the nonsense simplification in other estimators. What good is an ERA estimator that misses badly for anything but reasonably average or somewhat above average pitchers? I think it's more useful to be able to look at elite pitchers and figure out how good they really might be.

It'll all be on the stats page very soon, so it's not like anyone will need to calculate by hand.


It sorta looks like a regression equation, but it really isn't, is it? Cause you've got interaction terms and squared terms and a bunch of other fun stuff.


Yes, it is a regression equation. The benefit of that is that you can actually test for these effects rather than guessing, as well as pick up any pitcher effects on BABIP (which do correlate a little bit with K%, BB%, and GB%, so it's not ignoring them like other metrics) while getting rid of the BABIP luck and defense effects.

MattS
There's Our Old Friend
There's Our Old Friend
 
Posts: 3580
Joined: Thu Dec 28, 2006 21:17:00

Postby Bakestar » Mon Feb 08, 2010 18:06:54

Wheels Tupay wrote:Yea, when I clicked on this I was definitely thinking that you had a baby and that there was going to be pictures.


I was kind of imagining him emceeing at a really awful strip club.
Foreskin stupid

Bakestar
BSG MVP
BSG MVP
 
Posts: 14709
Joined: Thu Dec 28, 2006 17:57:53
Location: Crane Jackson's Fountain Street Theatre

Postby TenuredVulture » Mon Feb 08, 2010 18:13:08

Conclusion: Getting strikeouts is very, very good.
Be Bold!

TenuredVulture
You've Got to Be Kidding Me!
You've Got to Be Kidding Me!
 
Posts: 53243
Joined: Thu Jan 04, 2007 00:16:10
Location: Magnolia, AR

Postby MattS » Mon Feb 08, 2010 18:19:07

TenuredVulture wrote:Conclusion: Getting strikeouts is very, very good.


Haha, yeah, that's a pretty important conclusion. I think it gives a little more credit to strikeouts than other estimators because strikeout pitchers tend to have low BABIPs but it's impossible to see through all the noise (i.e. great strikeout pitchers will be at .290-ish instead of .300, but that really means the window is .260-.320 instead of .270-.330 so it's tough to say). This picks up all that stuff since it's a regression on real live pitchers!

MattS
There's Our Old Friend
There's Our Old Friend
 
Posts: 3580
Joined: Thu Dec 28, 2006 21:17:00

Postby joe table » Mon Feb 08, 2010 18:48:23

I called Dan Shaughnessy and gave him the heads up about this, he'll be sending a goon squad to your office ASAP

joe table
Plays the Game the Right Way
Plays the Game the Right Way
 
Posts: 41100
Joined: Thu Jul 30, 2009 14:56:43

Postby CalvinBall » Mon Feb 08, 2010 23:18:50

I haven't had a math class since sophomore year of college. Looking at that equation makes my head spin.

CalvinBall
You've Got to Be Kidding Me!
You've Got to Be Kidding Me!
 
Posts: 64951
Joined: Mon Jul 30, 2007 15:30:02
Location: Pigslyvania

Postby uncle milt » Mon Feb 08, 2010 23:22:50

fungible

uncle milt
Dropped Anchor
Dropped Anchor
 
Posts: 6205
Joined: Sun Dec 31, 2006 15:54:36

Postby Woody » Mon Feb 08, 2010 23:24:14

has joboggi chimed in on this yet
you sure do seem to have a lot of time on your hands to be on this forum? Do you have a job? Are you a shut-in?

Woody
BSG MVP
BSG MVP
 
Posts: 52472
Joined: Thu Dec 28, 2006 17:56:45
Location: captain of the varsity slut team

Postby MattS » Mon Feb 08, 2010 23:25:36

CalvinBall wrote:I haven't had a math class since sophomore year of college. Looking at that equation makes my head spin.


you don't need to carry the ones yourself. you can plug it into excel or get it from the statistics reports once it's up (which should be shortly I'm told).

MattS
There's Our Old Friend
There's Our Old Friend
 
Posts: 3580
Joined: Thu Dec 28, 2006 21:17:00

Postby TenuredVulture » Mon Feb 08, 2010 23:41:43

CalvinBall wrote:I haven't had a math class since sophomore year of college. Looking at that equation makes my head spin.


Really? It's pretty basic. I mean, it's derived from running a regression which involves matrices, but result is really just multiplication, addition, subtraction, and division.
Be Bold!

TenuredVulture
You've Got to Be Kidding Me!
You've Got to Be Kidding Me!
 
Posts: 53243
Joined: Thu Jan 04, 2007 00:16:10
Location: Magnolia, AR

Postby CalvinBall » Mon Feb 08, 2010 23:44:15

i was being a bit sarcastic. just all the PU-GB-FB+B-SEXYIME stuff makes it crazy looking. and the constants are crazy that they are actually derived from something.

CalvinBall
You've Got to Be Kidding Me!
You've Got to Be Kidding Me!
 
Posts: 64951
Joined: Mon Jul 30, 2007 15:30:02
Location: Pigslyvania

Postby HillMD » Tue Feb 09, 2010 00:05:22

This is some good stuff. Someone could probably make some cash betting MLB using this.

HillMD
Dropped Anchor
Dropped Anchor
 
Posts: 6609
Joined: Fri Jun 26, 2009 20:18:10

Postby danrosz » Tue Feb 09, 2010 01:43:29

HillMD wrote:This is some good stuff. Someone could probably make some cash betting MLB using this.


Sadly, I was thinking the same thing. Time for MattS to put his powers to good use and start posting in the Aces Up thread.

danrosz
There's Our Old Friend
There's Our Old Friend
 
Posts: 3142
Joined: Mon Oct 08, 2007 15:04:57

Postby phorever » Tue Feb 09, 2010 10:20:41

MattS wrote:For 2009:

Pitcher ERA SIERA FIP xFIP
Halladay 2.79 3.09 3.06 3.05
Hamels 4.32 3.55 3.72 3.69
Blanton 4.05 3.92 4.45 4.07
Happ 2.93 4.37 4.33 4.49
Moyer 4.89 4.68 4.94 4.74
Lidge 7.21 4.20 5.45 4.76
Madson 3.26 3.18 3.23 3.25



lee? kendrick? bastardo?

also, the nature of the test of siera as a predictor was a little unclear.
my guess is that "using 2003-08 data to generate a formula and then testing it on 2009 pitchers, SIERA emerged as the best estimator of park-adjusted ERA in the following year, and..." means that (a) 2003-8 data was used in the regression to determine the right formula for combining the components of pitching to predict era; then (b) that formula was used to calculate siera for pitcher each season, including 2008 and 2009; the reliability/accuracy of siera as an era predictor relative to other measures was tested BOTH by seeing how well 2008 siera predicted 2009 era AND by seeing how well 2009 siera matched up with 2009 era. do i have that right?

finally, it looks like your discussion of lidge elsewhere in this thread suggests that one reason siera beats fip in the next season despite losing out in-season is that siera treats injuries as luck. an injury is likely to produce a consistent streak of hr/fb bad luck during one season that doesn't persist into the next. is that a fair interpretation?
phorever
There's Our Old Friend
There's Our Old Friend
 
Posts: 3785
Joined: Fri Jan 05, 2007 08:25:07
Location: the netherlands

Postby Buzhardt » Tue Feb 09, 2010 11:37:00

danrosz wrote:
HillMD wrote:This is some good stuff. Someone could probably make some cash betting MLB using this.


Sadly, I was thinking the same thing. Time for MattS to put his powers to good use and start posting in the Aces Up thread.


As long as Matt and Eric are not bent on world domination, I am OK with all this.

Hey did everybody hear about Cliff Lee's foot surgery yesterday?

If I can come up with an intelligent question or comment on all this, I will. Promise.

Buzhardt
There's Our Old Friend
There's Our Old Friend
 
Posts: 253
Joined: Mon Jan 25, 2010 12:46:28
Location: Fredericksburg, VA

Postby MattS » Tue Feb 09, 2010 11:59:52

phorever wrote:
MattS wrote:For 2009:

Pitcher ERA SIERA FIP xFIP
Halladay 2.79 3.09 3.06 3.05
Hamels 4.32 3.55 3.72 3.69
Blanton 4.05 3.92 4.45 4.07
Happ 2.93 4.37 4.33 4.49
Moyer 4.89 4.68 4.94 4.74
Lidge 7.21 4.20 5.45 4.76
Madson 3.26 3.18 3.23 3.25



lee? kendrick? bastardo?

also, the nature of the test of siera as a predictor was a little unclear.
my guess is that "using 2003-08 data to generate a formula and then testing it on 2009 pitchers, SIERA emerged as the best estimator of park-adjusted ERA in the following year, and..." means that (a) 2003-8 data was used in the regression to determine the right formula for combining the components of pitching to predict era; then (b) that formula was used to calculate siera for pitcher each season, including 2008 and 2009; the reliability/accuracy of siera as an era predictor relative to other measures was tested BOTH by seeing how well 2008 siera predicted 2009 era AND by seeing how well 2009 siera matched up with 2009 era. do i have that right?

finally, it looks like your discussion of lidge elsewhere in this thread suggests that one reason siera beats fip in the next season despite losing out in-season is that siera treats injuries as luck. an injury is likely to produce a consistent streak of hr/fb bad luck during one season that doesn't persist into the next. is that a fair interpretation?


Lee: 3.73 combined (4.07 CLE, 3.05 PHI)

Bastardo: 4.69 (but that's a pretty small sample size...but I guess a benefit of SIERA is that it will stabilize quickly like QERA)

Kendrick: 4.92, 5.30, 4.26 in 07, 08, 09 (though the sample size plus pitching in relief makes the 09 number not really all that great)

You have some of the tests we ran right, though we ran tons and it pretty much won every way. It was the best at predicting next year ERA in every year, and the best among estimators that treat HR/FB as luck in estimating same year ERA in every year, I think. It was pretty consistently ahead in all of the tests. The primary ones, though, consisted of predicting same-year ERA better than QERA and xFIP, and next-year ERA better than QERA, xFIP, FIP, and tRA. We also tested data using coefficients generated from 2003-08 to test prediction in 09 just to make sure there were no biases. And using slightly different coefficients that were generated without knowing 2009 happened (in fact, generated last year), the 2008 SIERAs predicted 2009 ERAs best too.

I wouldn't say that SIERA treats injuries as luck or anything, just that injuries can clearly make a pitcher pitch way behind his peripherals as I learned in my more novice days where I declared Freddy Garcia fine, heheh :) FIP happens to pick up on HR/FB luck and credit it to the pitcher when the reality is that HR/FB is probably correlated with pitching while injured, and it's just that so many people with high HR/FB drop out of the sample that we don't pick up this injury effect in testing.

MattS
There's Our Old Friend
There's Our Old Friend
 
Posts: 3580
Joined: Thu Dec 28, 2006 21:17:00

PreviousNext