Category: Sports

How to Rank Fantasy Baseball Players for Your League Stats

If you play in fantasy baseball dynasty league, you have no doubt raked through countless projections and rankings, attempting to forecast the year to come and break-out players.

There is no shortage of useful information out there, but one of the problems with pre-season fantasy baseball rankings is that they are catch-all’s, intended to appease the masses. And that makes sense, but if you really need insight into the value of a player in your league you need to only be scoring players based on the categories you carry.

Below I’ll show you a fool-proof method for creating solid rankings based on your specific league stats. I mostly used Excel for this exercise, but it can easily be done in R as well.

Step 1 – Get You Some Data

There’s no point in re-inventing the wheel here. I downloaded 2018 projections from fantasypros.com. Their Zeile Consensus Projections are solid and built from a number of sources. I’ve been told FanGraphs does an excellent job with their projections, so those are worth a look too, but the FantasyPros data had complete games easily accessible and that is a category I needed, so there you have it.

Step 2 – Clean it Up

In this step, I simply deleted the categories in each spreadsheet (hitting and pitching data are separated…for now) that I did not want in my way. I also created a calculated field for K/BB using the K and BB columns, since that is a category in my league.

Step 3 – The Super Secret Formula

So, in the end, I ultimately need to come up with value for how a player is contributing in each category and add those values up to give the player an overall score relative to how everyone else in the league scores.

To do this, we need to standardize the data so that each category uses roughly the same scoring range. This can be done by calculating the Z-score for every player in each category. Don’t worry too much if you’re not familiar with Z-Score — it’s basically a way to tell how far away from the average (mean) a player is in each category. But instead of literally using the numbers in the categories, those numbers are standardized.

Example:

Let’s say the mean number of hits and batting average are 150 and .270, respectively. If a batter is projected to get 170 hits and bat .310, we wouldn’t want to say he is 20 hits better and add that to his .040 better batting average for a score of 20.040. That would give so much weight to hits and batting average would be negligible, even if you batted an absurd .400 for the season.

This is where the standardization comes in. We can give each category an even scoring system if instead we say the faux player above is, say, 1 standard deviation above the mean for batting average and 1.4 above the mean for hits. So how do we calculate this?

It’s actually quite easy. The most tedious part is actually adding the scores at the end. First, you will need to get the average and overall standard deviation for each category in order to make this work.

=AVERAGE(data range here)

=STDEVP(data range here)

I place these formulas below the row of players for each category. It looks something like this:

Now that we have an average and standard deviation for each category, we can calculate Z-scores. In Excel, there is a formula for this called “=STANDARDIZE.”

I created a new column to the right of my dataset and tallied up each players Z-score for each category to give them an overall score. The STANDARDIZE function in Excel accepts a few arguments that you will need to supply. First, you need to point to the players’ stat for a given category, followed by the average stat for the category and the standard deviation for that category (which we calculated earlier). Doing that for each category will look something like this:

=STANDARDIZE(E2,$E$303,$E$304)+STANDARDIZE(F2,$F$303,$F$304)+STANDARDIZE(G2,$G$303,$G$304)+STANDARDIZE(J2,$J$303,$J$304)+STANDARDIZE(L2,$L$303,$L$304)+STANDARDIZE(M2,$M$303,$M$304)+STANDARDIZE(N2,$N$303,$N$304)

This score basically says that if these projections hold true, then this is how valuable each player is relative to each other for your league. Some of the scores may surprise you, but remember that it is aligned with your league and does not give weight to irrelevant categories, regardless of how big of a name a player might be in reality.

If you’re good with the above, that’s fine. But I made one more adjustment to account for number of at-bat’s and innings pitched. In other words, I wanted to give more value to a player that batted .295 over 500 at-bat’s versus a player who maybe batter .300, but for only 200 at-bats.

In order to do this, I created a new column for any stat that was an average or a ratio (BA, OBP, ERA, WHIP, K/BB, etc.) and basically created a new metric that I would use instead. For each of these, I took the player stat, subtracted it from the average for that stat and then multiplied by the number of at-bats or innings pitched, depending.

ERA, for example, would now be:

(player ERA - league average ERA) * Innings Pitched

I then used that variable instead of the ERA variable to calculate my Z-score.

Step 4 – Merging the Data into one Sheet

If you harken back to the beginning of this post, recall that there are still two spreadsheets at this point — one for hitters and one for throwers. We need to bind these together and sort by the new score we created in order to get our rankings!

Here is my sheet after combining all of the columns together. I also added a new column called “Type” that indicates whether a player is a hitter or pitcher, because I knew I would need it later on for a separate project.

Step 5 – A Dose of Reality

I personally think this is a much better approach than looking at generic rankings that are spewed out annually. With that said, it has it’s flaws.

For some reason, pitchers seemed to be over-valued in my scoring and after some analysis I found that pitchers who were over-indexing in complete games were getting way to high of scores in that category. I tried a number of things to dilute this but ultimately landed on subtracting the number of CGs from each players overall score (most players had zero, of course) and somehow it seemed to work and I feel pretty good about what I’ve produced.

You may run into a similar scenario and may need to make modifications to your scores using trial and error. Hope that’s not the case and you end up with something great!

Scraping Sports Stats Using R (Part 2)

In Part 1 of this blog post I show how you can scrape tables of sports data from websites and store that data in a data frame for data analysis (have I said ‘data’ enough times yet?).

Whenever you are automating data collection in this sort of way, you always want to get a “health check” on your new table to make sure nothing went awry. There are countless things that can go wrong from missing data to web pages timing out or blocking you from collecting data — and it is critical we understand whether any of this has happened before moving on to an analysis phase. You might call this a “data cleaning” phase to get you into position to analyze.

With the head function you can quickly get a glimpse of what your variable names look like, along with a few observations. Let’s take a look:

head(fb_main)
            Date NU rank     Opponent Site Outcome Score <U+00A0>
1 Sept. 17, 1960             #4 Texas Away     Win 14-13  Details
2 Sept. 24, 1960     #12    Minnesota Home    Loss 26-14  Details
3   Oct. 1, 1960           Iowa State Home    Loss  10-7  Details
4   Oct. 8, 1960         Kansas State Home     Win  17-7  Details
5  Oct. 15, 1960                 Army Home     Win  14-9  Details
6  Oct. 22, 1960             Colorado Away    Loss  19-6  Details

Immediately a few concerns jump out at me. First, there is a column where all of the values contain the word “Details.” On the original site I drew this information from, this column linked out to details for each game. I do not need this column for any analysis, so I will remove it. There are numerous ways to do this. Since I know it is my seventh column, I’ll just do it this way. If you’re unsure about how to tackle this (football pun), then you may want to save the table as a different name and keep the original.

fb_main <- fb_main[,-7]
head(fb_main)
            Date NU rank     Opponent Site Outcome Score
1 Sept. 17, 1960             #4 Texas Away     Win 14-13
2 Sept. 24, 1960     #12    Minnesota Home    Loss 26-14
3   Oct. 1, 1960           Iowa State Home    Loss  10-7
4   Oct. 8, 1960         Kansas State Home     Win  17-7
5  Oct. 15, 1960                 Army Home     Win  14-9
6  Oct. 22, 1960             Colorado Away    Loss  19-6

Much better. But it sure would be nice if the score was split into two columns in case I wanted to sum or average any of the scores during analysis. One variable can easily be split into two variables with a convenient function called separate, which is part of the tidyr package. It would look like this:

fb_main <- separate(fb_main, Score, into = c("Win Score", "Lose Score"), sep = "-")

But not so fast my friend — this really isn’t very helpful at all. We want the Husker scores in one column and the opponent score in another column, not a mix. This creates complexity to our code, but we can still accomplish it in a few easy steps. You will need to use dplyr, so make sure that is activated in your library.

library(tidyr)
fb_main$NUScore <- as.numeric(ifelse(fb_main$Outcome=="Win", fb_main$`Winner Score`, fb_main$`Loser Score`))
fb_main$OppScore <- as.numeric(ifelse(fb_main$Outcome=="Win", fb_main$`Loser Score`, fb_main$`Winner Score`))
fb_main <- fb_main[, -c(6:7)]

The first two lines above create the new columns we want using an ifelse function. And I saved myself some time by making the columns numeric. The ifelse statement has three arguments: condition, value of new row if condition is true, value of new row if condition is false. The last line is simply deleting the Winner Score and Loser Score variables, which we no longer need. Now look at the data:

head(fb_main3)
            Date NU rank     Opponent Site Outcome NUScore OppScore
1 Sept. 17, 1960             #4 Texas Away     Win      14       13
2 Sept. 24, 1960     #12    Minnesota Home    Loss      14       26
3   Oct. 1, 1960           Iowa State Home    Loss       7       10
4   Oct. 8, 1960         Kansas State Home     Win      17        7
5  Oct. 15, 1960                 Army Home     Win      14        9
6  Oct. 22, 1960             Colorado Away    Loss       6       19

Some other observations I have made about this data is that the Opponent variable contains both the opponent name and their ranking (this could create difficulties down the road), some values are missing, and the rankings contain a hash fragment (#). These are all worth tidying up before analysis, but I’ll stop there since the above should provide enough direction to complete those tasks.

One final consideration to make once you have collected all of this data is where to store it. There are many options and each of them have different methods in R. But here are a few to consider so that you do not need to re-gather the data each time you need it for analysis:

  • Save as a data frame in R
  • Use write.csv to save it on your computer as a .csv file (a similar function exists for Excel)
  • Send to a local or cloud-based database
  • Upload to data.world

Here is the final data, uploaded to data.world: https://data.world/jeffgswanson/husker-football-game-results

Enjoy.

Scraping Sports Stats Using R (Part 1)

You can scrape sports data from Web pages and store them in your own data frame/table for future analysis using handy readHTMLTable and lapply packages in R. My code is below.

fb_urls <- paste0('http://dataomaha.com/huskers/history/seasons/', 1960:2017)

fb_urls <- unlist(fb_urls)

fb_main <- lapply(fb_urls, function(x){readHTMLTable(getURL(x), stringsAsFactors=F)[[1]]})

fb_main <- rbindlist(fb_main)

fb_main

In this two-part post, I’ll show you how to use R to scrape tables from multiple Web pages to use for your own analysis. There are a number of steps involved here and, depending on which page(s) you’re trying to get data from, it can get complicated, but thankfully R has some brilliant packages like readHTMLTable to do the heavy lifting.

First, let’s take a look at the web pages I’ll be scraping. I want to collect game summary data for Nebraska Cornhusker football games (Go Big Red), which I found here: http://dataomaha.com/huskers/history/seasons/1997.

In this example, there is only one table. If your page has multiple tables, you may need to modify the code, which you can do by specifying within the readHTMLTable function (see last argument ‘[1]‘). If you have further questions on this, leave a comment or search for documentation on this function.

Below, the first step I took in forming a script is to build a list of URLs I wanted to scrape. If you are only extracting table data from one page, things are much simpler. You can just use readHTMLTable or htmltab and use the URL as an argument in the function. In my case, I wanted to cycle through 57 different pages. You could create a list in Excel quite easily, but just as quickly, you can also use the paste0 function, which is loaded into R by default, to automatically create a list.

fb_urls <- paste0('http://dataomaha.com/huskers/history/seasons/', 1960:2017)

Basically, this function is saying to take the base URL (first argument) and paste a number to the end until the list is complete. In this case, I am pasting 1960-2017, since I know the URLs I am scraping are constructed this way. Those are then saved as fb_urls.

The next step is to take what we created and basically get it into the right format. We can do this using the unlist function.

fb_urls <- unlist(fb_urls)

Now that we have a list of Web pages to scrape, we need to write a function with information on what we want to do on each page. Here, we are using a terrifically simple function name lapply which is designed to apply a function through a list. The first argument is simply the list we want to apply the function to and next we will use a Web scraping function in R called readHTMLTable to pull back the first table it finds on each page and save it in an list we will call fb_main.

fb_main <- lapply(fb_urls, function(x){readHTMLTable(getURL(x), stringsAsFactors=F)[[1]]})

The result is a list of 57 separate data frames. But I just need one large data frame with all of the information. So I’ll merge everything into one data frame using rbindlist.

fb_main <- rbindlist(fb_main)

Done! We always want to double check our work, which we can do by simply calling the new data frame.

fb_main

In due time, I’ll follow up with Part 2, which will focus on reviewing and cleaning the returned data, as well as options for storing it in an accessible place for future data analysis.