yardstick is a package to estimate how well models are working using tidy data principals. The package webpage is https://topepo.github.io/yardstick/ for more information.
For example, suppose you create a classification model and predict a data set. You might have data that look like this:
library(yardstick)
library(dplyr)
head(two_class_example)## truth Class1 Class2 predicted
## 1 Class2 0.00359 0.996411 Class2
## 2 Class1 0.67862 0.321379 Class1
## 3 Class2 0.11089 0.889106 Class2
## 4 Class1 0.73516 0.264838 Class1
## 5 Class2 0.01624 0.983760 Class2
## 6 Class1 0.99928 0.000725 Class1
You can use a dplyr-like syntax to compute common performance characteristics of the model and get them back in a data frame:
metrics(two_class_example, truth, predicted)## # A tibble: 1 x 1
## accuracy
## <dbl>
## 1 0.838
# or
two_class_example %>% roc_auc(truth, Class1)## [1] 0.939
Quasiquotation can also be used:
# probability columns:
lvl <- levels(two_class_example$truth)
two_class_example %>% mnLogLoss(truth, !! lvl)## [1] -0.328
To install the package:
install.packages("yardstick")
## for development version:
require("devtools")
install_github("topepo/yardstick")