/validate

Professional data validation for the R environment

Primary LanguageR

CRAN Downloads status Mentioned in Awesome Official Statistics

Easy data validation for the masses.

The validate R-package makes it super-easy to check whether data lives up to expectations you have based on domain knowledge. It works by allowing you to define data validation rules independent of the code or data set. Next you can confront a dataset, or various versions thereof with the rules. Results can be summarized, plotted, and so on. Below is a simple example.

> library(validate)
> check_that(iris, Sepal.Width < 0.5*Sepal.Length) |> summary()
  rule items passes fails nNA error warning                       expression
1   V1   150     79    71   0 FALSE   FALSE Sepal.Width < 0.5 * Sepal.Length

With validate, data validation rules are treated as first-class citizens. This means you can import, export, annotate, investigate and manipulate data validation rules in a meaninful way.

To get started: see our data validation cookbook.

Citing

Please cite the JSS article

@article{van2021data,
  title={Data validation infrastructure for R},
  author={van der Loo, Mark PJ and de Jonge, Edwin},
  journal={Journal of Statistical Software},
  year={2021},
  volume ={97},
  issue = {10},
  pages = {1-33},
  doi={10.18637/jss.v097.i10},
  url = {https://www.jstatsoft.org/article/view/v097i10}
}

To cite the theory, please cite our Wiley StatsRef chapter.

@article{loo2020data,
  title = {Data Validation},
  year = {2020},
  journal = {Wiley StatsRef: Statistics Reference Online},
  author = {M.P.J. van der Loo and E. de Jonge},
  pages = {1--7},
  doi = {https://doi.org/10.1002/9781118445112.stat08255},
  url = {https://onlinelibrary.wiley.com/doi/10.1002/9781118445112.stat08255}
}

Other Resources

Installation

The latest release can be installed from the R command-line

install.packages("validate")

The development version can be installed as follows.

git clone https://github.com/data-cleaning/validate
cd validate
make install

Note that the development version likely contain bugs (please report them!) and interfaces that may not be stable.