writeAlizer: An R Package to Generate Automated Writing Quality and Curriculum-Based Measurement (CBM) Scores
This repository hosts code for an R package to apply research-based writing scoring models (see references below). In addition, this repository hosts documentation as an electronic supplement to published research articles in the repository wiki.
The writeAlizer R package (a) imports ReaderBench, Coh-Metrix, and GAMET output files into R, (b) downloads existing predictive scoring models to the local machine, and (c) uses the predictive scoring models to generate predicted writing quality scores or Correct Word Sequences and Correct Minus Incorrect Word Sequences scores from the ReaderBench, Coh-Metrix, and/or GAMET files.
- 1.0.5 (Aug03-2020): Revised ReaderBench file import to allow for input with any length
- 1.0.4 (July-29-2020): Fixed ReaderBench file import
- 1.0.3 (May-29-2020): Accepts ReaderBench output files as .csv
- 1.0.2 (May-27-2020): predict_quality() also returns an ID variable
- 1.0.1 (May-14-2020): Added model fit objects to package and test data
- 1.0.0 (May-1-2020): Initial Version
writeAlizer accepts the following output files as inputs:
- ReaderBench: writeAlizer supports output files (.csv format) generated from the standalone version of ReaderBench that is described here and can be downloaded from here.
- Coh-Metrix: writeAlizer supports output files from Coh-Metrix version 3.0 (.csv format).
- GAMET: writeAlizer supports output files from GAMET version 1.0 (.csv format).
The writeAlizer scoring models assume that column names in the output files have been unchanged (exactly the same as generated from the program). For programs that list file paths in the first column, the writeAlizer file import functions will parse the file names from the file paths and store the file names as an identification variable (ID). File names/ID variables need to be numeric.
writeAlizer is not available on CRAN due to file size (~500 mb). To install writeAlizer in R, first make sure that the package devtools is installed in R
install.packages("devtools")
With devtools installed, you can install writeAlizer in R directly from this Github repository.
devtools::install_github("shmercer/writeAlizer")
Due to the large file size, it may take a while for writeAlizer to download and install.
Information on the various scoring models available and how they were developed is in this respository's wiki:
- Description of the general process used to develop scoring algorithms.
- Description of the following specific scoring models, including information on the relative importance of metrics and weighting of algorithms:
- Sterett H. Mercer - University of British Columbia - UBC Faculty Profile - ResearchGate - Google Scholar
Mercer, S. H., Keller-Margulis, M. A., Faith, E. L., Reid, E. K., & Ochs, S. (2019). The potential for automated text evaluation to improve the technical adequacy of written expression curriculum-based measurement. Learning Disability Quarterly, 42, 117-128. https://doi.org/10.1177/0731948718803296
Mercer, S. H., Keller-Margulis, M. A., & Matta, M. (2020, February). Validity of automated vs. hand-scored written expression curriculum-based measurement samples. Poster presented at the Pacific Coast Research Conference, Coronado, CA, USA.
Mercer, S. H., & Cannon, J. E. (2020, February). Monitoring the written expression gains of learners during intensive writing intervention. Poster presented at the Pacific Coast Research Conference, Coronado, CA, USA.
Keller-Margulis, M. A., & Mercer, S. H. (2019, August). Validity of automated scoring for written expression curriculum-based measurement. Poster presented at the meeting of the American Psychological Association, Chicago, IL, USA.
Mercer, S. H., Tsiriotakis, I., Kwon, E., & Cannon, J. E. (2019, June). Evaluating elementary students' response to intervention in written expression. Paper presented at the meeting of the Canadian Association for Educational Psychology (Canadian Society of the Study of Education), Vancouver, BC, Canada.
This project is licensed under the GNU General Public License Version 3 (GPLv3).
- The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305A190100. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education. Principal Investigator: Milena Keller-Margulis (University of Houston). Co-Principal Investigator: Sterett Mercer (University of British Columbia). Co-Principal Investigator: Jorge Gonzalez (University of Houston). Co-Investigator: Bruno Zumbo (University of British Columbia).
- This work was supported by a Partnership Development Grant (Assessment for Effective Intervention in Written Expression for Students with Learning Disabilities) from the Social Sciences and Humanities Research Council of Canada. Principal Investigator: Sterett Mercer (University of British Columbia). Co-Investigators: Joanna Cannon (UBC) and Kate Raven (Learning Disabilities Society of Greater Vancouver).