scheduleR is a framework that can be used to deploy R tasks, reports and apps.
- Tasks are 'regular' R scripts that you want to schedule to be executed on a regular basis (often ETL related scripts).
- Reports are Rmarkdown (.Rmd) reports that can be converted to a PDF or HTML. See rmarkdown for more info.
- Apps are Shiny apps, support for these in scheduleR is experimental.
An easy web interface for scheduling is provided for adding tasks, maintenance and viewing logs. scheduleR provides extensive logging support and error/success notifications.
scheduleR is built to be used on a server. It can be used locally but that mean that you have to keep a mongodb server and scheduleR running at all times.
Minimal dependencies:
Optional dependencies (necessary for generating Rmarkdown reports):
scheduleR's web interface is built using Node.js and tested under GNU/Linux and Windows. Feedback on Mac compatibility is appreciated.
Make sure you have access to a running MongoDB server. You can set one up locally or use a service such as mongolab for testing.
First download the repository using the latest tarball / zip file and extract it or simply clone the repository. Cloning the repository is advised as this makes updating scheduleR a piece of cake.
git clone https://github.com/Bart6114/scheduleR.git
Next, run the following command to install dependencies (might require admin rights).
npm install
If when using npm install you get an error like Error: ENOENT ... try updating npm to the latest version: npm update -g npm (might require admin rights).
To configure scheduleR go to the directory containing the installation and edit the user.config.json file to your likings:
- uploadDir path to upload your scripts to (is essential!)
- RScriptExecutable how to call the
Rscriptexecutable (best to simply putRscripton the path) - RstandardArguments standard arguments to add to
Rscript(should be fine as is) - errorNotificationMailAddresses (array of) email addresses that will be added to the recipients of error notification mails
- mailer.from the from address to use in notification/report mails
- mailer.options settings for
nodemailerto user - db.url the address of the mongodb server
- db.suffix the name used to create a db on the mongodb server
- port the port to serve scheduleR on
You can start scheduleR by simply running in the directory where it was downloaded.
npm start
You can then point your browser at the configured port & server address (default is http://localhost:3000).
At the first start of scheduleR an initial signup option is available. After the initial signup new users can be invited (an account will be created for them). An invite email will be send to them with their username and initial password. If something goes wrong in sending this email (e.g. when you don't have access to a mail server) the new user's username and initial password are shown in the server-side console.
The list tasks view gives an overview of all the scheduled tasks. One can choose a specific task to get a detailed overview on run times and logs. The edit mode is also available via the detailed view.
Using new task one can schedule a new script:
- name the name of the task
- description a description of the task
- script file upload the .R file
- extra arguments potential extra arguments to call the script with
- enabled should the task be enabled after saving
- ignore lock if disabled, tasks are locked when executing, meaning that no other instance can be started when previous instance hasn't finished
- mail report to addresses to the resulting report to (as attachment) - make sure to press enter after each address
- schedule the desired schedule (see the cron manual)
- on success addresses to send successful execution notification to - make sure to press enter after each address
- on error addresses to send unsuccesful execution notification to - make sure to press enter after each address
The list reports view gives an overview of all the scheduled reports. One can choose a specific report to get a detailed overview on run times and logs. The edit mode is also available via the detailed view.
Markdown reports (pdf / html) can be copied to a specified path (either on the server on one that is available on the network) or be sent by mail to a number of recipients.
Using new report one can schedule a new report:
- name the name of the report
- description a description of the script
- script file upload the .Rmd file
- extra arguments potential extra arguments to call the script with
- enabled should the task be enabled after saving
- Rmd output path where to copy the resulting report (optional)
- Rmd mail message a message for the body of the report mail (optional)
- mail report to addresses to the resulting report to (as attachment) - make sure to press enter after each address
- schedule the desired schedule (see the cron manual)
- on success addresses to send successful execution notification to - make sure to press enter after each address
- on error addresses to send unsuccesful execution notification to - make sure to press enter after each address
Beware, shiny app support is very experimental.
When adding new app, the following input fields are available:
- name the name of the app
- description a description of the app
- script file the script file should be a zipped (.zip) file with at is root the required file to call
shiny::runApp()on (e.g.server.randui.r). - URL suffix the URL to access the app by, if the suffix is for example
test-appyou can access the app by using http://address-to-scheduleR/app/test-app.
See an example below:
In the examples directory you can find a few example scripts to test scheduleR's functionality with.
Up to now only tested on a recent Chrome browser. Testing on different browsers is appreciated.
When a new version is available it will be shown in the footer. Updating is easiest when you've done an install using git clone, in this case you can simply do a pull:
git pull
It is a good idea to always run the following after an update:
npm install
npm prune
For support and bug reports create a new issue at scheduleR's GitHub site.
Bug fixes are very welcome.
- switched scheduling engine from
node-crontoschedule-node(cron string specification now more in line with crontab) - some extra refactoring of the scheduling code
- removed 'detached' setting when spawning processes from node (caused some problems on Windows machines)
- fixed bug where some files didn't get properly renamed before copying
- fixed up some jslint warnings
- report generation bugfix: files could only be copied if email addresses were present
- tasks are locked when executing, meaning that no other instance of the task can be started if the previous instance hasn't finished (behaviour can be overruled in the task's settings, see docs)
- fixed bug that didn't correctly show report status in list view
- correct link to reports from 'enabled reports' button
- added list of mail-to addresses to detailed report view
- Added experimental support for Shiny apps
- Separated R tasks from R markdown reports
- A lot of refactoring
- Added (limited) pagination to logs in detailed task view in order to improve performance
- Improved file renaming method
- Functionality to add timestamps to the filename of generated Rmd reports
- Small optimizations to retrieving big log lists
- Inviting users depended on the ability to send emails (hence have access to a mail server). Quick fix: if there is an error in sending the account details to the user the username and initial password will be printed on the server-side console.
- The filenames of Rmarkdown reports are renamed back to the original filename (instead of the randomly generated upload filename)
- Added a "notificationMailAddresses" variable to user.config.json, email addresses specified here will be added to the recipients of all notification mails
- Added a "run once" button to test scheduled tasks
- Added status (success / error) to notification mails