Computational Experiment Scheduling System (for Machine Learning) using Google Spreadsheets as a dashboard.
- Using this tutorial create API credentials.
- Create google spreadsheet with columns that correspond to arguments in your argparse parser
- Add special columns: 'time_started', 'last_update', 'progress_bar', 'server', 'status', 'comment'
- Add columns with performance characteristics of your script (e.g. 'test_accuracy')
- Write your worker script by Inheriting from
ParamsSheet
class, add credentials created on step 1 and give id of google spreadsheet that you created in step 2. - Make sure your 'train' function takes parsed arguments as input (output of
parse_args
method) - Add experiment parameters to a spreadsheet and watch worker do its thing.
- You can run as many workers in parallel as you like (the only limit is Google API, 500 requests / 100 seconds)
- Anything you can put as argparse optional argument can be a parameter in a spreadsheet.
- If your program raises an exception status of the experiment will be changed to 'error' and traceback will be copied to 'comment' column.
- You can specify environment variable "SERVERNAME" to override servers hostname.
- If you specify 'server' in spreadsheet only workers on that server will be able to take the task.