gpu job schedule via file system.
- do some cleaning work
- if there is a free gpu, and a running job file marked that gpu, then remark the file as
[{$GPU}UNKNOWN{$TIME}]
- if there is a free gpu, and a running job file marked that gpu, then remark the file as
- when there is a free gpu
- pick a job file (shell script) in
jobs/queue/ - move the job under
jobs/and mark as[{$GPU}RUN{$TIME}]- do a backup
[{$GPU}BK{$TIME}]
- do a backup
- start that job and redirect output to job's log file
- if fail, or exception raised, mark the job as
[{$GPU}ERROR{$TIME}] - if success, mark as
[DONE{$TIME}], and move job file and log file tojobs/done/