neuro-inc/platform-api

Create a script for generating daily billing reports

dalazx opened this issue · 1 comments

The required script should:

  1. work in UTC
  2. given the current date, retrieve all jobs using /jobs which were run within the date, combine a report with the required fields
  3. (re)upload the report to S3 under the known agreed key format (this may be split into writing to the local FS, then uploading using a CLI)

Please pay attention to some edge cases such as handling timezones, midnights etc.

Presumably this script might be run lots of times during a day. The existing report for the given day, thus, should be successfully updated.

Please add some logging statements for better observability.

Qs:

  • script consumes two parameters, start and end date(time), we calculate its cost in a given timeframe. (to confirm)
  • AFAIR, we keep the pod for "failed" jobs for ~5 mins or so. It does reserve resources on the node which also should be included in the job cost, right?
  • if the job was running for a couple of days, should it be included included in reports for each days with a partial cost, or only in the final one with entire cost?