Add support for local logging of metrics

[0xtristan]

GCC bill for tensorboard is starting to add up due to a lot of write operations ($0.05 per 10k writes).
Need to both decrease freq of logging for TB and add better support for local logging.