Speed up generating inferred images
Closed this issue · 1 comments
I brought this up in the Optics FYDP Messenger chat at 2019-07-16 around 5:55 p.m. Currently the code in the notebook that loads a (post-training) checkpoint for the cGAN and goes through the test set generating the inferred OMAG-like OCTA images is pretty slow. It spawns a Dataset of length one for every B-scan file, to guarantee the order of predicted images and create the file so that it corresponds to that B-scan number. (Just using glob.glob and sticking the file results into a single Dataset of length ~1280 (full length) doesn't give that property.)
It shouldn't take too long, but requires some thinking to figure out how to avoid this inefficiency. This task is to time-optimize the process of generating the inferred OCTA files as much as possible, based on the metric of total running time of that snippet of code (from the start of loading the checkpoint until the end of the loop that generates all of the images.)
Links that may be partially useful:
- https://stackoverflow.com/questions/51162500/how-can-i-access-the-filenames-gathered-by-tf-data-dataset-list-files
- https://stackoverflow.com/questions/48896762/how-to-use-the-tensorflow-dataset-api-to-read-files-with-different-names-without
- Documentation for
from_generator: https://www.tensorflow.org/api_docs/python/tf/data/Dataset#from_generator
As part of this task, we can also look into whether we want to add any functionality to skip over N B-scans. For example, with sedated rat data where we have 4 acquisitions (B-scans) of the same cross section, the code should actually output N/4 (1280/4 = 320) inferred images, one for each different cross-section (every group of consecutive 4 inferred images should look almost exactly the same in this case, since it's the same cross section taken over a short period of time!) But for human data, the point is we'll end up only having 1 acquisition and want to infer blood flow, so there, we'll want all the B-scans run through the for loop. We should ask Zohreh if this should be configurable (with a default of 1) or if this isn't worth spending time on.
This code now lives at the bottom of src/utils.py.