Multiple threads for tf-trt

[hoangmt]

Assume I have two threads, each run same structure, different weights tf-trt models. Will the time cut by half, compared to running two models in sequence? I am experimenting with efficientdet models but two threads approach does not reduce the time that much. Anyone has ever tried multiple threading with tf-trt? Using tensorrt I see the time is reduced by half in several cases but I was not able to convert efficientdet to tensorrt.