tensorflow/tensorrt

INT8 quantization failing

Opened this issue · 0 comments

I'm trying to quantize my model using some random numbers like this:

converter = trt.TrtGraphConverter(
    input_saved_model_dir=output_saved_model_dir, # For frozen graphs, you need to pass in input_graph_def and nodes_blacklist parameters. nodes_blacklist is a list of output nodes.
    precision_mode=trt.TrtPrecisionMode.INT8,
    is_dynamic_op=True,
    use_calibration=True)

frozen_graph = converter.convert()

converted_graph_def = converter.calibrate(
    fetch_names=['detection_boxes_l1:0','detection_scores_l1:0','detection_classes_l1:0','detection_boxes_l2:0','detection_scores_l2:0','detection_classes_l2:0'],
    num_runs=10,
    feed_dict_fn=lambda: {'ved/level1/import/image_tensor:0': np.random.normal(size=(1, 1200, 1200, 3))},)

But it is throwing me following error:

python3: cask/shaderlist_impl.h:50: void cask::ShaderList<ShaderType, OperationType>::sortHandles() const [with ShaderType = cask::GemmShader; OperationType = cask::Gemm]: Assertion `((*i)->handle != (*prevI)->handle) && "Internal error: CASK: all shaders must have unique names"' failed.
python3: cask/shaderlist_impl.h:50: void cask::ShaderList<ShaderType, OperationType>::sortHandles() const [with ShaderType = cask::GemmShader; OperationType = cask::Gemm]: Assertion `((*i)->handle != (*prevI)->handle) && "Internal error: CASK: all shaders must have unique names"' failed.
Aborted (core dumped)