INT8 quantization failing
Opened this issue · 0 comments
anshkumar commented
I'm trying to quantize my model using some random numbers like this:
converter = trt.TrtGraphConverter(
input_saved_model_dir=output_saved_model_dir, # For frozen graphs, you need to pass in input_graph_def and nodes_blacklist parameters. nodes_blacklist is a list of output nodes.
precision_mode=trt.TrtPrecisionMode.INT8,
is_dynamic_op=True,
use_calibration=True)
frozen_graph = converter.convert()
converted_graph_def = converter.calibrate(
fetch_names=['detection_boxes_l1:0','detection_scores_l1:0','detection_classes_l1:0','detection_boxes_l2:0','detection_scores_l2:0','detection_classes_l2:0'],
num_runs=10,
feed_dict_fn=lambda: {'ved/level1/import/image_tensor:0': np.random.normal(size=(1, 1200, 1200, 3))},)
But it is throwing me following error:
python3: cask/shaderlist_impl.h:50: void cask::ShaderList<ShaderType, OperationType>::sortHandles() const [with ShaderType = cask::GemmShader; OperationType = cask::Gemm]: Assertion `((*i)->handle != (*prevI)->handle) && "Internal error: CASK: all shaders must have unique names"' failed.
python3: cask/shaderlist_impl.h:50: void cask::ShaderList<ShaderType, OperationType>::sortHandles() const [with ShaderType = cask::GemmShader; OperationType = cask::Gemm]: Assertion `((*i)->handle != (*prevI)->handle) && "Internal error: CASK: all shaders must have unique names"' failed.
Aborted (core dumped)