mit-han-lab/inter-operator-scheduler

IOS Inference results

Closed this issue · 3 comments

is the IOS inference results correct?

I define a small sample network with all convolution operations and set the input to 1 and the convolution operator's weight and bias to 1 for the whole array by changing the random part of the code.
I am expecting an Integer result (no fractional part) but getting a floating point result with some fractional value. I mean the result does not match my expected result.
Is there any reason to get fractional results after setting all input, weight, and bias values to 1? I am confused whether I am missing something.

def sample_network():
output_shape = (1, 5, 5)
v = ios.placeholder(output_shape)
block = ios.Block(enter_node=v.node)
v1 = ios.conv2d(block, inputs=[[v]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
v2 = ios.conv2d(block, inputs=[[v]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
v3 = ios.conv2d(block, inputs=[[v]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
v1 = ios.conv2d(block, inputs=[[v1]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
out = ios.identity(block, inputs=[[v1, v2],[v3]], is_exit=True) # reduce v1, v2, and concat v3
block1 = ios.Block(enter_node=out.node)
v11 = ios.conv2d(block1, inputs=[[out]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
v21 = ios.conv2d(block1, inputs=[[out]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
v31 = ios.conv2d(block1, inputs=[[out]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
v11 = ios.conv2d(block1, inputs=[[v11]], out_channels=1, kernel=(3, 3), stride=(1, 1), padding=(1, 1), act='relu')
out1 = ios.identity(block1, inputs=[[v11, v21, v31]], is_exit=True) # reduce v1, v2, and v3
graph = ios.Graph(name="demo", input=v.node, blocks=[block, block1])
graph.init_weights()

weight Initialization:
self.weight = np.ones(shape=self.weight_shape)
self.bias = np.ones(shape=self.bias_shape)
self.weight.fill(1)
self.bias.fill(1)

    #self.weight = np.random.uniform(low=-bound, high=bound, size=self.weight_shape).astype(np.float32)
    #self.bias = np.random.uniform(low=-bound, high=bound, size=self.bias_shape).astype(np.float32)

Input:
dummy_inputs = np.ones(output_shape)
dummy_inputs.fill(1)
output = ios.ios_runtime.graph_inference(graph, batch_size=1, input=dummy_inputs)

By default, np.ones(output_shape) will use float64 data type instead of float32. Consider using

dummy_inputs = np.ones(output_shape, dtype=np.float32)

Thanks for your response. Now getting only zeros as output. could be some other datatype issue. Thanks.

I can get the >0 result when the input has float32 dtype.