Wrong results when using `hcl.cast` to convert `hcl.Int` to `hcl.Fixed`
sqPoseidon opened this issue · 1 comments
sqPoseidon commented
I'm using the hcl-mlir
branch and want to convert the arbitrary integer data to fixed point data.
In some test cases, I got the wrong results which are not due to the precision loss.
import heterocl as hcl
import numpy as np
# in_dtype, out_dtype = hcl.Int(8), hcl.Fixed(16, 8) # failed
# in_dtype, out_dtype = hcl.Int(8), hcl.Int(32) # succeed
# in_dtype, out_dtype = hcl.Int(6), hcl.Int(8) # succeed
# in_dtype, out_dtype = hcl.Int(6), hcl.Fixed(16, 8) # failed
for in_precision in range(6, 33, 2):
for out_precision in range(8, 33, 2):
in_dtype, out_dtype = hcl.Int(in_precision), hcl.Fixed(out_precision, int(out_precision/2))
def cast(A):
return hcl.compute(A.shape, lambda *args: hcl.cast(out_dtype, A[args]), "cast", dtype=out_dtype)
A = hcl.placeholder((1, 10), "A", dtype=in_dtype)
s = hcl.create_schedule([A], cast)
f = hcl.build(s)
a_np = np.random.randint(0, 5, A.shape)
a = hcl.asarray(a_np, dtype=in_dtype)
c = hcl.asarray(np.zeros(A.shape), dtype=out_dtype)
try:
f(a, c)
except Exception as e:
print("{} -> {} failed: {}".format(in_dtype, out_dtype, e), flush=True)
else:
c_np = c.asnumpy()
if (a_np == c_np).all():
print("{} -> {} succeeded.".format(in_dtype, out_dtype), flush=True)
else:
print("{} -> {} failed, wrong result value".format(in_dtype, out_dtype), flush=True)
print("a_np: ", a_np)
print("c_np: ", c_np)
For example,
Int(8) -> Fixed(12, 6) failed, wrong result value
a_np: [[3 4 2 2 3 2 4 2 2 1]]
c_np: [[-1. 0. -2. -2. -1. -2. 0. -2. -2. 1.]]
Please help check the problem.
zzzDavid commented
This issue was caused by left shift overflow, fixed by cornell-zhang/hcl-dialect@e026ca3
The test case is added to regression tests here:
Lines 633 to 670 in afe4471