[Frontend] Python int too large to convert to C long
Closed this issue · 4 comments
chhzh123 commented
It seems we cannot support arrays whose element type's bitwidth is larger than 64? I got the following error when constructing a hcl.UInt(64) using hcl.asarray.
Traceback (most recent call last):
File "main_packed.py", line 296, in <module>
hcl_array.append(hcl.asarray(params[name], dtype=dtype))
File "/scratch/users/hc676/heterocl/python/heterocl/mlir/operation.py", line 47, in asarray
return Array(np_array, dtype)
File "/scratch/users/hc676/heterocl/python/heterocl/mlir/tensor.py", line 316, in __init__
np_array = np_array.astype(np.int64)
OverflowError: Python int too large to convert to C long
zzzDavid commented
It's a numpy limitation that hcl.asarray couldn't store array whose element data type is wider than 64-bit. This means we can't input/output or initialize constant tensors wider than 64-bit, because we can't store the data with numpy arrays. We still have anywidth integer support inside the top function body though.
If we can find a fancy math library for python to store/arith/print wide integer, we can support such cases. Or the other way I can think of is to break each element in a wide data type tensors and store them with many numpy arrays, and then pack them together in top function body.
…________________________________
From: Hongzheng Chen ***@***.***>
Sent: Friday, July 1, 2022 2:13:17 AM
To: cornell-zhang/hcl-dialect-prototype ***@***.***>
Cc: Niansong Zhang ***@***.***>; Assign ***@***.***>
Subject: Re: [cornell-zhang/hcl-dialect-prototype] [Frontend] Python int too large to convert to C long (Issue #94)
Assigned #94<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcornell-zhang%2Fhcl-dialect-prototype%2Fissues%2F94&data=05%7C01%7C%7C713cf987b8a84b007d7808da5b28caae%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637922528024485039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=XsYoc8n2R0ceyVIfeNa96pGArPtRII7FufpuFv6p5%2BQ%3D&reserved=0> to @zzzDavid<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FzzzDavid&data=05%7C01%7C%7C713cf987b8a84b007d7808da5b28caae%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637922528024485039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ftrkpdNZzMDoMICP7RUTUWJe6GfLmq4vL0EFJo%2FLcs4%3D&reserved=0>.
—
Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcornell-zhang%2Fhcl-dialect-prototype%2Fissues%2F94%23event-6915874413&data=05%7C01%7C%7C713cf987b8a84b007d7808da5b28caae%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637922528024485039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1uMxzaeuAnY000vToJ89Ea9tYDhryWVFhXnWy%2Bjgsmk%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAIAFRLZFE4YTIECR76AVHJDVR2D73ANCNFSM52LTFRTQ&data=05%7C01%7C%7C713cf987b8a84b007d7808da5b28caae%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637922528024485039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=BYYtZIu7VcviDvaZWyAd%2BobtC3Nfe1IYMobz%2FmoiaYg%3D&reserved=0>.
You are receiving this because you were assigned.Message ID: ***@***.***>
chhzh123 commented
Okay, this is not a sound fix. We still need to figure out whether the input array is unsigned or not, and use the corresponding np.uint64 or np.int64 to implement. Otherwise, some tests may fail.