Segfault on large inputs?
corani opened this issue · 2 comments
corani commented
When I run the the build/bin/main example with a larger input I get a segfault:
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 271388624, available 260703040)
Segmentation fault
I can work around this by doing an N *= 2; near the bottom of bert_load_from_file, but obviously that isn't the right solution. It seems somewhere a calculation is off (probably with mem_per_token).