jina-ai/late-chunking

RuntimeError: [enforce fail at alloc_cpu.cpp:114] data. DefaultCPUAllocator: not enough memory: you tried to allocate 33507323568 bytes.

Closed this issue · 1 comments

Godly-GM commented

I tried to pass the context from a 19-page PDF to the model, but I encountered this error:
RuntimeError: [enforce fail at alloc_cpu.cpp:114] data. DefaultCPUAllocator: not enough memory: you tried to allocate 33507323568 bytes.

here input_text is the content of pdf.
Screenshot 2024-11-02 200912

guenthermi commented

It looks like your machine doesn't have enough memory to encode very long sequences of text. You could use the long late chunking method, which is implemented in the _embed_with_overlap method (

late-chunking/chunked_pooling/mteb_chunked_eval.py

Line 237 in db558c3

model_outputs = self._embed_with_overlap(model, model_inputs)
) in our evaluation code together with a lower number of tokens ( long_late_chunking_embed_size in the function) property to circumvent this issue.