abetlen/llama-cpp-python

AttributeError: function 'llama_sampler_init_tail_free' not found after compiling llama.pcc with hipBLAS

Opened this issue · 2 comments

Micromanner commented

Prerequisites

  • [ x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
  • [ x] I carefully followed the README.md.
  • [ x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • [x ] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

Run the model

Current Behavior

I get an attribute error when importing the module.


# Environment and Context

Windows 10, 7900XTX, 5800X3D, python3.10.11, gcc version 13.2.0 

# Failure Information (for bugs)

I think its a bug, I have no problem running llama.cpp, only the python package is making problems.

# Steps to Reproduce

1. Compile llama.cpp for hipBLAS (I followed the instructions)
2. python -m pip install . --no-cache-dir
3. Run my code (model) with python run.py

Failure Logs

Traceback (most recent call last):

  File "D:\test\run.py", line 7, in <module>
    from llama_cpp import Llama
  File "D:\test\env\lib\site-packages\llama_cpp\__init__.py", line 1, in <module>
    from .llama_cpp import *
  File "D:\test\env\lib\site-packages\llama_cpp\llama_cpp.py", line 3201, in <module>
    def llama_sampler_init_tail_free(z: float, min_keep: int) -> llama_sampler_p:
  File "D:\test\env\lib\site-packages\llama_cpp\_ctypes_extensions.py", line 113, in decorator
    func = getattr(lib, name)
  File "C:\Users\01tee\AppData\Local\Programs\Python\Python310\lib\ctypes\__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "C:\Users\01tee\AppData\Local\Programs\Python\Python310\lib\ctypes\__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function 'llama_sampler_init_tail_free' not found. Did you mean: 'llama_sampler_init_min_p'?
djeffkanda commented

I had the same issue, but solved it by pulling and installing the latest version. The fix is in the following commit[commit]