LucienShui/huggingface-vscode-endpoint-server

support ReplitCode

Closed this issue · 5 comments

luckyfish0826 commented

Hi Lucien, Thanks for your work. I saw your code support replitcode，yet seems it could not load the model as I tried.

I am still try to fix it, how I can work with you on the. Replit is also a good model.

Hope you can see this.

By the way, I currently encountered below error msg:
ValueError: Unrecognized configuration class <class 'transformers_modules.replit.replit-code-v1-3b.a97a3fd23f03aa31b2a35a71b000ef67c60d0071.configuration_mpt.MPTConfig'> to build an AutoTokenizer.

luckyfish0826 commented

update: above error is because of the transformers version. 4.28.1 not working, 2.29.2 working.

thus fix: pip3 uninstall transformers && pip3 install transformers==4.29.2

luckyfish0826 commented

Oh I saw your requirements.txt specified the transformers version as 4.28.1, maybe for replit it need 4.29

luckyfish0826 commented

also for replit, it needs sentencepiece module

luckyfish0826 commented

server is up, but failed at first try:

curl -X POST http://localhost:8000/api/generate/ -d '{"inputs": "hellow_world()", "parameters": {"max_new_tokens": 64}}'

response: 500 Internal Server Error

stack trace:
File "/home/notebook/code/group/huggingface-vscode-endpoint-server-main/generators.py", line 69, in generate
params.pop('stop')
KeyError: 'stop'

luckyfish0826 commented

update: just comment that line (params.pop('stop')) , it works well.