Unable to run Gradio demo: VILA with TinyChat
Opened this issue · 1 comments
mitraavi commented
I've encountered a persistent issue while running the Gradio demo: "Gradio demo: VILA with TinyChat" on a local server, despite following the steps here: GitHub Link.
Problem: The model fails to load, and the Model dropdown menu in the Gradio UI doesn’t appear.
Details:
- Steps 1 and 3 (launching the controller and model worker) seem to work, with the controller returning a 200 response.
- Path to models "Efficient-Large-Model/VILA1.5-3b" (fp16-hf-model) and "vila-1.5-3b-w4-g128-awq.pt" provided in the model worker
- If I run Step-1,2,3, sequentially i noticed model passed in step-3 to controller never getting loaded in Gradio server. This means controller is unable to interact with Gradio server in the subsequent time as it is evident from screenshot here:
- if I run Step-1,3, first and step-2 at last then I can see that model in the "Model" section although unable to select it.
Overall I feel Controller is unable to interact with Graidio server after the 1st run. Any guidance on resolving this would be greatly appreciated.