OpenELM example?
MatthewWaller opened this issue · 3 comments
Thanks for the awesome work! I would request an OpenELM example. Or if we can tweak one of the existing examples for OpenELM, that would be great. Any guidance?
Have unsucessfully tried to run the swift LLM tool with this model: mlx-community/OpenELM-1_1B-Instruct-4bit
It's now working in MLX core with the Python code. The output is impressive for the very small model size, meaning it should offer great utility on older iOS devices with less RAM. However, it's not working in MLX-Swift. It has the same error as per the issue below. "Error: unhandledKeys(base: "Embedding", keys: ["biases", "scales"])". So it appears that something fundamental needs to be done to get this to work at all in iOS and Swift.
Good to close when the other PR is accepted! Would love to figure out how to get the proper instruct prompt working, but I can open a different PR for that.