/mlx-engine

Apple MLX engine for LM Studio

Primary LanguagePythonMIT LicenseMIT

lmstudio + MLX

mlx-engine - Apple MLX LLM Engine for LM Studio


Discord

mlx-engine

MLX engine for LM Studio


Built with

  • mlx-lm - Apple MLX inference engine (MIT)
  • Outlines - Structured output for LLMs (Apache 2.0)
  • mlx-vlm - Vision model inferencing for MLX (MIT)

How to use in LM Studio

LM Studio 0.3.4 and newer for Mac ships pre-bundled with mlx-engine. Download LM Studio from here


Standalone Demo

To run a demo of model load and inference:

  1. Clone the repository
git clone https://github.com/lmstudio-ai/mlx-engine.git
cd mlx-engine
  1. Create a virtual environment (optional)
 python -m venv myenv
 source myenv/bin/activate
  1. Install the required dependency packages
pip install -r requirements.txt
  1. Run the demo.py script
python demo.py --model ~/.cache/lm-studio/models/mlx-community/Meta-Llama-3.1-8B-Instruct-4bit