mdp/litellm-cline

LiteLLM and Cline

This project demonstrates how to run a demo for LiteLLM starting with the Codestral model for use with Cline. Below are the steps to set up and run the demo:

Prerequisites

• Docker
• A .env file with the necessary environment variables, see .env.sample

Setup

1. Create a .env file:

   • Copy the .env.example file to .env:

     cp .env.example .env

   • Open the .env file update the placeholders
     • The CODESTRAL_API_KEY is not the same at the Mistral API keys, it can be found under Codestral Dashboard section
     • Although this is limited to localhost, please set a good LITELLM_MASTER_KEY

2. Configuration:

All the configuration details exists in config.yaml

This example configuration is setup for the latest Codestral model, but you would enter more models and API keys here if you wanted them

Running the Demo

1. Start the demo:

   • Run the start.sh script to start the LiteLLM demo:

     ./start.sh

2. Setup Cline:

Once the LiteLLM server is up and running you can set it up in Cline

• Base URL should be http://0.0.0.0:4000/v1
• API Key should be the one you set in .env for LITELLM_MASTER_KEY
• Model ID is codestral or whatever you named it under config.yaml