llm_metrics

Core Repository for Evaluating Prompts and their metrics

Setup

git clone https://github.com/priyanshu-sharma/llm_metrics.git

cd llm_metrics/

bash setup.sh

To start all the components without any prior installation use

make non_container_dev

This will up the postgres db, redis and Rabbitmq server

Open a new Terminal and configure Postgres Database with: -

sudo -u postgres psql

This will open postgres prompt. Enter the following commands to configure the database user. and '\q' to quit the postgres prompt.

create user pd_admin with encrypted password 'password';

grant all privileges on database product_design to pd_admin;

Open a new Terminal and start Redis Server with: -

redis-server

Open a new Terminal and start Redis CLI with: -

redis-cli

Open a new Terminal and start RabbitMQ Server with: -

rabbitmq-server

Open a new Terminal and install Metrics Server dependencies with: -

pip install -r requirements.txt
cd src/metrics_server/
python manage.py migrate

And start Metrics Server with: -

python manage.py runserver

Open a new Terminal and start Metrics Server Celery with: -

cd src/metrics_server/

celery -A server_config.celery.app worker -c 1 -l info -P eventlet

Open a new Terminal and install UI dependencies using: -

cd src/metrics_client/

npm install

And start the UI Server with: -

npm start

.env file for metrics-client (will on the same level as of metrics_client's src)
local-settings.py in server_config in metrics server