uptrain-ai/uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
PythonApache-2.0
Issues
- 12
- 1
Return tokens count/cost for checks
#656 opened by ssaha1567 - 2
FastAPI deployment
#654 opened by HasnainKhanNiazi - 2
Support for LLMs hosted on AWS Bedrock
#589 opened by bithikaj - 1
Support for LLMs running on Vertex AI
#641 opened by verdverm - 1
Update the Integrations table in README.md
#624 opened by Dominastorm - 0
Create a Use Case Example
#522 opened by Dominastorm - 9
- 1
Too many values to unpack (resp_text.split)
#615 opened by lbux - 0
Revise the API Reference documentation
#521 opened by Dominastorm - 1
Update test_builtins.py
#520 opened by Dominastorm - 1
Ollama integration
#514 opened by sourabhagr - 1
Gemma Integration
#554 opened by Dominastorm - 5
Private / Azure OpenAI endpoints
#440 opened by nielsuit227 - 0
Add an operator to compute MT-Bench
#605 opened by Dominastorm - 0
Mamba Integration
#604 opened by Dominastorm - 2
- 1
- 1
New Project missing .jsonl file will not indicate it as a required file on Safari
#600 opened by cfrenchi - 1
Fresh venv install following the Quickstart example fails due to ModuleNotFoundError: No module named 'fsspec'
#599 opened by cfrenchi - 1
Mistral Endpoint Integration
#513 opened by sourabhagr - 1
Fix Jailbreak detection tutorial variable name
#579 opened by anas-rabhi - 1
Add test data for conversation evals
#540 opened by sourabhagr - 1
Add data for response matching eval
#542 opened by sourabhagr - 0
Add an operator to compute MMLU score
#557 opened by Dominastorm - 2
Llama Index integration for UpTrain
#389 opened by sourabhagr - 1
Together AI Integration
#512 opened by sourabhagr - 0
- 0
Modularize the operators
#532 opened by Dominastorm - 0
- 0
Fix the schema for conversation evals
#530 opened by sourabhagr - 0
Better Few shot examples for prompt injection eval
#529 opened by sourabhagr - 0
- 3
Why set request timeout just for OpenAI?
#373 opened by krrishdholakia - 2
Support for non-English LLMs.
#508 opened by Nkluge-correa - 1
Create a tutorial for ROUGE score
#487 opened by sourabhagr - 0
- 2
Can't use Checks without uptrain_access_token
#398 opened by danielgwilson - 4
Langchain & Langsmith integration for UpTrain
#388 opened by sourabhagr - 0
Unused parameter
#470 opened by sky-2002 - 4
Add perplexity evaluation operator
#276 opened by sourabhagr - 6
Add an operator for BLEU metric as well as example for machine translation task
#278 opened by sourabhagr - 0
- 2
- 0
Why not use ContextWindowExceedError instead?
#372 opened by krrishdholakia - 2
- 0
DOC: Adding Urdu in readme file
#257 opened by MuhammadNizamani - 0
- 0
- 0
Dynamically join Buckets in data drift clustering
#255 opened by sourabhagr