whylabs/langkit
🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance metrics, & sentiment analysis. 📊 A comprehensive tool for LLM observability. 👀
Jupyter NotebookApache-2.0
Issues
- 2
Specify Python version compatibility
#261 opened by BVasquez07 - 0
Import error, architecture not supported for arm64
#310 opened by xiaoboshg - 1
Add support for python 3.12
#195 opened by jamie256 - 1
Possible merge issue with 0.0.31 release
#300 opened by jonbaiernx - 0
SWE-Agent: Implement the hallucinations metric from the main branch in the workflow branch
#298 opened by naddeoa - 3
Use custom prompt in response_hallucination
#263 opened by pradeepdev-1995 - 0
reduce latency in lazy initialization
#278 opened by FelipeAdachi - 0
add support for detoxify local models
#277 opened by FelipeAdachi - 1
Explain refusal similarity
#253 opened by pradeepdev-1995 - 1
consistency_check in response_hallucination
#262 opened by pradeepdev-1995 - 2
- 0
Better error message if OpenAI key is not provided in response_hallucination
#245 opened by FelipeAdachi - 3
- 1
Original data
#223 opened by iazdan - 3
Importing metrics issue since there is not a way to pass the model path if stored locally
#230 opened by Starignus - 0
- 0
- 0
Need example of prompt column name override
#226 opened by jamie256 - 0
Add feature to use bedrock agent to do the evaluation
#220 opened by rajib76 - 0
text-davinci-003 OpenAI model deprecated
#215 opened by FelipeAdachi - 8
Add support for AzureOpenAI
#197 opened by rajib76 - 0
Response Hallucination: decouple LLMs for sample generation and consistency checking
#205 opened by FelipeAdachi - 0
- 0
Add support for armv6l, armv7l chipset architecture
#196 opened by jamie256 - 2
- 0
- 0
Consider asserting input types
#184 opened by jamie256 - 1
need a way to override metric names in LangKitConfig
#174 opened by jamie256 - 0
- 0
example notebooks use older style why.init
#159 opened by jamie256 - 0
xformers causes crash on mac with injections
#161 opened by jamie256 - 0
Don't create an iterable
#155 opened by andyndang - 0
langkit's pypi page does not show a homepage link
#150 opened by jamie256 - 0
Documentation Upgrades
#143 opened by FelipeAdachi - 0
- 0
make has_patterns directly callable for strings
#125 opened by FelipeAdachi - 3
Any plan to support multilingual?
#74 opened by jack1981 - 6
load tests interfere with eachother
#109 opened by jamie256 - 0
aggregage_reading_level should output a float
#131 opened by jamie256 - 1
Support multiple embedding models
#120 opened by zmacks - 0
Themes module don't allow custom groups
#119 opened by FelipeAdachi - 0
regexes expansion
#112 opened by FelipeAdachi - 2
Release workflow is hitting a permission error
#102 opened by jamie256 - 0
- 0
- 1
Support prometheus metrics
#73 opened by andyndang - 0
- 0
add docs & tests for openai wrapper
#52 opened by andrewelizondo - 1
udf_schema should not track frequent items
#90 opened by jamie256 - 3