Tracing Auto Instrumentation allows you to easily instrument popular LLM frameworks for tracing your LLM application. It is built with and on top of the LastMile Rag Debugger: https://rag.lastmileai.dev/.
Sure, here's the table with the example links formatted as bullet points:
Framework/Library | Example Link |
---|---|
OpenAI | - openai_instrumentation.ipynb -openai_streaming.ipynb |
IBM | - ibm.ipynb -ibm_granite_tutorial.ipynb |
LangChain | - langchain.ipynb |
LLamaIndex | - llama_index.ipynb -llama_index_function_calls.ipynb |
Getting started is easy. Simply choose the framework you want to instrument and follow the instructions below. If you want to instrument multiple frameworks, you can install all
.
pip install "tracing-auto-instrumentation[all]"
pip install "tracing-auto-instrumentation[openai]"
import openai
from tracing_auto_instrumentation.openai import wrap_openai
from lastmile_eval.rag.debugger.tracing.sdk import get_lastmile_tracer
tracer = get_lastmile_tracer(
tracer_name="OpenAI Function Calling",
)
client = wrap_openai(openai.OpenAI(), tracer)
pip install "tracing-auto-instrumentation[langchain]"
import langchain
from tracing_auto_instrumentation.langchain import LangChainInstrumentor
# Create an instance of LangChainInstrumentor and instrument the code
instrumentor = LangChainInstrumentor(project_name="Plan-and-Execute Example")
instrumentor.instrument()
pip install "tracing-auto-instrumentation[llamaindex]"
import llama_index.core
from tracing_auto_instrumentation.llama_index import LlamaIndexCallbackHandler
llama_index.core.global_handler = LlamaIndexCallbackHandler(
project_name="LlamaIndex with Paul Graham",
)
pip install "tracing-auto-instrumentation[ibm]"
# todo