phospho: Text Analytics Platform for LLM Apps

Phospho is the text analytics platform for LLM apps. Detect issues and extract insights from text messages of your users or your app. Gather user feedback and measure success. Iterate on your app to create the best conversational experience for your users.

Ship your LLM app in production with confidence, and iterate on it with insights from your users.

Key Features

Flexible logging
Automatic evaluation
Insights extraction
Data visualization
Collaboration

Demo

phospho-demo-feb24.mp4

Quickstart: Discover the phospho lab

The phospho lab is the core analytics component of phospho. The phospho lab helps you run batched evaluations and event detections on your messages.

Step 1 - Install the phospho lab:

pip install "phospho[lab]"

We will run a simple Event detection job that use OpenAI to read a message and figure out whether an Event happened or not.

export OPENAI_API_KEY=your_openai_api_key

Step 2 - Run the event detection job:

Phospho helps you define a bunch of jobs to efficiently run in a workload on users messages.

from phospho import lab

# A workload is a set of jobs to run on messages
# A job is a Python function that leverages LLM and ML models to extract insights from text
workload = lab.Workload()
workload.add_job(
   lab.Job(
         id="question_answering",
         job_function=lab.job_library.event_detection,
         config=lab.EventConfig(
            event_name="question_answering",
            event_description="The user asks a question to the assistant",
         ),
   )
)

# The workload runs the jobs in parallel on all the messages to perform analytics
await workload.async_run(
   messages=[
      lab.Message(
            id="my_message_id",
            role="User",
            content="What is the weather today in Paris?",
         )
   ],
)

message_results = workload.results["my_message_id"]
print(f"The event question_answering was detected: {message_results['question_answering'].value}")

Step 3 - Check the results:

The event question_answering was detected: True

This event detection pipeline can then be linked back to your product or alerting solution (eg. Slack) through webhooks or used to notice patterns in the user data.

You can use other jobs from the phospho library or create your own jobs to run on your messages.

See more examples of what you can do with phospho in the examples folder.

Self deploy

This repository contains the implementation of the platform frontend, the API backend, and the insights extraction pipeline.

phospho-python: Python client with analytics engine
extractor: FastAPI analytics service wrapping the analytics engine
backend: FastAPI backend
platform: NextJS frontend
internal-tools: Platform management tools

Clone the repo:

git clone git@github.com:phospho-app/monorepo.git

Register to the core external services:
- OpenAI (or another OpenAI-compatible model provider)
- Cohere
- MongoDB Atlas (Alternative: self host a MongoDB)
(Optional) Extend the platform capabilities by registering to additional services:
- Resend (emails)
- Sentry (performance analytics)
- Propelauth (authentication)
- Stripe (payment)
Follow the deployment instructions in backend/README.md and platform/README.md
Enjoy !

Access the hosted version

To manage the phospho lab evaluations on a collaborative platform, the easiest way is to register to the hosted version.

Create a phospho account
Install a phospho client: pip install phospho or npm i phospho
Create environment variables for PHOSPHO_API_KEY and PHOSPHO_PROJECT_ID
Initialize phospho: phospho.init()
Log to phospho with phospho.log(input="question", output="answer")

Follow this guide to get started.

Licence

This project is licensed under the Apache 2.0 License - see the LICENSE file for details

StanGirard/phospho