ParchmentProphet-ui

ParchmentProphet-ui is a Streamlit-based application designed to evaluate and compare human-generated and AI-generated text samples. This tool leverages the ParchmentProphet package to compute linguistic and conceptual similarity metrics between human and AI text samples.

A hosted version is available here: https://parchmentprophet-ca.bravesea-a3cf7791.uksouth.azurecontainerapps.io

Features

Upload text files containing human and AI-generated text samples for evaluation.
Compare the performance of different AI models based on the provided text samples.
View summary statistics and error bar plots for various linguistic features.
Explore example text generations for the best and worst performing models.

File Format Requirements

Your uploaded evaluation data must adhere to the following format:

Plain text format with a .txt extension.
Each file should contain multiple sections separated by *****, representing different models.
Within each model section, pairs of human and AI-generated text should be separated by [----].
Example

Human text 1
[----]
AI text 1
[----]
Human text 2
[----]
AI text 2
*****
Human text 3
[----]
AI text 3
...

Installation

Clone the repository

git clone https://github.com/Sankgreall/ParchmentProphet-ui.git
cd ParchmentProphet-ui

Copy .env.sample to .env and fill in the required variables. You will need an OpenAI API key.
Build the Docker image

docker build -t parchmentprophet-ui .

Run the Docker container

docker run -p 8501:8501 parchmentprophet-ui

Access the application on `http://localhost:8501