Pinned Repositories
Denzel-Crocker-Hunting-For-Fairly-Odd-Prompts
A serverless set of functions for evaluating whether incoming messages to an LLM system seem to contain instances of prompt injection; uses cascading cosine similarity and ROUGLE-L calculation against known good and bad prompts
Eliza-Thornberry-and-the-conformal-prediction-of-LLM-Behavior
Python code for use in predicting a range of LLM outputs by dimension (e.g. sentiment, topic) across fixed domain values (e.g. positive, neutral, negative) or other attributes (e.g. account details, feedback, help needed), and thus monitoring the anticipated possible drift based on the most recent outputs, as a measure of LLM Drift Detection
Enterprise-Executive-Summaries
A repository for the development of one page summaries intended to communicate important complex ideas succinctly; for enterprise distribution
LatentSpace.Tools
Latent Space Tools help conceptualize, visualize, and subsequently operationalize the necessary architecture and software components for secure LLM Deployment & Monitoring
Serverless-Latent-Space-Monitoring
A series of serverless functions/resources (and Terraform) for consuming language model inputs and outputs to S3, enriching the data via sentiment analysis and topic modelling, loading to DynamoDB and subsequently monitoring for configurable deviation within the latent vector space.
SpongeBob-SquarePants-and-the-Imaginary-OpenAI-endpoint
A Terraform script, and set of tfvars, that allow provisioning of an Nginx reverse proxy via EC2 for the puprose of redirecting traffic for SDKs and applications not already configured to support open-source models
Squidward-Tentacles-and-Spying-on-Outputs-via-Conformal-Prediction
A serverless function is utilized to evaluate the divergence of a particular output from the established log-likelihood set by a language model. This function is designed to compute the log-likelihood per message. Subsequently, p-values are generated and used as a prediction interval to categorize, appropriately append, and sort LLM output
StoopKid-Event-Driven-Input-Monitoring-for-Language-Models
A set of serverless functions designed to assist in the monitoring of inputs to language models, including routine and specific inspection of the message queue, as well as event-driven triggering of more complex metric calculation based on (what will eventually be) configurable environment variables; alongside a suite of other tools/functions
Turing-s-Labyrinth--Escape-the-Simulation
An inverted turing test, wherein the player proceeds through a series of increasingly complex paradoxes. This done by solving the puzzle, decoding the riddle, or rejecting the premise. They are assisted by an AI who's context and role evolves throughout the game.
ZeroDay.Tools
Gen AI Hardening x Attack Suite
rabbidave's Repositories
rabbidave/Denzel-Crocker-Hunting-For-Fairly-Odd-Prompts
A serverless set of functions for evaluating whether incoming messages to an LLM system seem to contain instances of prompt injection; uses cascading cosine similarity and ROUGLE-L calculation against known good and bad prompts
rabbidave/LatentSpace.Tools
Latent Space Tools help conceptualize, visualize, and subsequently operationalize the necessary architecture and software components for secure LLM Deployment & Monitoring
rabbidave/Eliza-Thornberry-and-the-conformal-prediction-of-LLM-Behavior
Python code for use in predicting a range of LLM outputs by dimension (e.g. sentiment, topic) across fixed domain values (e.g. positive, neutral, negative) or other attributes (e.g. account details, feedback, help needed), and thus monitoring the anticipated possible drift based on the most recent outputs, as a measure of LLM Drift Detection
rabbidave/Serverless-Latent-Space-Monitoring
A series of serverless functions/resources (and Terraform) for consuming language model inputs and outputs to S3, enriching the data via sentiment analysis and topic modelling, loading to DynamoDB and subsequently monitoring for configurable deviation within the latent vector space.
rabbidave/Enterprise-Executive-Summaries
A repository for the development of one page summaries intended to communicate important complex ideas succinctly; for enterprise distribution
rabbidave/SpongeBob-SquarePants-and-the-Imaginary-OpenAI-endpoint
A Terraform script, and set of tfvars, that allow provisioning of an Nginx reverse proxy via EC2 for the puprose of redirecting traffic for SDKs and applications not already configured to support open-source models
rabbidave/Squidward-Tentacles-and-Spying-on-Outputs-via-Conformal-Prediction
A serverless function is utilized to evaluate the divergence of a particular output from the established log-likelihood set by a language model. This function is designed to compute the log-likelihood per message. Subsequently, p-values are generated and used as a prediction interval to categorize, appropriately append, and sort LLM output
rabbidave/StoopKid-Event-Driven-Input-Monitoring-for-Language-Models
A set of serverless functions designed to assist in the monitoring of inputs to language models, including routine and specific inspection of the message queue, as well as event-driven triggering of more complex metric calculation based on (what will eventually be) configurable environment variables; alongside a suite of other tools/functions
rabbidave/Turing-s-Labyrinth--Escape-the-Simulation
An inverted turing test, wherein the player proceeds through a series of increasingly complex paradoxes. This done by solving the puzzle, decoding the riddle, or rejecting the premise. They are assisted by an AI who's context and role evolves throughout the game.
rabbidave/ZeroDay.Tools
Gen AI Hardening x Attack Suite