/p6-rose

Perl 6 Discord bot for content moderation and toxicity analysis.

Primary LanguagePerl 6BSD 3-Clause "New" or "Revised" LicenseBSD-3-Clause

image

Perl 6 Discord bot for content moderation and toxicity analysis. This project was created initially for Discord Community Hack Week. Rose performs content moderation via message analysis using Google's Perspective API. She scores users based on the percieved 'toxicity' of their messages and provides server administrators with an overall threat level metric based on aggregate results.

Want to add Rose to your own server, but don't have the time or know-how to self-host her? Click the button below!

image

Once she's in, check out the +help command to get started.

Thank you for checking out our Discord Hack Week project. Good luck to all participants.

Prerequisites

Core

Dependencies

All of these can be obtained via zef install in the case of Perl 6 modules, or apt-get in the case of packages.

Configuration

Create the file config.json in the root of the project directory. It should look something like this:

{
  "discord-token": "",
  "perspective-token": "",
  "command-prefix": "+",
  "auto-moderation": false,
  "postgresql-host": "127.0.0.1",
  "postgresql-port": 5432,
  "postgresql-user": "rose",
  "postgresql-password": "password",
  "postgresql-database": "rose"
}

You will need to provide your own Discord bot token, and your own Google Perspective API token. If you do not have a Perspective API token you can follow the steps to generate one for free here.

If you would like Rose to perform auto-moderation actions on messages, set the auto-moderation configuration value to true and she will delete messages and/or kick users who cross predetermined thresholds.

In order to see more of the inner workings of Rose, you can set PERSPECTIVE_DEBUG=true in your environment and she will provide a debug emote reaction to all detected messages, and respond with their score from the Perspective API.

image

Commands

+aggregate $user-id

Returns the average toxicity score of a user over their last 200 messages, and categorises them as either a low, medium or high risk user. Future iterations of Rose will have the option to automatically kick high risk users (those with an average score of 0.75 or higher across their last 200 messages).

image

Docker

There is some somewhat experimental Docker relevant material in this repository - mostly included for my own testing and debugging purposes but it's usable if you know what you're doing. Note that in all cases you still must provide an appropriate PostgreSQL server (whether running remotely or in a container on the same Docker virtual network) for Rose to function. This information should be defined within your config.json and bind-mounted into the container. You can use the ROSE_CONFIG variable to inform Rose of where her configuration file has been mounted.

Usage

... via docker build

docker build \                
  --build-arg BUILD_AUTHORS="Kane 'kawaii' Valentine <kawaii@cute.im>" \
  --build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') \
  --build-arg BUILD_SHA1SUM=80549a2f1e7c35bccfd91007f3984c68b07cd4ec \
  --build-arg BUILD_VERSION=0.1rc4 \
  --tag kawaii/rose:0.1rc4 \
  --tag kawaii/rose:latest \
$PWD

... via docker container run

docker container run -e ROSE_CONFIG="/opt/config.json" -v $PWD/config.json:/opt/config.json:ro kawaii/rose:0.1rc4

... via docker stack deploy or docker-compose

services:
  rose:
    environment:
      ROSE_CONFIG: "/opt/config.json"
    image: kawaii/rose:0.1rc4
    restart: on-failure
    volumes:
    - $PWD/config.json:/opt/config.json:ro
version: '3.7'

FAQ

Why Perl 6?

Why not Perl 6? This is Discord Hack Week, not Discord write-everything-in-JS week.

Want to get started with Perl 6 yourself? Check out the Rakudo compiler.

What library does Rose use?

Rose uses the experimental and unfinished Perl 6 API::Discord library.

What is API::Discord?

API::Discord is a free and open source Discord API wrapper written in Perl 6 by Shuppet Laboratories.

At the time of writing this document, the library is unfinished and only features a fraction of the full Discord API specification. Due to this setback we were not able to implement all of the features we wanted, although it did enable us to fix a number of existing bugs in the library.

There are dangers of using such a prototype component in any project, but I think that Rose is a good example of what people can achieve in the space of a few evenings - hacking away in true open source spirit.

Perspective API? Why don't you use Tensorflow's toxicity model?

A number of reasons. Tensorflow's model was trained on a single dataset of toxic comments (albeit 2 million of them) whereas Perspective is receiving new data for analysis on a daily basis. It seems to be a far more accurate gauge of toxicity at the current time. Another reason is processing, it's much better to offload content analysis to another service, and let your application just take action on the results.

The bot crashes, disconnects and does other weird things, what the heck?

Due to upstream bugs in Cro::WebSocket (#15 and #22), there will be occasions when the bot drops out, fails to reconnect or just crashes for seemingly no reason.

Who is Rose?

My girlfriend, you can join her server here. :)

image