Kotoba

Kotoba is a project with several semi-independent pieces. Those are:

In addition to the three main projects there are a few other directories:

/common - Common code that is intended to be shared between node processes and browser.
/node-common - Common code that is intended to be shared between node processes but not browser.
/nginx - An nginx configuration for proxying HTTP requests to the frontend and API.
/backup - Tools for backing up user data to Google Cloud Storage.
/worker - A worker process for doing some heavy lifting for other processes whose event loops should not be blocked.

Configuration

After cloning the repo, fill out config/config_sample.js and rename it to config.js. You only have to fill out the sections for the components you want to run. For example if you're only going to run the bot, you only have to fill out the bot section.

Self-hosting the bot

After following the configuration instructions:

sudo apt install docker docker-compose
sudo docker-compose up kotoba-bot kotoba-worker mongo_readwrite

The bot will take some time to build and should then come online. Note that it will be missing some fonts and features that are not in this repo. These instructions are for Ubuntu Linux.

Developing

Node v12 is recommended

Discord bot

cd ./bot
npm install
npm start

The bot will start and come online. Some commands won't work. There are additional steps required to make certain commands work:

Quiz command
1. In the ./bot directory, run npm run buildquiz.
2. In the ./bot directory, run npm run buildfontcharactermap.
Pronunciation command
1. Install MongoDB and start it on port 27017 (the default port). You can install it using the instructions for your operating system here.
2. In the ./bot directory, run npm run buildpronunciation.
Shiritori command
1. Install MongoDB and start it on port 27017 (the default port). You can install it using the instructions for your operating system here.
2. In the ./bot directory, run npm run buildshiritori.

KotobaWeb

Install cairo and pango. You can install them using the instructions for your operating system here.
Install MongoDB and start it on port 27017 (the default port). You can install it using the instructions for your operating system here.

cd ./api
npm install
npm run buildall
npm run startdev_nix # Or on Windows: npm run startdev_win

cd ../kotobaweb
npm install
npm start

The API will start on port 3000 and the React dev server will start on port 3001. You'll need to setup a reverse proxy server to forward to them appropriately. You can use nginx with a configuration like this:

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;

    server {
        listen 80;

        location /api {
            client_max_body_size 4M;
            proxy_pass http://localhost:3000;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection "upgrade";
            proxy_set_header Host $host;
        }

        location / {
            proxy_pass http://localhost:3001;
        }
    }
}

Worker process

The worker process handles some heavy lifting so that the event loop doesn't have to block in the bot or API. The bot and API communicate with it via HTTP. It doesn't do much at the moment, and most features work without it, so you can probably ignore it until you're working with a feature that requires it and you notice that HTTP requests to it are failing.

cd ./worker
npm install
npm start

Help

Support

Third party links

Data from the following third parties has been used in Kotoba.

Jisho.org
Princeton University Japanese WordNet
KanjiVG
Forvo
Merriam-Webster
Oxford Dictionaries
Japanese Wiktionary
EDICT
ENAMDICT
Kanjimaji
Google Translate
YouTube
Jonathan Waller's site

In addition various people have contributed quiz data and are credited in the relevant quiz descriptions.

Child libraries

The following other codebases were written in the course of this project:

fpersist - On disk persistence with safer writes. GitHub NPM
unofficial-jishi-api - Encapsulates the official Jisho.org API and also provides kanji and example search. GitHub NPM
render-furigana - Render Japanese text with furigana into a PNG buffer. GitHub NPM
monochrome - A flexible node.js Discord bot core based on Eris. GitHub NPM
jp-verb-deconjugator - Unconjugate conjugated Japanese verbs. GitHub NPM
shiritori - A backend engine for playing shiritori. GitHub NPM
array-on-disk - A module for storing and accessing large arrays on disk. GitHub NPM

Lb-lea/kotoba