Mira Resource Indexer

A part of Project Mira

requirements: mongodb

Indexer can be configured to different modes: dmhy, Bangumi.moe

Set environment variables

INDEXER_MODE what mode this indexer will work at. set dmhy or bangumi_moe or nyaa
DB_HOST host for database instance, default is mongo
DB_PORT port for database, default is 27017
DB_USER user for database access, default is admin
DB_NAME database name, default is dmhy_indexer
DB_PASS password for database access, default is 123456
AUTH_SOURCE see https://docs.mongodb.com/manual/core/authentication/, default is admin
SERVER_HOST the REST API server host, default is 0.0.0.0
MIN_INTERVAL the minimal time between two query on source site, default is 10000 millisecond
MIN_CHECK_INTERVAL the minimal time between two query on list page of source site, default is 900000 millisecond
MAX_PAGE_NO the scraping max page number, default is 5
MAX_SEARCH_COUNT the REST API search max result count, default is 100
SENTRY_DSN (optional) the dsn required for your sentry project. see the Sentry section

Start from docker

Setup Requirements

Install Docker

Boot up

You can start container by:

Start all service defined in docker-compose.yml using docker-compose up

$ docker-compose up --build

To run it in background (detached mode), just add -d option

$ docker-compose up --build -d

Run main container in bash mode

$ docker-compose start mongo
$ docker-compose run --no-deps --rm --service-ports main bash
node@add8121dc7c5:/irohalab/indexer$ npm run start

Sentry

When deploy to test or production server. it's important to know the health of application. For example, you may need to know a unusual network issue caused by source sites or you may experience incorrect format of scrapped page. All of these information help you improve the application by analyzing data collected from error log.