null2028/AniSync

Mapping sites to AniList and back.

AniSync

Mapping sites to AniList and back.
Inspired by MalSync, this project is made for taking search queries from popular tracking sites such as AniList and matching them with sites such as Zoro.To, AnimeFox, and more.

How it Works

The concept of AniSync is relatively simple. Using a string similarity algorithm (Dice's Coefficient, courtesy of the NPM package string-similarity), AniSync first sends a request to AniList and then to all providers. Then, a comparison will be started between the title of the AniList result and the provider's result. AniSync will then try and find the best result and return it. This process is ran twice: once for sending a request to AniList and once for sending a request to the providers. This is done to ensure that the best result is returned. If you are confused, take a look at the search() function in the Sync.ts file.

export default class Sync extends API {
    /**
     * @description Searches on AniList and on providers and finds the best results possible.
     * @param query Media to search for.
     * @param type Type of media to search for.
     * @returns Promise<FormattedResponse[]>
     */
    public async search(query:string, type:Type): Promise<FormattedResponse[]> {
        let result:FormattedResponse[] = [];
        // Searches first on the database for a result
        const possible = await this.db.search(query, type);
        if (!possible || possible.length === 0) {
            if (config.debug) {
                console.log(colors.yellow("No results found in database. Searching providers..."));
                console.log(colors.gray("Searching for ") + colors.blue(query) + colors.gray(" of type ") + colors.blue(type) + colors.gray("..."));
            }
            // Search on AniList first
            const aniSearch = await this.aniSearch(query, type);
            if (config.debug) {
                console.log(colors.gray("Received ") + colors.blue("AniList") + colors.gray(" response."));
            }
            const aniList = this.searchCompare(result, aniSearch);
            // Then search on providers
            const pageSearch = await this.pageSearch(query, type);
            if (config.debug) {
                console.log(colors.gray("Received ") + colors.blue("Provider") + colors.gray(" response."));
            }
            // Find the best results possible
            const pageList = this.searchCompare(aniList, pageSearch, 0.5);
            await this.db.insert(pageList, type);
            return pageList;
        } else {
            return possible;
        }
    }
}

Installation

For convienience sake, I'll be mentioning a full-depth guide on how to install AniSync and crawl on Linux. It is recommended that if you are setting up an API using AniSync you use a Linux server on Digital Ocean or via a VPS from a hosting company like ForestRacks. Anyways:

1. Install NodeJS and NPM on your server. You can do this via the following commands:

# Installs NodeJS
sudo apt-add-repository -r ppa:chris-lea/node.js
sudo apt update -q
curl -fsSL https://deb.nodesource.com/setup_14.x | sudo -E bash - &&\
sudo apt-get install -y nodejs

# Installs NPM
sudo apt install npm

2. Install the dependencies for AniSync:

# SQLite3
sudo apt install sqlite3

3. Clone the repository:

# Clones the repository
git clone https://github.com/Eltik/AniSync

# Enters the directory
cd AniSync

4. If necessary, pull:

git pull

5. Install the dependencies:

npm install

6. Build the project:

npm run build

7. Init the database:

# Creates the database
sqlite3 db.db
.exit

# Creates all the tables necessary
npm run init

You can now crawl through the providers. To do so, run the following command:

# Crawls through anime
npm run crawl:anime

# Crawls through manga
npm run crawl:manga

If you want the process to run infinitely install screen and run screen npm run crawl:anime.

# Installs screen
sudo apt install screen

# Runs the crawler in a screen
screen npm run crawl:anime
screen npm run crawl:manga

# To get a list of instances
screen -ls

# To reattach to a screen
screen -r [id]

# To detach from a screen
ctrl + a + d

# To kill a screen
screen -X -S [id] quit

Contributing

Contribution would very much be appreciated. If you have any suggestions or requests, create a pull request or issue. Adding other "connectors" or sites such as TVDB, MyAnimeList, 4anime, etc. is super easy. The only thing required would be to create a new file under /[anime/manga] and add the search() function:

export default class FourAnime extends Provider {
  constructor() {
    super("https://4anime.gg", ProviderType.ANIME);
  }
  
  public async search(query:string): Promise<Array<SearchResponse>> {
    ...
  }
}

Then, just add the connector to the classDictionary variable in AniSync.ts:

export default class AniSync extends API {
    constructor() {
        super(...);

        ...
        this.classDictionary = [
            {
                name: "TMDB",
                object: new TMDB()
            },
            ...
            {
                name: "FourAnime",
                object: new FourAnime()
            }
        ]
    }
}

That's it! Feedback would be appreciated...

Providers

Name	Link	Notes
Zoro.To	Link	N/A
GogoAnime	Link	N/A
AnimeFox	Link	N/A
AnimePahe	Link	N/A
Enime	Link	N/A
ComicK	Link	N/A
MangaDex	Link	N/A
Mangakakalot	Link	N/A
TVDB*	Link	Meta
4anime*	Link	N/A
9anime*	Link	Requires special keys/mapping
MyAnimeList*	Link	Meta

*Not finished yet.