zlib(libgen) searcher

Search zlib/libgen index to get ipfs_cid.

We don't save and provide files, we provide search.

I hope everyone have a copy of the index locally, so that no need to rely on any centralized service.

Usage

1. Download the pre-compiled binary from Release.

Or you can compile by yourself. Refer to Build from source for instructions.

2. Download the `index` file that has been created.

We will give the corresponding index download links for each version in the release page.

Or you can make your own via bin/index.rs.

Extract the index folder to the same level as the program, it should look like the following:

zlib_searcher_dir
├── index
│   ├── some index files...
│   └── meta.json
└── zlib-searcher

3. Run `zlib-searcher`, it will listen to `127.0.0.1:7070`.

Access http://127.0.0.1:7070/ to use webui, or you can use the original api.

original search api

You can search by the following fields:

title
author
publisher
extension
language
isbn
zlib_id

Examples:

http://127.0.0.1:7070/search?limit=30&query=余华
http://127.0.0.1:7070/search?limit=30&query=title:机器学习 extension:azw3 publisher:清华
http://127.0.0.1:7070/search?limit=30&query=zlib_id:18557063
http://127.0.0.1:7070/search?limit=30&query=isbn:9787302423287

Build from source

1. Build `zlib-searcher`

You need the frontend code in the frontend branch, and organize it like this:

├── zlib-searcher // this repo
│   └── src
└── zlib-searcher-frontend
    ├── dist   // this is what we need.
    ├── public
    └── src

Then run

cd ../zlib-searcher-frontend
npm install && npm run build

cd ../zlib-searcher
cargo build --release

# move the compiled binary to the project root directory
mv target/release/zlib-searcher .

2. Build `index`

Download zlib_index_books.csv.zip and libgen_index_books.csv.zip and extract the csv files to the project root directory.

Then run cargo run --bin index --release. You may need to mkdir index or rm index/* first to make sure index is an existing empty folder.

The finally folder structure should look like this:

zlib_searcher_dir // in the example above, it is project root directory.
├── index
│   ├── some index files...
│   └── meta.json
└── zlib-searcher

Raw data

We downloaded libgen sql and zlib sql and exported the necessary data from them.

id, title, author, publisher, extension, filesize, language, year, pages, isbn, ipfs_cid

This raw data is used to generate our index, you can download the raw data from here:

License

Blog zu1k.com · GitHub @zu1k · Twitter @zu1k_lv · Telegram Channel @peekfun

Shanshan-IC/zlib-searcher