Description of the project aim to acheive
To make a crawler that can crawl and store data fast.
-
Tuning the RocksDB to balance the amplification rates
-
Use Rust to write an low latency crawler and RocksDB writer agent
The stack of what tools using for the project
- Rust
- RocksDB
The settings, installations for starting the solution