This tool (by default) will mirror the entirety of crates.io for you.
Unsurprisingly, this requires quite a lot of space. Assume you need a terabyte, as of my writing this in December 2023. (Strongly consider using a compressed btrfs or other filesystem to host this, in which case you can probably cut it by half.)
First we need to build this (no shocks here):
cargo build --releaseThen we need to check out the Git crate index (in this example, into the ./index
directory, which will be created if it doesn't exist):
./target/release/librarian -i ./index index-updateFinally, we need to download the crates (in this example, into ./corpus, which
will again be created if it doesn't exist):
./target/release/librarian -i ./index populate -c ./corpusNote that the corpus will have some extra levels based on the first 1-2 characters of the crate name, just to not stress your filesystem too much.
The index-update and populate commands can be run again to update existing
indices and corpora: you don't have to do a full redownload each time.