rust-lang/docs.rs

rustdoc-json: Output is is zstd compressed twice.

Closed this issue · 1 comments

The new rustdoc json enpoints (https://docs.rs/about/rustdoc-json) return files that need to be decompressed twice before getting the json:

alona@Ashtabula:~/tmp/docsrszstd$ curl --location  https://docs.rs/crate/clap/latest/json -o ./docs-rs-output
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 68106  100 68106    0     0   241k      0 --:--:-- --:--:-- --:--:--  241k
alona@Ashtabula:~/tmp/docsrszstd$ file ./docs-rs-output
./docs-rs-output: Zstandard compressed data (v0.8+), Dictionary ID: None
alona@Ashtabula:~/tmp/docsrszstd$ unzstd ./docs-rs-output -o decompressed-once
./docs-rs-output    : 68097 bytes
alona@Ashtabula:~/tmp/docsrszstd$ file ./decompressed-once
./decompressed-once: Zstandard compressed data (v0.8+), Dictionary ID: None
alona@Ashtabula:~/tmp/docsrszstd$ unzstd ./decompressed-once -o decompressed-twice
./decompressed-once : 422580 bytes
alona@Ashtabula:~/tmp/docsrszstd$ file ./decompressed-twice
./decompressed-twice: JSON text data
alona@Ashtabula:~/tmp/docsrszstd$ du -sh * | sort -h
68K     decompressed-once
68K     docs-rs-output
416K    decompressed-twice

This doesn't save any space, but makes using this more cumbersome and slower.

The files served from docs.rs should be the json compressed once, not twice.

oh damn, very valid bug report, I also know why.

Thanks!