This project is currently in the proof-of-concept stage, the codebase and its output is quite rough, so it is not yet intended for actual use, but for interested developers.
The first milestone will be to make it useable for quickly checking
documentation emitted by cargo doc
to the local filesystem.
-
Local, recursive link checking for HTML documents, using the
html5ever
tokenizer to extract the links. -
Support for URL fragments (anchors).
-
Concurrent, futures-based architecture using
tokio
. The retrieval side can operate on many documents in parallel, making use of Tokio's threadpool. The extraction is done concurrently as well, but in a single thread, ashtml5ever
is not thread-safe. So if you can ingest more data than a single core can extract URLs from, this will be a bottleneck. Otherwisehakelig
aims to be as scalable as possible.Although the code has not been optimized at the slightest,
hakelig
can churn through about 27MiB of link-rich documentation output bycargo doc
in about a second.
-
Reasonable output and progress indication, probably taking linkchecker as a source of inspiration.
-
HTTP support.
The code and documentation in the hakelig
crate is free software,
licensed under the GNU GPL version 2, or (at your option)
any later version of the GNU GPL.