__ __ __
/ /_ _____ / /_ / /
/ __ \ / ___/ / __/ / /
/ / / // /__ / /_ / /
/_/ /_/ \___/ \__/ /_/
Hostlists tools cli hctl is app written in rust to manage (merge, transform, clear from comments etc.) your hostlists from diffrent formats to selected by you!
- *nix systems (linux, BSD, mac): should be already there, if not install it from your package repo like
brew install openssl@3 - windows: install from
chocoorvcpkgor whatever other way that won't welcome viruses.
Note that you can get rid of openssl dependency if you want, by building from source with modified dependencies, see building from source.
Available:
- Alpine Linux edge:
apk add hctl --repository=https://dl-cdn.alpinelinux.org/alpine/edge/testing/(from testing repository) - OpenSUSE Linux 15.6/Tumbleweed:
opi hctl(using opi community repo helper) - Arch Linux (based):
yay -Su hctl(using AUR helper)
Packaging progress issue
| ❗ BUILD REGRESSION |
|---|
OpenBSD and DragonflyBSD doesn't have rust tier 1 support and their repo version is too old to compile on stable
| Platform | Downloads | Build Status | ||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
- NSU - Not Set Up Yet, may (or not) appear in the future
- Merge hostlist
- Remove duplicates
- Remove commented lines (default behaviour)
- Fetch lists directly from network (https)
- YAML config
- Parallel processing
- Rejected lines may be saved with information about source list
- Support for Windows encoded text files (line ending with
\n\r) - Output to stdout to be used with unix pipe stream control (partial support)
- Whitelists (enumerated, from external lists file/https)
- Whitelists including subdomains
- Record existance validation from DNS servers (unencrypted and over tls)
- User-defined DNS and DOT Resolvers
- Could use diffrent input types (like dnsmasq, bind) in yaml config
- Interacive status indication
- Settings yaml could be fetched from http(s)
- Auto scheme recognition
- YAML has full settings (remote sources, whitelist ,
args from cli) - Subdomains family recognition, save with subdomain order (like adaway does)
- Fault-tolerance, better error handling
- Deep rejected analysis
- Quiet option
- Progressbar on save
Jump here to see supported input and output formats
Jump here to feature-request issue.
# hctl -D --help
Manipulate on domains
Usage: hctl {domain|--domain|-D} [OPTIONS]
Options:
-p, --path <path> Path to file to be read [path without quotes]
-o, --out <out> Path to the out file [stdout/path without quotes]
-z, --optimize <optimize> Optimize for memory or speed, default: memory [memory/speed]
-c, --config <config> Path to config [without qoutes]
-m, --mode <mode> Process single or multiple lists [single/folder/config/url]
-i, --intro <intro> Whether append intro "Assembled From:..." to out file from config [true/false]
-r, --rejected <rejected> Whether save rejected to ./rejected.txt [true/false]
-f, --format <format> Type of out format [dnsmasq/loopback/empty/linewise/bind/snort/unbound/machosts/hostperm1/junkbuster/littlesnitch/pdnsd]
-t, --validate <validate> (Non-intrusive): Verbose validate your records in single file issuing dns-over-tls query in hardcoded servers [yes/no]
-d, --dns <dns> Validate your processed records issuing dns query in predefined servers, config mode allows custom ones [yes/no]
-a, --pattern <pattern> Input format of list [hosts/dnsmasq]
-h, --help Print helpThere is heavy usage of rust iterators to enable lazy-reading
Processing is done by firstly removing addreses characteristic for host file like 127.0.0.1 and comments after the records. Next tool removes all whitespace characters and non-urls using regex. See rules.rs if you want to embed similar solution in your app.