/humble

A humble, and fast!, security-oriented HTTP headers analyzer.

Primary LanguagePythonMIT LicenseMIT

humble



HTTP Headers Analyzer

"A journey of a thousand miles begins with a single step. - Lao Tzu"

"And if you don't keep your feet, there's no knowing where you might be swept off to. - Bilbo Baggins"

Table of contents

Features
Screenshots
Installation & Update
Usage
Advanced Usage
Linux: Show only the analysis summary
Windows: In spanish. Show only the analysis summary (PowerShell >= 7 required)
Linux: Show only the URL, date and analysis summary
Linux: Show only the deprecated headers/protocols and insecure values
Linux: Check for HTTP client errors (4XX)
Linux: Analyze multiple URLs and save the results as PDFs
Checks: Missing Headers
Checks: Fingerprint Headers
Checks: Deprecated Headers and Insecure Values
Checks: Empty Values
Guidelines included
To-Do
Further Reading
Contribute
Acknowledgements
License

Features

✔️ 13 checks of missing HTTP response headers.
✔️ 996 checks of fingerprinting through HTTP response headers.
✔️ 85 checks of deprecated HTTP response headers/protocols or with insecure/wrong values.
✔️ Browser compatibility check for enabled security headers.
✔️ Two types of analysis: brief and detailed, along with HTTP response headers.
✔️ Export of analysis to HTML5, PDF 1.4 and TXT.
✔️ The analysis includes dozens of references, official documentation and technical articles.
✔️ i10n: analysis results in English or Spanish.
✔️ Saves each analysis, showing (at the end) the improvements or deficiencies in relation to the last one.
✔️ Shows analysis statistics: either against a specific URL or all of them.
✔️ Shows fingerprint statistics: either against a specific term or the Top 20.
✔️ Code reviewed via pycodestyle, SonarLint and Sourcery.
✔️ Tested, one by one, on thousands of URLs.
✔️ Fully tested and working on Windows (10 20H2 - 19042.985) and Linux (Kali 2021.1).
✔️ All code under one of the most permissive licenses: MIT.
✔️ Regularly updated.
✔️ Developed in my spare time, with no relationship with clients/companies: try it and integrate it into your projects!.
✔️ Technical resource accepted in the OWASP Secure Headers Project and Kali Linux.

Screenshots

.: (Windows) - Brief analysis.

Brief Analysis


.: (Linux) - Brief analysis and retrieved HTTP headers.

Brief analysis + retrieved headers


.: (Linux) - Detailed analysis in Spanish.

Full analysis


.: (Linux) - List of HTTP fingerprint headers based on a specific term.

Specific fingerprint headers


.: (Windows) - Detailed analysis exported to PDF. Example.

Export analysis to PDF


.: (Linux) - Detailed analysis exported to HTML. Example.

Export analysis to HTML


.: (Linux) - Analysis history file: Date, URL, Missing, Fingerprint, Deprecated/Insecure, Empty headers & Total warnings (the four previous totals).

History of analysis performed


.: (Linux) - Statistics of the analysis performed against a specific URL.

Statistics of the analysis performed against a URL


.: (Linux) - Statistics of the analysis performed against all URLs.

Global statistics of the analysis performed


Installation & Update

NOTE: Python 3.9 or higher is required.

# install python3 and python3-pip if not exist
(Windows) https://www.python.org/downloads/windows/
(Linux) if not installed by default, install them via, e.g. Synaptic, apt, dnf, yum ...

# install git
(Windows) https://git-scm.com/download/win
(Linux) https://git-scm.com/download/linux

# clone the repository
$ git clone https://github.com/rfc-st/humble.git

# change the working directory to humble
$ cd humble

# install the requirements
$ pip3 install -r requirements.txt

# update humble (every week, inside humble's working directory)
$ git pull

# or download the latest release (every four to five weeks)
https://github.com/rfc-st/humble/releases

Usage

(Windows) $ py humble.py
(Linux)   $ python3 humble.py

usage: humble.py [-h] [-a] [-b] [-f [TERM]] [-g] [-l {es}] [-o {html,pdf,txt}] [-r] [-u URL] [-v]

humble (HTTP Headers Analyzer) - https://github.com/rfc-st/humble

options:
  -h, --help         show this help message and exit
  -a                 show statistics of the performed analysis (will be global if '-u URL' is omitted)
  -b                 show a brief analysis (if omitted, a detailed one will be shown)
  -f [TERM]          show fingerprint statistics (will be the Top 20 if "TERM", e.g. "Google", is omitted)
  -g                 show guidelines for securing popular web servers/services
  -l {es}            show the analysis in the indicated language (if omitted, English will be used)
  -o {html,pdf,txt}  save analysis to file (with the format URL_headers_yyyymmdd.ext)
  -r                 show full HTTP response headers and a detailed analysis
  -u URL             schema and URL to analyze. E.g. https://google.com
  -v, --version      show the version of this tool and check for updates

Advanced Usage

Linux: Show only the analysis summary

$ python3 humble.py -u https://www.spacex.com | grep -A 8 "\!." | sed $'1i \n'

Show only the analysis summary (Linux)

Windows (in Spanish): show only the analysis summary (PowerShell >= 7 required)

$ py humble.py -u https://www.spacex.com -l es | Select-String -Pattern '!.' -Context 1,8 -NoEmphasis

Show only the analysis summary (Windows, in Spanish. PowerShell >= 7 required)

Linux: Show only the URL, date and analysis summary

$ python3 humble.py -u https://www.spacex.com | grep -A7 -E "0. Info|\!." | grep -v "^\[1\." | sed 's/[--]//g' | sed -e '/./b' -e :n -e 'N;s/\n$//;tn' | sed $'1i \n'

Show URL, date and the analysis summary (Linux)

Linux: Show only the deprecated headers/protocols and insecure values

$ python3 humble.py -u https://www.spacex.com | sed '/3. /,/4. /!d' | sed '$d' | sed $'1i \n'

Show only the deprecated headers/protocols and insecure values (Linux)

Linux: Check for HTTP client errors (4XX)

$ python3 humble.py -u https://block.fiverr.com | grep -A1 -B5 'Note : \|Nota : ' --color=never

Check for HTTP client errors (4XX) (Linux)

Linux: Analyze multiple URLs and save the results as PDFs

$ datasets=('https://facebook.com' 'https://www.microsoft.com' 'https://www.spacex.com'); for dataset in "${datasets[@]}"; do python3 humble.py -u "$dataset" -o pdf; done

Analyze multiple URLs and save the results as PDFs

Checks: Missing Headers


Show / Hide
Cache-Control Clear-Site-Data Content-Type
Content-Security-Policy Cross-Origin-Embedder-Policy Cross-Origin-Opener-Policy
Cross-Origin-Resource-Policy NEL Permissions-Policy
Referrer-Policy Strict-Transport-Security X-Content-Type-Options
X-Frame-Options

Checks: Fingerprint headers

Check this file.

Checks: Deprecated headers/protocols and insecure values

Check this file.

Checks: Empty values

Any HTTP response header.

Guidelines included to enable security HTTP headers

  • Amazon AWS
  • Apache HTTP Server
  • Cloudflare
  • MaxCDN
  • Microsoft Internet Information Services
  • Nginx

To-do

  • Add more header/value checks (only security-oriented)
  • Google Style Python Docstrings and maybe documentation via Sphinx.

Further reading

https://caniuse.com/
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
https://github.com/search?q=http+headers+analyze
https://github.com/search?q=http+headers+secure
https://github.com/search?q=http+headers+security
https://owasp.org/www-project-secure-headers/
https://securityheaders.com/
https://scotthelme.co.uk/
https://webtechsurvey.com/common-response-headers
https://www.w3.org

Contribute

Thanks for your time!! :).

Acknowledgements

License

MIT © 2020-2023 Rafa 'Bluesman' Faura (rafael.fcucalon@gmail.com)
Original Creator - Rafa 'Bluesman' Faura (rafael.fcucalon@gmail.com)