🚥 Fast website monitoring backend service

Highlights

🍀 Feature-rich

Each check tracks:
- timestamp
- response time
- HTTP response status code
- optionally, given regex is (safely) tested if it matches the response body or not
- info about possible connection errors, like timeouts and/or unreachable host
Websites to check & their results are stored in postgres by default 🐘 (the library is ready for other data stores / sockets).
- You can use postgres locally installed, running on docker, or with a DBaaS, e.g. Aiven.
Monitor stored websites once, at configurable-scheduled intervals (each website check can use an independent interval or use a default), or even with your system's cron.
The scheduling keeps running even if the computer goes to sleep.
Nice, configurable logging.
CLI API & Python's (Python >= 3.11).
- A webserver is planned.
...and more!

🚀 Speed

All operations are asynchronous. This app sits on 3 giants:
- aiohttp
- psycopg (v3)
- APScheduler (v4)
Written in Python 3.11 for maximum speed 🐍
Speedy regex checking thanks to google-re2 regex. Note that google-re2 syntax is very similar to python's native re but not equal. In particular, backreferences are not supported, to gain on speed and safety.
No ORM libraries. Just good old (safely-escaped) SQL queries.

🧘 Safety

Binary or too big responses will not be read.
Safe regex thanks to google-re2 regex.
Safe escaping of SQL queries with psycopg.
Security static analysis with bandit, snyk, and GitHub CodeQL.
Code is fully type-annotated and type-checked with mypy.
Further type checking with Pydantic (v2).
Further static analysis with pyflakes.

Install

Via pip

# You need to run this with a Python 3.11 environment -- You can manage different python versions for instance with `pyenv`
pip install -U fastchecks

or via source

You need to have python poetry installed. Then:

# clone this repository
_reponame="fastchecks";
_branch="main";
git clone -b "${_branch}" "https://github.com/juanmirocks/${_reponame}";
cd ${_reponame}

# Install project
poetry install
# Enter into the project's shell environment for simplicity with the running commands
poetry shell

Run

Create a postgres DB and take note of its postgres URL conninfo. NOTE: this app was tested with Postgres v15 only; some older versions should work too.

For instance, if you have a local postgres installation:

_dbname="fastchecks";
createdb "${_dbname}"

# Its postgres URL (assuming default user) will be:
# postgres://localhost/fastchecks

# Then you need to pass the conninfo to the CLI,
# * either with the explicit optional parameter `--pg_conninfo`, or
# * by setting the envar: `FC_POSTGRES_CONNINFO`
# For simplicity, commands below assume you've set `FC_POSTGRES_CONNINFO`, e.g.:
export FC_POSTGRES_CONNINFO='postgres://localhost/fastchecks'

Add some website URLs to later check for:

fastchecks upsert_check 'https://example.org'  # Add a simple URL check
fastchecks upsert_check 'https://example.org' --regex 'Example D[a-z]+'  # Update the URL check to match the response body with a regex
fastchecks upsert_check 'https://python.org' --interval 5  # Add another URL check with a specific interval (in seconds)

Run the checks at the scheduled intervals in the foreground until stopped.

fastchecks check_all_loop_fg  # checks without interval will run with a default (configurable; see command help)

That's it! You might want to explore further options:
- For all possibilities, run: fastchecks -h
- For instance, might you want to run all checks only once (e.g. to schedule with cron), run: fastchecks check_all_once
- Or run a single website check once (without registering it): fastchecks check_website 'https://www.postgresql.org/'

Copyright / License

SPDX-License-Identifier: Apache-2.0

See files: LICENSE & NOTICE.