This is an implementation of a Certificate Transparency monitor. It intends to be compliant with RFC6962, and all subsequent published CT-related specifications.

Installation

You'll need a fair few bits and pieces to make this all work:

Ruby 2.1 or later;
PostgreSQL 9.4 or later;
An AMQP server, such as RabbitMQ;
A lot of diskspace, if you want to monitor existing logs.

Configuration

Configu is all done via environment variables (in line with the principles of the 12 Factor App). The following env vars are available:

AMQP_URL (default `"amqp://guest:guest@localhost")

The URL to an AMQP server (such as RabbitMQ).
BROWN_LOG_LEVEL (default "info")

The log level of the agent framework (and, as a side-effect, the web application). Set this to, say, "debug" in order to get more detailed log messages.
DATABASE_URL (default "postgres:///ctmonitor_development")

A Sequel-compatible URL pointing to a PostgreSQL 9.4 or later database. You include all parameters (username, password, etc) in the URL.
DEBUG_SQL (default "")

If set, then all sorts of debugging into will be spewed out about the SQL commands being sent to the database. You'll rarely need this, and hopefully only in development.
MAIL_EXCEPTIONS_TO (default nil)

If set, and RACK_ENV is set to something other than "development", then all exceptions generated by the webapp will be e-mailed to the address (or addresses, separated by commas) given in this env var.
RACK_ENV (default "development")

Specify the Rack-level environment. If left as the default, various debugging-related things are enabled which you almost certainly don't want to have enabled in production.
REDIS_URL (default "")

If set, the agent which calculates Merkle Tree hashes (to compare against the hashes provided by logs in their STHs) will cache intermediate results in the specified Redis datastore. You really, really want this if you're running a real-world monitor. Without it, verifying each tree head of a decent-sized log will take 8-12 hours (and we're retrieving them every minute!). Also, you want to allow at least 4GB of memory, and an allkeys-lru eviction policy.

Deployment

You'll need a few hundred GB of fast disk for the database, a few dozen GB of fast disk for RabbitMQ, and some way to start and restart the agent processes. In a pinch, you can just run foreman, but you'll probably want to setup something more robust (investigate foreman export).

Get your env vars configured, then setup the DB schema by running:

rake migrate_db

Finally, add one or more logs to monitor:

bin/add_log "Log Name" https://ct.example.com base64publickey==

Within a minute or so, an STH should be retrieved from the logs, and entries will start downloading. All the other agents will then spring to life and do their part.

Operation

As long as all the agents are running, an STH will be fetched for each log every minute, and any new entries will be retrieved and processed into the database.

Getting information out of the database is up to you. This framework provides no particular facilities for analysing the collected data, as it is assumed that users will have many and varying analysis requirements. You can write additional agents to do further processing on the database contents, or web applications in your choice of language for interactive access.

Getting Help

Community Support

At present, there is no dedicated mailing list or IRC channel for this software. Try the general certificate-transparency mailing list.

Commercial Support

If you need assistance deploying or operating your monitor, please contact me to discuss your requirements.

Contributing

Bug reports and patches should be sent to the Github issue tracker.

Licence

Unless otherwise stated, everything in this repo is covered by the following copyright notice:

Copyright (C) 2015  Tobermory Technology Pty Ltd

This program is free software: you can redistribute it and/or modify it
under the terms of the GNU General Public License version 3, as
published by the Free Software Foundation.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.

tobermorytech/certificate-transparency-monitor