SightingDB is a database designed for Sightings, a technique to count items. This is helpful for Threat Intelligence as Sightings allow to enrich indicators or attributes with Observations, rather than Reputation.

Simply speaking, by pushing data to SightingDB, you will get the first time it was observed, the last time, its count.

However, it will also provide the following features:

Keep track of how many times something was searched
Keep track of the hourly statistics per item
Get the consensus for each item (how many times the same value exists in another namespace)

SightingDB is designed to scale writing and reading.

Building

Make sure you have Rust and Cargo installed
Run ''make''

Running

To run from the source directory:

Generate a certificate: cd etc; mkdir ssl; cd ssl; openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout key.pem -out cert.pem; cd ../..
ln -s etc/ssl ssl
ln -s etc/sighting-daemon.ini sighting-daemon.ini
Start the Sighting DB: ./target/debug/sighting-daemon

Client Demo

Writing

$ curl -k https://localhost:9999/w/my/namespace/?val=127.0.0.1
{"message":"ok"}	
$ curl -k https://localhost:9999/w/another/namespace/?val=127.0.0.1
{"message":"ok"}
$ curl -k https://localhost:9999/w/another/namespace/?val=127.0.0.1
{"message":"ok"}

Reading

$ curl -k https://localhost:9999/r/my/namespace/?val=$(b64 127.0.0.1)
{"value":"127.0.0.1","first_seen":1566624658,"last_seen":1566624658,"count":1,"tag":"","ttl":0,"consensus":2}

$ curl -k https://localhost:9999/r/another/namespace/?val=127.0.0.1
{"value":"127.0.0.1","first_seen":1566624686,"last_seen":1566624689,"count":2,"tag":"","ttl":0,"consensus":2}

$ curl -k https://localhost:9999/rs/my/namespace/?val=127.0.0.1
{"value":"127.0.0.1","first_seen":1593719022,"last_seen":1593721509,"count":10,"tags":"","ttl":0,"stats":{"1593716400":2,"1593720000":8},"consensus":1}

Authentication

$ curl -H 'Authorization: changeme' -k https://localhost:9999/w/my/namespace/?val=127.0.0.1
{"message":"ok"}

REST Endpoints

/w: write (GET)
/wb: write in bulk mode (POST)
/r: read (GET)
/rs: read with statistics (GET)
/rb: read in bulk mode (POST)
/rbs: read with statistics in bulk mode (POST)
/d: delete (GET)
/c: configure (GET)
/i: info (GET)