Vulnerability Database

Persistent store for vulnerabilities data obtained from Vulcan scans.

The Vulnerability Database acts as a worker which reads from a queue containing the checks status changes and data, it then processes this data, including checks report, in order to maintain a historic representation of vulnerabilities lifecycle, affected assets, executed checks, etc.

For running the component locally, clone and run at the root of the repo the following:

go install ./...
cd db && source postgres-start.sh && cd -
cd db && source flyway-migrate.sh && cd -
vulnerability-db-consumer -c _resources/config/local.toml

How to run the Vulnerability DB in development mode

You can test the Vulnerability DB Consumer locally in your machine. The commands bellow will launch the necessary components required by the application.

# Navigate to the local_deployment folder
cd local_deployment

# Start the dependencies
docker-compose up -d

# Build and run the vulnerability-db-consumer
./start.sh

You can test that everything works by sending a message to the mocked SNS topic, using the AWS cli:

AWS_ACCESS_KEY_ID=fake AWS_SECRET_ACCESS_KEY=fake aws sns publish \
    --region local \
    --endpoint-url http://localhost:4100  \
    --topic-arn arn:aws:sns:local:012345678900:VulcanLocalhostChecks \
    --message '
        {
            "status":"FINISHED",
            "id":"old-model-happy-path-01",
            "tag":"team:test",
            "target":"api.example.com",
            "checktype_name":"vulcan-http-headers",
            "report":"http://localhost:8080/old-model-happy-path-01.json"
        }' \
    --message-attributes '{"status":{"DataType":"String","StringValue":"FINISHED"}}'

To stop the dependencies, run:

docker-compose down --remove-orphans

To purge local mocked SQS queue:

AWS_ACCESS_KEY_ID=fake AWS_SECRET_ACCESS_KEY=fake aws sqs purge-queue \
    --region local \
    --endpoint-url http://localhost:4100 \
    --queue-url http://localhost:4100/012345678900/VulcanLocalhostVulnDBChecks

Docker execute

Those are the variables you have to use:

Variable Description Sample
MAX_EVENT_AGE Defines the max age for which check events are processed. Older events are discarded 365
LOG_LEVEL error
PG_HOST Database host localhost
PG_NAME Database name vulnerabilitydb
PG_USER Database user vulnerabilitydb
PG_PASSWORD Database password vulnerabilitydb
PG_PORT Database port 5432
PG_SSLMODE One of these (disable,allow,prefer,require,verify-ca,verify-full) disable
PG_CA_B64 A base64 encoded CA certificate
SQS_QUEUE_ARN Checks queueu ARN arn:aws:sqs:xxx:123456789012:yyy
SNS_TOPIC_ARN ARN of topic to publish new vulnerabilities arn:aws:sns:xxx:123456789012:yyy
RESULTS_URL External vulcan-results URL https://results.vulcan.com
RESULTS_INTERNAL_URL Internal vulcan-results URL http://vulcan-results
AWS_SQS_ENDPOINT Endpoint for SQS creation queue (optional) http://custom-aws-endpoint
AWS_SNS_ENDPOINT Endpoint for SNS topic (optional) http://custom-aws-endpoint
docker build . -t vdb

# Use the default config.toml customized with env variables.
docker run --env-file ./local.env -e AWS_SECRET_ACCESS_KEY -e AWS_ACCESS_KEY_ID  --name vdb --rm vdb

# Use custom config.toml
docker run -v `pwd`/custom.toml:/app/config.toml vdb