muchdogesec/stix2arango

stix2arango is a command line tool that takes a group of STIX 2.1 objects in a bundle and inserts them into ArangoDB. It can also handle updates to existing objects in ArangoDB imported in a bundle.

stix2arango

Overview

stix2arango is a command line tool that takes a group of STIX 2.1 objects in a bundle and inserts them into ArangoDB. It can also handle updates to existing objects in ArangoDB imported in a bundle.

1. STIX 2.1 bundle entered
2. User chooses database/collection names (stix2arango creates as needed)
3. stix2arango inserts objects (or updates them) and then generates any relationships between them

tl;dr

Watch the demo.

Usage

Install the script

# clone the latest code
git clone https://github.com/muchdogesec/stix2arango
# create a venv
cd stix2arango
python3 -m venv stix2arango-venv
source stix2arango-venv/bin/activate
# install requirements
pip3 install -r requirements.txt

Note, the installation assumes ArangoDB is already installed locally.

You can install ArangoDB here. stix2arango is compatible with both the Enterprise and Community versions.

A note for Mac users

Fellow Mac users, ArangoDB can be installed and run using homebrew as follows;

## Install
brew install arangodb
## Run
brew services start arangodb
## will now be accessible in a browser at: http://127.0.0.1:8529 . Default username is root with no password set (leave blank) 
## Stop
brew services stop arangodb

Sorry for the lack of other OS's -- I use a Mac.

Setup configoration options

You will need to create an .env file as follows;

cp .env.example .env

You will then need to specify details of your ArangoDB install (host, user, and password). It is important the user chosen has the ability to write/update new databases, collections and records.

Run

python3 stix2arango.py \
	--file PATH/TO/STIX.json \
	--database NAME \
	--collection NAME \
	--stix2arango_note SOMETHING \
	--ignore_embedded_relationships BOOLEAN

Where;

• --file (required): is the path to the valid STIX 2.1 bundle .json file
• --database (required): is the name of the Arango database the objects should be stored in. If database does not exist, stix2arango will create it
• --collection (required): is the name of the Arango collection in the database specified the objects should be stored in. If the collection does not exist, stix2arango will create it
• --stix2arango_note (optional): Will be stored under the _stix2arango_note custom attribute in ArangoDB. Useful as can be used in AQL. a-z characters only. Max 24 chars.
• --ignore_embedded_relationships (optional): boolean, if true passed, this will stop any embedded relationships from being generated. Default is false

For example, using the MITRE ATT&CK Enterprise bundle;

python3 stix2arango.py \
	--file cti_knowledge_base_store/mitre-attack-enterprise/enterprise-attack-15_1.json \
	--database stix2arango_demo \
	--collection demo_1 \
	--stix2arango_note v15.1 \
	--ignore_embedded_relationships true

If you want to include embedded relationships as edges in the ArangoDB collection, you would run;

python3 stix2arango.py \
	--file cti_knowledge_base_store/mitre-attack-enterprise/enterprise-attack-15_1.json \
	--database stix2arango_demo \
	--collection demo_2 \
	--stix2arango_note v15.1 \
	--ignore_embedded_relationships false

A note on embedded relationships

stix2arango can handle all embedded references to other STIX objects under _ref and _refs properties in a STIX object when --ignore_embedded_relationships is set to false.

e.g.

        {
            "type": "x-mitre-tactic",
            "spec_version": "2.1",
            "id": "x-mitre-tactic--b977ad29-eb0c-5f09-bb2f-6d3f23e2a175",
            "created_by_ref": "identity--8700e156-6ce9-5090-8589-f9d0aef7bdb7",

For the above object a STIX SRO would be generated by stix2arango with source_ref=x-mitre-tactic--b977ad29-eb0c-5f09-bb2f-6d3f23e2a175, target_ref=identity--8700e156-6ce9-5090-8589-f9d0aef7bdb7 and relationship_type=created-by.

Quickstart

We store a lot of STIX data from popular knowledgebases on Cloudflare R2.

This is a useful resource to quickly populate STIX data using stix2arango if you want to see what it can do. It is also what is used to populate the data required by arango_cti_processor

To populate this data for yourself, check out utilities/README.md

Useful supporting tools

• To generate STIX 2.1 Objects: stix2 Python Lib
• The STIX 2.1 specification: STIX 2.1 docs
• ArangoDB docs

Support

Minimal support provided via the DOGESEC community.

License

Apache 2.0.