The repository has been migrated to vmware-labs: https://github.com/vmware-labs/feed-manager-for-misp
Utilities and classes to generate and consume MISP feeds.
We support two types of feeds:
- Indicators feeds: made of simple objects, like hashes, domains, etc; this is the basic feed type we use to share labelled indicators.
- Telemetry feeds: made of complex objects coming from our telemetry; each item has multiple indicators associated (for example md5 and sha1) and can contain complex objects (for example the list of behaviors associated to a sandbox analysis).
Below we give an example of both. The generate_feed.py provides an example of how both feeds
can be generated:
./bin/generate_feed.py -o ./tmp/
> Daily feed of indicators written to: ./tmp/indicators
> Daily feed of telemetry objects written to: ./tmp/telemetryConsuming an indicator feed extracts all attributes and print them as separate entities; note that it is still possible to group them by object (file) as the object uuid is not discarded and included in the provided output; this is useful because, for example, many hashes might describe the same file.
./bin/consume_feed.py -i ./tmp/indicators
> Fetching items since 2022-08-20 13:19:04.856733
> {
> "tags": [
> "misp-galaxy:malpedia=\"GootKit\"",
> "misp-galaxy:threat-actor=\"Sofacy\""
> ],
> "timestamp": "2022-10-11 14:01:56",
> "event_uuid": "ca324c99-a9d2-45e0-947d-d864d70df9c5",
> "object_uuid": "31ae2789-392e-40a7-971b-d80ee8f78fca",
> "attribute_uuid": "0bd619cc-4692-4c5e-84fd-c45fcd0e0d93",
> "attribute_type": "md5",
> "attribute_value": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
> }
> {
> "tags": [
> "misp-galaxy:malpedia=\"GootKit\"",
> "misp-galaxy:threat-actor=\"Sofacy\""
> ],
> "timestamp": "2022-10-11 14:01:56",
> "event_uuid": "ca324c99-a9d2-45e0-947d-d864d70df9c5",
> "object_uuid": "31ae2789-392e-40a7-971b-d80ee8f78fca",
> "attribute_uuid": "6c6578a9-fd33-4ae9-8443-2bdb0435aa9f",
> "attribute_type": "sha1",
> "attribute_value": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
> }
> {
> "tags": [
> "misp-galaxy:malpedia=\"GootKit\"",
> "misp-galaxy:threat-actor=\"Sofacy\""
> ],
> "timestamp": "2022-10-11 14:01:56",
> "event_uuid": "ca324c99-a9d2-45e0-947d-d864d70df9c5",
> "object_uuid": "31ae2789-392e-40a7-971b-d80ee8f78fca",
> "attribute_uuid": "6929d4ca-3b14-4d7b-a021-f3442b0eca01",
> "attribute_type": "sha256",
> "attribute_value": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
> }Instead of further filtering and processing, it is also possible to request the attribute type at consumption time. For example, when processing the same feed we can do the following:
./bin/consume_feed.py -i ./tmp/indicators -t sha1
> Fetching items since 2022-08-20 13:23:48.005220
> {
> "tags": [
> "misp-galaxy:malpedia=\"GootKit\"",
> "misp-galaxy:threat-actor=\"Sofacy\""
> ],
> "timestamp": "2022-10-11 14:01:56",
> "event_uuid": "ca324c99-a9d2-45e0-947d-d864d70df9c5",
> "object_uuid": "31ae2789-392e-40a7-971b-d80ee8f78fca",
> "attribute_uuid": "6c6578a9-fd33-4ae9-8443-2bdb0435aa9f",
> "attribute_type": "sha1",
> "attribute_value": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
> }And finally, an example of consuming a telemetry feed:
./bin/consume_feed.py -i ./tmp/telemetry/
> Fetching items since 2022-08-20 13:12:12.802821
> {
> "tags": [],
> "techniques": [],
> "task.portal_url": "https://user.lastline.com/portal#/analyst/task/30f48c17e9db002005baa7d440ca275a/overview",
> "task.score": "70",
> "analysis.activities": [
> "Anomaly: AI detected possible malicious code reuse",
> "Evasion: Detecting the presence of AntiMalware Scan Interface (AMSI)",
> "Execution: Subject crash detected",
> "Signature: Potentially malicious application/program"
> ],
> "file.md5": "37840d4e937db0385b820d4019071540",
> "file.sha1": "a1f7670cd7da7e331db2d69f0855858985819873",
> "file.sha256": "492bfe8d2b1105ec4045f96913d38f98e30fe349ea50cc4aaa425ca289af2852",
> "file.name": "unknown"
> }