Receiving an error when converting data sources to json
RedneckHutch opened this issue · 5 comments
I am able to convert the ICS and Mobile data source to a json but am unable to convert the sample data for endpoints. Any recommendations?
root@1100301d5:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
Traceback (most recent call last):
File "/opt/DeTTECT/./dettect.py", line 424, in <module>
_menu(_init_menu())
File "/opt/DeTTECT/./dettect.py", line 304, in _menu
generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, layer_settings)
File "/opt/DeTTECT/data_source_mapping.py", line 199, in generate_data_sources_layer
my_techniques = _map_and_colorize_techniques(my_data_sources, systems, exceptions, domain, layer_settings)
File "/opt/DeTTECT/data_source_mapping.py", line 73, in _map_and_colorize_techniques
applicable_data_sources = get_applicable_data_sources_platform(system['platform'], domain)
File "/opt/DeTTECT/generic.py", line 642, in get_applicable_data_sources_platform
applicable_data_sources.update(data_sources[p])
KeyError: 'Windows'
hi @RedneckHutch,
Can you try to clear the cache directory and try it again?
And if it still fails, can you please check the content of data/data_source_platforms.json
and check if you see the "ATT&CK Enterprise" entry and within that the "Windows" entry?
root@1100301d1075:/opt/DeTTECT# rm -r ~/.cache/*
root@1100301d1075:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
Traceback (most recent call last):
File "/opt/DeTTECT/./dettect.py", line 424, in <module>
_menu(_init_menu())
File "/opt/DeTTECT/./dettect.py", line 304, in _menu
generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, layer_settings)
File "/opt/DeTTECT/data_source_mapping.py", line 199, in generate_data_sources_layer
my_techniques = _map_and_colorize_techniques(my_data_sources, systems, exceptions, domain, layer_settings)
File "/opt/DeTTECT/data_source_mapping.py", line 73, in _map_and_colorize_techniques
applicable_data_sources = get_applicable_data_sources_platform(system['platform'], domain)
File "/opt/DeTTECT/generic.py", line 642, in get_applicable_data_sources_platform
applicable_data_sources.update(data_sources[p])
KeyError: 'Windows'
It doesn't look like windows in included.
{
"ATT&CK-Enterprise": {},
"ATT&CK-ICS": {
"Field Controller/RTU/PLC/IED": [
"Application Log Content",
"Asset Inventory",
"Command Execution",
"Device Alarm",
"File Access",
"File Deletion",
"File Metadata",
"File Modification",
"Firmware Modification",
"Logon Session Creation",
"Logon Session Metadata",
"Network Traffic Content",
"Network Traffic Flow",
"OS API Execution",
"Process Creation",
"Process History/Live Data",
"Process Termination",
"Process/Event Alarm",
"Script Execution",
"Service Creation",
"Software",
"User Account Authentication",
"Windows Registry Key Modification"
],
"Safety Instrumented System/Protection Relay": [
"Application Log Content",
"Asset Inventory",
"Command Execution",
"Device Alarm",
"File Access",
"File Deletion",
"File Metadata",
Added Windows.
{
"ATT&CK-Enterprise": {
"Windows": []
},
"ATT&CK-ICS": {
"Field Controller/RTU/PLC/IED": [
"Application Log Content",
"Asset Inventory",
Now get the following
root@1100301d1075:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
Traceback (most recent call last):
File "/opt/DeTTECT/./dettect.py", line 424, in <module>
_menu(_init_menu())
File "/opt/DeTTECT/./dettect.py", line 304, in _menu
generate_data_sources_layer(file_ds, args.output_filename, args.layer_name, layer_settings)
File "/opt/DeTTECT/data_source_mapping.py", line 199, in generate_data_sources_layer
my_techniques = _map_and_colorize_techniques(my_data_sources, systems, exceptions, domain, layer_settings)
File "/opt/DeTTECT/data_source_mapping.py", line 73, in _map_and_colorize_techniques
applicable_data_sources = get_applicable_data_sources_platform(system['platform'], domain)
File "/opt/DeTTECT/generic.py", line 642, in get_applicable_data_sources_platform
applicable_data_sources.update(data_sources[p])
KeyError: 'Linux'
I am going to pull down a new data/data_source_platforms.json
file.
Update:
Replacing data/data_source_platforms.json
with an updated copy appears to have resolved the issue.
root@1100301d1075:/opt/DeTTECT# python3 ./dettect.py ds -fd sample-data/data-sources-endpoints.yaml -l --health
File written: output/data_sources_data-sources-sample.json
Great! The data_source_platforms.json is auto generated by pipelines and I see that for a short time this file was not filled for ATT&CK-Enterprise, maybe beause it couldn't fetch the data from MITRE at that moment of time. Good action to resolve the action by getting the latest version from the repo.