winevtx: unable to parse evtx files without dependency identifier and string integer and FILETIME values
Closed this issue · 5 comments
tomchop commented
Describe the problem:
log2timeline is unable to parse the evtx files provided in the FCSC cybersecurity challenge (run by ANSSI). More info: france-cybersecurity-challenge.fr (seems to be in french only)
To Reproduce:
The version of Plaso you used:
The operating system you are running Plaso on (Not the operating system of the image/files you're trying to analyze):
For example: Docker images.
Steps to reproduce the behavior including command line and arguments and output:
wget https://files.france-cybersecurity-challenge.fr/dl/soc-simulator/soc_events.zip
unzip soc_events.zip
docker run --rm -v "$(pwd):/data" -w /data log2timeline/plaso log2timeline soc_events --storage_file soc.plaso
docker run --rm -v "$(pwd):/data" -w /data log2timeline/plaso psort soc.plaso -w soc.csv
Expected behavior:
The EVTX files being parsed by plaso
Actual behavior:
Only the filestat events get output in the CSV file:
$ wc -l soc.csv
1354 soc.csv
$ grep -v filestat soc.csv
datetime,timestamp_desc,source,source_long,message,parser,display_name,tag
Debug output/tracebacks:
the pinfo output of the plaso file
docker run --rm -v "$(pwd):/data" -w /data log2timeline/plaso pinfo soc.plaso
************************** Plaso Storage Information ***************************
Filename : soc.plaso
Format version : 20230327
Serialization format : json
--------------------------------------------------------------------------------
*********************************** Sessions ***********************************
5c28fe93-ac0c-433b-ab6d-e2d31d6b6f1b : 2024-04-16T13:36:48.806632+00:00
--------------------------------------------------------------------------------
******************************** Event sources *********************************
Total : 451
--------------------------------------------------------------------------------
************************* Events generated per parser **************************
Parser (plugin) name : Number of events
--------------------------------------------------------------------------------
filestat : 1353
Total : 1353
--------------------------------------------------------------------------------
No events labels stored.
******************* Extraction warnings generated per parser *******************
Parser (plugin) name : Number of warnings
--------------------------------------------------------------------------------
winevtx : 1698310
--------------------------------------------------------------------------------
************** Path specifications with most extraction warnings ***************
Number of warnings : Pathspec
--------------------------------------------------------------------------------
8607 : type: OS, location: /data/soc_events/20220706T143356.evtx
6925 : type: OS, location: /data/soc_events/20220706T095008.evtx
6706 : type: OS, location: /data/soc_events/20220706T130407.evtx
6680 : type: OS, location: /data/soc_events/20220705T151955.evtx
6676 : type: OS, location: /data/soc_events/20220705T134811.evtx
6660 : type: OS, location: /data/soc_events/20220705T134544.evtx
6657 : type: OS, location: /data/soc_events/20220705T132936.evtx
6656 : type: OS, location: /data/soc_events/20220705T133524.evtx
6655 : type: OS, location: /data/soc_events/20220705T130540.evtx
6652 : type: OS, location: /data/soc_events/20220705T143505.evtx
--------------------------------------------------------------------------------
******************** Recovery warnings generated per parser ********************
Parser (plugin) name : Number of warnings
--------------------------------------------------------------------------------
winevtx : 51066
--------------------------------------------------------------------------------
*************** Path specifications with most recovery warnings ****************
Number of warnings : Pathspec
--------------------------------------------------------------------------------
567 : type: OS, location: /data/soc_events/20220704T215510.evtx
552 : type: OS, location: /data/soc_events/20220706T105527.evtx
539 : type: OS, location: /data/soc_events/20220704T165547.evtx
536 : type: OS, location: /data/soc_events/20220706T011031.evtx
526 : type: OS, location: /data/soc_events/20220705T142203.evtx
523 : type: OS, location: /data/soc_events/20220706T145524.evtx
500 : type: OS, location: /data/soc_events/20220706T032138.evtx
498 : type: OS, location: /data/soc_events/20220704T204049.evtx
475 : type: OS, location: /data/soc_events/20220705T231039.evtx
463 : type: OS, location: /data/soc_events/20220705T021043.evtx
--------------------------------------------------------------------------------
No analysis reports stored.
tomchop commented
Warnings contain
************************** Extraction warning: 210200 **************************
Message : unable to parse event record: 868 with error:
pyevtx_file_get_record_by_index: unable to retrieve
record: 868. libfwevt_xml_document_read_element: invalid
element data offset value out of bounds.
libfwevt_xml_document_read_fragment: unable to read
element. libfwevt_xml_document_read_with_template_values:
unable to read fragment header.
libfwevt_xml_document_read: unable to read XML document.
libevtx_record_values_read_xml_document: unable to read
binary XML document. libevtx_io_handle_read_chunk: unable
to read record values XML document.
libfdata_list_get_element_value: unable to read element
data at offset: 11511464 (0x00afa6a8).
libfdata_list_get_element_value_by_index: unable to
retrieve element value. libevtx_file_get_record_by_index:
unable to retrieve record values: 868.
Parser chain : winevtx
Path specification : type: OS, location: /data/soc_events/20220705T160016.evtx
joachimmetz commented
Looking at one file (20220706T143356.evtx), it looks like it is "dirty"
evtxinfo 20220706T143356.evtx
evtxinfo 20240204
Windows Event Viewer Log (EVTX) information:
Version : 3.1
Number of records : 8607
Number of recovered records : 27
Flags:
Is dirty
ibfwevt_xml_document_read_fragment_header: data offset : 0x00000218
libfwevt_xml_document_read_fragment_header: fragment header data:
00000000: 0f 01 01 00 ....
libfwevt_xml_document_read_fragment_header: type : 0x0f
libfwevt_xml_document_read_fragment_header: major version : 1
libfwevt_xml_document_read_fragment_header: minor version : 1
libfwevt_xml_document_read_fragment_header: flags : 0x00
libfwevt_xml_document_read_element: data offset : 0x0000021c
libfwevt_xml_document_read_element: element data:
00000000: 41 ac 0c 00 00 25 02 A....%.
libfwevt_xml_document_read_element: type : 0x41
libfwevt_xml_document_read_element: dependency identifier : 3244 (0x0cac)
libfwevt_xml_document_read_element: size : 35979264
libfwevt_xml_document_read_element: name offset : 0x00000000
Given what is known about the format the element data size 00 00 25 02 is off
Test skipping dependency identifier
libfwevt_xml_document_read_element: data offset : 0x0000021c
libfwevt_xml_document_read_element: element data:
00000000: 41 ac 0c 00 00 A....
libfwevt_xml_document_read_element: type : 0x41
libfwevt_xml_document_read_element: size : 3244
libfwevt_xml_document_read_element: name offset : 0x00000225
libfwevt_xml_document_read_name: data offset : 0x00000225
libfwevt_xml_document_read_name: name header data:
00000000: 00 00 00 00 ba 0c 05 00 ........
This does not appears to be related to the dirty flag being set and format version, given the dependency identifier appears to be missing from all records
joachimmetz commented
Also looks like SystemTime is a string and not a FILETIME
From: 20220706T143356.evtx)
libfwevt_xml_tag_name_debug_print: name : SystemTime
libfwevt_xml_document_read_value: data offset : 0x0000051d
libfwevt_xml_document_read_value: value data:
00000000: 05 01 1e 00 ....
libfwevt_xml_document_read_value: type : 0x05
libfwevt_xml_document_read_value: value type : 0x01 (UTF-16 string)
libfwevt_xml_document_read_value: number of characters : 30
libfwevt_xml_document_read_value: data offset : 0x00000521
libfwevt_xml_document_read_value: value data:
00000000: 32 00 30 00 32 00 32 00 2d 00 30 00 37 00 2d 00 2.0.2.2. -.0.7.-.
00000010: 30 00 36 00 54 00 31 00 32 00 3a 00 32 00 34 00 0.6.T.1. 2.:.2.4.
00000020: 3a 00 34 00 30 00 2e 00 36 00 30 00 38 00 31 00 :.4.0... 6.0.8.1.
00000030: 31 00 35 00 35 00 30 00 30 00 5a 00 1.5.5.0. 0.Z.
libfwevt_xml_tag_value_debug_print: value : 2022-07-06T12:24:40.608115500Z
From Application.evtx
libfwevt_xml_tag_name_debug_print: name : SystemTime
libfwevt_xml_document_read_optional_substitution: data offset : 0x000005a5
libfwevt_xml_document_read_optional_substitution: optional substitution data:
00000000: 0e 06 00 11 ....
libfwevt_xml_document_read_optional_substitution: type : 0x0e
libfwevt_xml_document_read_optional_substitution: identifier : 6
libfwevt_xml_document_read_optional_substitution: value type : 0x11 (Filetime)
libfwevt_xml_document_substitute_template_value: value: 06 offset : 0x0000081b
libfwevt_xml_document_substitute_template_value: value: 06 size : 8
libfwevt_xml_document_substitute_template_value: value: 06 type : 0x11 (Filetime)
libfwevt_xml_document_substitute_template_value: value: 06 data:
00000000: e8 7d ac a5 3a d0 d6 01 .}..:...
libfwevt_xml_tag_value_debug_print: value : 2020-12-12T03:55:36.023396000Z
Looks like various other values like version are stored as strings as well.
joachimmetz commented
@tomchop it would be good to understand which tool / setting created these evtx files given they are in a slightly different format.
joachimmetz commented
Support for variant of evtx added to libevtx 20240427, closing issue.