AttributeError: 'NullTypeNode' object has no attribute 'find_end_of_stream'
linedash opened this issue · 7 comments
Hi,
When attempting to parse an evtx file as gathered by a collector on a Win server 2012 R2 box ; I get the following errors:
./evtx_dump.py Archive-ForwardedEvents-2017-07-19-09-54-122.evtx
<?xml version="1.1" encoding="utf-8" standalone="yes" ?>
<Events>
Traceback (most recent call last):
File "./evtx_dump.py", line 46, in <module>
main()
File "./evtx_dump.py", line 41, in main
print(record.xml())
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Evtx.py", line 481, in xml
return e_views.evtx_record_xml_view(self)
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Views.py", line 204, in evtx_record_xml_view
return render_root_node(record.root())
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Views.py", line 182, in render_root_node
for sub in root_node.substitutions():
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/BinaryParser.py", line 64, in __call__
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 1001, in substitutions
ofs = self.tag_and_children_length()
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 962, in tag_and_children_length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/BinaryParser.py", line 64, in __call__
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 952, in children
return self._children(end_tokens=[SYSTEM_TOKENS.EndOfStreamToken])
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 159, in _children
ofs += child.length()
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/BinaryParser.py", line 64, in __call__
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 177, in length
for child in self.children():
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/BinaryParser.py", line 64, in __call__
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 334, in children
SYSTEM_TOKENS.CloseEmptyElementToken])
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 162, in _children
if child.find_end_of_stream():
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/BinaryParser.py", line 64, in __call__
cache[key] = self.func(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/python_evtx-0.6.0-py2.7.egg/Evtx/Nodes.py", line 186, in find_end_of_stream
ret = child.find_end_of_stream()
AttributeError: 'NullTypeNode' object has no attribute 'find_end_of_stream'
running Python 2.7.9 on Debian 8. I installed the current version of python-evtx using the setup.py utility from the github download. Please let me know if you need any more details.
Hi @linedash
Thanks for taking the time to report this issue!
Are you able to share an evtx file that demonstrates this issue? I've learned that its critical to add unit and regression tests as we find issues to 1) demonstrate bugs, and 2) ensure they don't creep back in. I can always try to blindly fix a bug, but I strongly prefer not to do this.
Would it be possible to get an email address or somewhere I could send one a link to one? I wouldn't be comfortable releasing it into the wild due to the nature of the data.
I'll see if I can get someone to sign off on me releasing the data in the meantime.
Hello,
After checking couple of document. It looks like for this error there are some new specifications. Is it possible to let me know which file I have to change to handle this exception.
In the new RPC document published by microsoft.
https://msdn.microsoft.com/en-us/library/cc231282.aspx
I am seeing a section which could possibly fix this issue.
Before emitting anything, the tool SHOULD verify that the attribute data, as specified by the
AttributeCharData rule in 2.2.12, is not empty. If the attribute data is empty, the tool SHOULD NOT
emit anything. If the attribute data is not empty, emit the space character " " and the text, as
specified by the Name rule in 2.2.12, the character "=", the character "'", the text, as specified by the
AttributeCharData rule in 2.2.12, and, finally, the character "'".
Any help to fix this would be wonderful.
Thank you in advance for taking a looking into this.
Hi @williballenthin,
I have been trying to convert the .evtx logs to XML. With the help of your code, I am able to convert a set of logs and few logs I'm not able to convert. I'm getting this error :
Traceback (most recent call last):
File "<pyshell#19>", line 1, in
EvtxToElk.evtx_to_elk("C:\Users\517214\Downloads\Archive-ForwardedEvents-2018-01-09.evtx",'http://localhost:9200')
File "C:\Python36\lib\site-packages\evtxtoelk_init_.py", line 34, in evtx_to_elk
for xml, record in evtx_file_xml_view(fh):
File "C:\Python36\lib\site-packages\Evtx\Views.py", line 240, in evtx_file_xml_view
record_str = evtx_record_xml_view(record)
File "C:\Python36\lib\site-packages\Evtx\Views.py", line 204, in evtx_record_xml_view
return render_root_node(record.root())
File "C:\Python36\lib\site-packages\Evtx\Views.py", line 182, in render_root_node
for sub in root_node.substitutions():
File "C:\Python36\lib\site-packages\Evtx\BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 1001, in substitutions
ofs = self.tag_and_children_length()
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 962, in tag_and_children_length
for child in self.children():
File "C:\Python36\lib\site-packages\Evtx\BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 952, in children
return self._children(end_tokens=[SYSTEM_TOKENS.EndOfStreamToken])
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 159, in _children
ofs += child.length()
File "C:\Python36\lib\site-packages\Evtx\BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 177, in length
for child in self.children():
File "C:\Python36\lib\site-packages\Evtx\BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 334, in children
SYSTEM_TOKENS.CloseEmptyElementToken])
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 162, in _children
if child.find_end_of_stream():
File "C:\Python36\lib\site-packages\Evtx\BinaryParser.py", line 64, in call
cache[key] = self.func(*args, **kw)
File "C:\Python36\lib\site-packages\Evtx\Nodes.py", line 186, in find_end_of_stream
ret = child.find_end_of_stream()
AttributeError: 'NullTypeNode' object has no attribute 'find_end_of_stream'
Please help me in resolving the issue...
I just triggered this same issue on a file from a memory sample.
I've been trying to parse some .evtx aswell and experienced the same behaviour. From what I've tried and what I see in the comments above (the names of the test files of @sanju1323 and @linedash ), it seems to be an issue with the ForwardedEvents type. I looked at the structure of the BinXML of this type of events and it seems to be quite different, but I'm unable to adapt the library so far.
On an Application .evtx I don't trigger the error :
evtx_structure.py Application.evtx | head -50
File header
magic: ElfFile
oldest_chunk: 0x0
current_chunk_number: 0x7e
next_record_number: 0x4ecf
header_size: 0x80
minor_version: 0x1
major_version: 0x3
header_chunk_size: 0x1000
chunk_count: 0x7f
flags: 0x1
checksum: 0x87589e25
verify: True
dirty: True
full: False
Chunk
offset: 0x1000
magic: ElfChnk
file_first_record_number: 0x1
file_last_record_number: 0x74
log_first_record_number: 0x1
log_last_record_number: 0x74
header_size: 0x80
last_record_offset: 0xfdf8
next_record_offset: 0xfec8
data_checksum: 0xe8788113
header_checksum: 0x51b42232
verify: True
templates: 0
Record
offset: 0x1200
magic: 0x2a2a
size: 0x7b8
number: 0x1
timestamp: 2012-07-09 07:02:00
verify: True
RootNode(offset=0x18)
StreamStartNode(offset=0x18)
TemplateInstanceNode(offset=0x1c, resident=True, length=0x575)
TemplateNode(offset=0x26)
StreamStartNode(offset=0x3e)
OpenStartElementNode(offset=0x42) --> Event
AttributeNode(offset=0x65) --> xmlns
ValueNode(offset=0x7e)
WstringTypeNode(offset=0x80) --> http://schemas.microsoft.com/win/2004/08/events/event
CloseStartElementNode(offset=0xec)
OpenStartElementNode(offset=0xed) --> System
CloseStartElementNode(offset=0x10e)
OpenStartElementNode(offset=0x10f) --> Provider
AttributeNode(offset=0x138) --> Name
[...]On a ForwardedEvent I trigger errors :
evtx_structure.py Forward.evtx | head -50
File header
magic: ElfFile
oldest_chunk: 0x0
current_chunk_number: 0x10
next_record_number: 0xf0
header_size: 0x80
minor_version: 0x1
major_version: 0x3
header_chunk_size: 0x1000
chunk_count: 0x11
flags: 0x0
checksum: 0xf278fc1e
verify: True
dirty: False
full: False
Chunk
offset: 0x1000
magic: ElfChnk
file_first_record_number: 0x1
file_last_record_number: 0xa
log_first_record_number: 0x1
log_last_record_number: 0xa
header_size: 0x80
last_record_offset: 0xc628
next_record_offset: 0xea10
data_checksum: 0xea19cd0c
header_checksum: 0x5e0c09b8
verify: True
templates: 0
Record
offset: 0x1200
magic: 0x2a2a
size: 0x1400
number: 0x1
timestamp: 2019-04-10 10:42:26.529150
verify: True
RootNode(offset=0x18)
ERROR: 'NullTypeNode' object has no attribute 'find_end_of_stream'
Record
offset: 0x2600
magic: 0x2a2a
size: 0x1148
number: 0x2
timestamp: 2019-04-10 10:42:26.529150
verify: True
RootNode(offset=0x18)
ERROR: Tried to parse beyond the end of the file (read: 0x731006, buffer length: 0x111000)
Record
offset: 0x3748
magic: 0x2a2a
size: 0x1148
number: 0x3
timestamp: 2019-04-10 10:42:26.529150
verify: True
RootNode(offset=0x18)
ERROR: Tried to parse beyond the end of the file (read: 0x731006, buffer length: 0x111000)
Record
offset: 0x4890
magic: 0x2a2a
size: 0x1148
number: 0x4
timestamp: 2019-04-10 10:42:26.529150
verify: True
RootNode(offset=0x18)
ERROR: Tried to parse beyond the end of the file (read: 0x731006, buffer length: 0x111000)
Record
offset: 0x59d8
magic: 0x2a2a
size: 0x1148
number: 0x5
[...]Hope it helps resolving this issue