Application Hangs after Traceback errors

Question

Application Hangs after Traceback errors

RITOps opened this issue 4 years ago · 7 comments

🐛 Summary

Traceback error comes up and app seems to freeze while trying to scan files during YARA section.

To reproduce

Log into Win2012R2 server as domain admin, go to chirp directory and kick off app via Powershell (admin mode) /.chirp.exe
Left the process running overnight. Following day found app window with errors:
Traceback errors (see attached).

CHIRP process still in Task Manager, but stuck at 0% CPU utilization.

This occurs on version 1.03 and 1.04 on Win2012R2

Ran version 1.05 on Win2012R2 and getting Traceback error with Unicode errors as shown below. This is preceded by Traceback lines that are identical with each occurrence.
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4544: invalid start byte
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4447: invalid start byte
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 3871: invalid start byte

Expected behavior

Tool is expected to run to completion.

Any helpful log output or screenshots

Version 1.05
PS C:\kworking\chirp> cd..
PS C:\kworking> cd chirp1.05
PS C:\kworking\chirp1.05> ./chirp.exe
16:20:23 EVENTS Reading Windows Powershell event logs. scan.py:69
16:20:24 EVENTS Reading KernelMode event logs. scan.py:69
EVENTS Reading Application event logs. scan.py:69
16:20:25 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65
REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93
REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65
REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65
Options
REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47
REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47
REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47
YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141
'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of
procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike
Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is
going to take a while.
16:20:49 EVENTS Reading Security event logs. scan.py:69
16:29:26 YARA Beginning processing. run.py:100
Traceback (most recent call last):
File "C:\Users<account>\AppData\Local\Temp\ONEFIL3\chirp.py", line 17, in
run.run()
File "C:\Users<username>\AppData\Local\Temp\ONEFIL3\chirp\run.py", line 20, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL3\chirp\run.py", line 30, in run_plugins
File "C:\Users<username>\AppData\Local\Temp\ONEFIL3\asyncio\base_events.py", line 616, in run_until_complete
File "C:\Users<username>\AppData\Local\Temp\ONEFIL3\chirp\run.py", line 44, in _run_coroutines
File "C:\Users<username>\AppData\Local\Temp\ONEFIL3\chirp\plugins\network\scan.py", line 44, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL~3\chirp\plugins\network\network.py", line 37, in parse_dns
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4544: invalid start byte
16:35:55 YARA We're still working on scanning files. 50000 processed. run.py:96
16:40:34 YARA We're still working on scanning files. 100000 processed. run.py:96
16:43:17 YARA We're still working on scanning files. 150000 processed. run.py:96
16:45:09 YARA We're still working on scanning files. 200000 processed. run.py:96

This is another Win2012R2 server, with CHIRP v1.05 - UnicodeError 0xff in position 4447 error.
11:05:40 EVENTS Reading KernelMode event logs. scan.py:69
EVENTS Reading Application event logs. scan.py:69
11:05:41 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Wind scan.py:65
ows\CurrentVersion\sibot
REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Wind registry.py:93
ows\CurrentVersion\sibot does not exist.
REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65
REGISTRY Reading scan.py:65
HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows
NT\CurrentVersion\Image File Execution Options
REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47
REGISTRY Found 0 hit(s) for Teardrop - Registry Activity scan.py:47
indicator.
REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47
YARA Enumerating the entire filesystem due to run.py:141
['simpleseesharp : Webshell Unclassified',
'reGeorgTunnel : Webshell Commodity', 'sportsball
: Webshell', 'Detection for the use of procdump to
dump LSASS process memory.', 'CISA Solar Fire',
'CISA Teardrop', 'CrowdStrike Rempack',
'CrowdStrike Sunspot', 'FireEye Cosmic Gale',
'FireEye Sunburst']... this is going to take a
while.
11:09:35 YARA Beginning processing. run.py:100
Traceback (most recent call last):
File "C:\Users<username>\AppData\Local\Temp\ONEFIL4\chirp.py", line 17, in
run.run()
File "C:\Users<username>\AppData\Local\Temp\ONEFIL4\chirp\run.py", line 20, in r
un
File "C:\Users<username>\AppData\Local\Temp\ONEFIL4\chirp\run.py", line 30, in r
un_plugins
File "C:\Users<username>\AppData\Local\Temp\ONEFIL4\asyncio\base_events.py", lin
e 616, in run_until_complete
File "C:\Users<username>\AppData\Local\Temp\ONEFIL4\chirp\run.py", line 44, in _
run_coroutines
File "C:\Users<username>\AppData\Local\Temp\ONEFIL4\chirp\plugins\network\scan.p
y", line 44, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL~4\chirp\plugins\network\networ
k.py", line 37, in parse_dns
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 4447: inval
id start byte
11:11:52 EVENTS Reading Windows Powershell event logs. scan.py:69
11:12:14 EVENTS Reading Security event logs. scan.py:69

Add any screenshots of the problem here.

Answer 1 · 2021-04-07T12:28:28.000Z

One of our Win2016 servers is also hanging on version 1.05. See output below:

17:24:02 NETWORK Read 507 records, found 0 IoC hits. scan.py:56
REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65
REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93
REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65
REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65
Options
REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47
REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47
REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47
YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141
'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of
procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike
Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is
going to take a while.
17:24:03 EVENTS Reading KernelMode event logs. scan.py:69
EVENTS Reading Security event logs. scan.py:69
18:09:53 EVENTS Reading Application event logs. scan.py:69
18:11:59 YARA Beginning processing. run.py:100
18:14:55 EVENTS Reading Windows Powershell event logs. scan.py:69
18:16:20 EVENTS Read 399688 logs, found 0 matches. scan.py:138
18:16:32 YARA We're still working on scanning files. 50000 processed. run.py:96
18:19:16 YARA We're still working on scanning files. 100000 processed. run.py:96
Traceback (most recent call last):
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\chirp.py", line 17, in
run.run()
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\chirp\run.py", line 20, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\chirp\run.py", line 30, in run_plugins
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\asyncio\base_events.py", line 616, in run_until_complete
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\chirp\run.py", line 44, in _run_coroutines
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\chirp\plugins\yara\run.py", line 162, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\aiomultiprocess\pool.py", line 145, in results_generator
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\aiomultiprocess\pool.py", line 308, in results
aiomultiprocess.types.ProxyException: Traceback (most recent call last):
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\aiomultiprocess\pool.py", line 110, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL1\chirp\plugins\yara\run.py", line 111, in _run
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud8d0' in position 33: surrogates not allowed

Answer 2 · 2021-04-07T12:55:46.000Z

Interesting, two separate Unicode errors. I will try to push a patch this evening! I will ping when I merge and build.

Answer 3 · 2021-04-07T14:43:39.000Z

We have another Win2016 Server that is having a similar issue, with a different Unicode error. The program, like the others, hangs after a while and never completes. Please see below:

16:42:58 EVENTS Reading Security event logs. scan.py:69
16:42:58 EVENTS Reading KernelMode event logs. scan.py:69
EVENTS Reading Windows Powershell event logs. scan.py:69
16:42:58 EVENTS Reading Application event logs. scan.py:69
16:42:58 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65
REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93
REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65
16:42:59 REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65
Options
REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47
REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47
REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47
YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141
'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of
procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike
Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is
going to take a while.
16:46:54 YARA Beginning processing. run.py:100
Traceback (most recent call last):
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp.py", line 17, in
run.run()
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 20, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 30, in run_plugins
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\asyncio\base_events.py", line 616, in run_until_complete
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 44, in _run_coroutines
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\plugins\network\scan.py", line 44, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\network.py", line 37, in parse_dns
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 5617: invalid start byte
16:51:21 YARA We're still working on scanning files. 50000 processed. run.py:96
16:57:46 YARA We're still working on scanning files. 100000 processed. run.py:96
17:01:50 YARA We're still working on scanning files. 150000 processed. run.py:96
17:05:25 YARA We're still working on scanning files. 200000 processed. run.py:96
17:08:16 YARA We're still working on scanning files. 250000 processed. run.py:96
17:11:27 YARA We're still working on scanning files. 300000 processed. run.py:96
17:14:19 YARA We're still working on scanning files. 350000 processed. run.py:96
17:18:14 YARA We're still working on scanning files. 400000 processed. run.py:96

Answer 4 · 2021-04-07T14:44:03.000Z

Interesting, two separate Unicode errors. I will try to push a patch this evening! I will ping when I merge and build.

Thank you for looking into this.

Answer 5 · 2021-04-07T14:55:21.000Z

Another server having similar issues with Unicode errors, but different position.
17:00:04 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65
REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93
REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65
REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65
Options
REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47
REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47
REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47
YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141
'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of
procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike
Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is
going to take a while.
17:00:05 EVENTS Reading Application event logs. scan.py:69
17:00:08 EVENTS Reading KernelMode event logs. scan.py:69
17:00:10 EVENTS Reading Windows Powershell event logs. scan.py:69
17:00:20 EVENTS Reading Security event logs. scan.py:69
17:02:22 YARA Beginning processing. run.py:100
Traceback (most recent call last):
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp.py", line 17, in
run.run()
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 20, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 30, in run_plugins
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\asyncio\base_events.py", line 616, in run_until_complete
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 44, in _run_coroutines
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\plugins\network\scan.py", line 44, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\network.py", line 37, in parse_dns
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 5910: invalid start byte
17:07:03 YARA We're still working on scanning files. 50000 processed. run.py:96
17:12:38 YARA We're still working on scanning files. 100000 processed. run.py:96

Answer 6 · 2021-04-07T15:01:32.000Z

Win2016 Server with similar UnicodeError with different location. Adding output below.

16:45:57 EVENTS Reading Security event logs. scan.py:69
16:46:05 REGISTRY Reading HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot scan.py:65
16:46:06 REGISTRY Key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\sibot does not exist. registry.py:93
REGISTRY Reading HKEY_LOCAL_MACHINE\Software\Microsoft\CTF scan.py:65
REGISTRY Reading HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution scan.py:65
Options
REGISTRY Found 0 hit(s) for Sibot - Registry indicator. scan.py:47
REGISTRY Found 0 hit(s) for Teardrop - Registry Activity indicator. scan.py:47
REGISTRY Found 0 hit(s) for IFEO Persistence indicator. scan.py:47
YARA Enumerating the entire filesystem due to ['simpleseesharp : Webshell Unclassified', run.py:141
'reGeorgTunnel : Webshell Commodity', 'sportsball : Webshell', 'Detection for the use of
procdump to dump LSASS process memory.', 'CISA Solar Fire', 'CISA Teardrop', 'CrowdStrike
Rempack', 'CrowdStrike Sunspot', 'FireEye Cosmic Gale', 'FireEye Sunburst']... this is
going to take a while.
16:57:18 EVENTS Reading Windows Powershell event logs. scan.py:69
16:57:26 EVENTS Reading Application event logs. scan.py:69
16:58:10 EVENTS Reading KernelMode event logs. scan.py:69
17:00:57 YARA Beginning processing. run.py:100
Traceback (most recent call last):
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp.py", line 17, in
run.run()
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 20, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 30, in run_plugins
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\asyncio\base_events.py", line 616, in run_until_complete
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\run.py", line 44, in _run_coroutines
File "C:\Users<username>\AppData\Local\Temp\ONEFIL2\chirp\plugins\network\scan.py", line 44, in run
File "C:\Users<username>\AppData\Local\Temp\ONEFIL~2\chirp\plugins\network\network.py", line 37, in parse_dns
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 5105: invalid start byte
17:20:09 YARA We're still working on scanning files. 50000 processed. run.py:96
17:36:38 YARA We're still working on scanning files. 100000 processed. run.py:96
17:48:44 YARA We're still working on scanning files. 150000 processed. run.py:96
17:54:40 YARA We're still working on scanning files. 200000 processed. run.py:96
17:56:18 YARA We're still working on scanning files. 250000 processed. run.py:96
18:03:33 YARA We're still working on scanning files. 300000 processed. run.py:96
18:15:59 YARA We're still working on scanning files. 350000 processed. run.py:96
18:25:48 YARA We're still working on scanning files. 400000 processed. run.py:96
18:38:49 YARA We're still working on scanning files. 450000 processed. run.py:96
18:55:09 YARA We're still working on scanning files. 500000 processed. run.py:96
19:20:22 YARA We're still working on scanning files. 550000 processed. run.py:96
19:44:00 YARA We're still working on scanning files. 600000 processed. run.py:96
19:58:25 YARA We're still working on scanning files. 650000 processed. run.py:96
20:21:57 YARA We're still working on scanning files. 700000 processed. run.py:96
20:37:50 YARA We're still working on scanning files. 750000 processed. run.py:96

Answer 7 · 2021-04-07T20:27:28.000Z

When #33 is merged, I will build the release for this fix.