release oletools-0.60.1 fails test_xlm tests with python 3.11
xambroz opened this issue · 2 comments
xambroz commented
Affected tool:
olevba
Describe the bug
during the build if the setup.py test is executed it fails with the tests/olevba/test_basic.py
From the testing directory tests/test-data/excel4-macros the olevba tool fails to detect macro in:
excel4_sample_macro.xlsb
excel4_sample_macro.xlsm
excel4_sample_macro.xltm
Actually excel4_sample_macro.xlsm and excel4_sample_macro.xltm was detected as containing macros by olevba from 0.56.2 release so missing this detection in 0.60.1 is regression.
File/Malware sample to reproduce the bug
files are already part of the 0.60.1 release package
How To Reproduce the bug
python3.11 setup.py test
pytest -v
Expected behavior
should pass the test
Console output / Screenshots
$ pytest
============================================ test session starts =============================================
platform linux -- Python 3.11.0rc2, pytest-7.1.3, pluggy-1.0.0
rootdir: /home/mambroz/rpmbuild/BUILD/oletools-0.60.1
plugins: cov-3.0.0
collected 81 items
tests/common/test_clsid.py . [ 1%]
tests/common/test_encoding_handler.py ...... [ 8%]
tests/common/log_helper/test_log_helper.py .......... [ 20%]
tests/ftguess/test_basic.py . [ 22%]
tests/msodde/test_basic.py .............. [ 39%]
tests/msodde/test_blacklist.py .. [ 41%]
tests/msodde/test_crypto.py . [ 43%]
tests/msodde/test_csv.py ... [ 46%]
tests/oleform/test_basic.py . [ 48%]
tests/oleid/test_basic.py ...... [ 55%]
tests/oleid/test_issue_166.py . [ 56%]
tests/oleobj/test_basic.py ........... [ 70%]
tests/oleobj/test_external_links.py . [ 71%]
tests/olevba/test_basic.py ....F [ 77%]
tests/olevba/test_crypto.py . [ 79%]
tests/ooxml/test_basic.py .... [ 83%]
tests/ooxml/test_zip_sub_file.py ..... [ 90%]
tests/ppt_parser/test_basic.py . [ 91%]
tests/rtfobj/test_is_rtf.py ..... [ 97%]
tests/rtfobj/test_issue_185.py . [ 98%]
tests/rtfobj/test_issue_251.py . [100%]
================================================== FAILURES ==================================================
__________________________________________ TestOlevbaBasic.test_xlm __________________________________________
self = <tests.olevba.test_basic.TestOlevbaBasic testMethod=test_xlm>
def test_xlm(self):
"""Test that xlm macros are found."""
XLM_DIR = join(DATA_BASE_DIR, 'excel4-macros')
ADD_ARGS = ['-j']
for filename in os.listdir(XLM_DIR):
full_name = join(XLM_DIR, filename)
suffix = splitext(filename)[1]
out_str, ret_code = call_and_capture('olevba',
args=[full_name, ] + ADD_ARGS,
accept_nonzero_exit=True)
output = json.loads(out_str)
> self.assertEqual(len(output), 3)
E AssertionError: 2 != 3
tests/olevba/test_basic.py:123: AssertionError
============================================== warnings summary ==============================================
oletools/rtfobj.py:272
/BUILD/oletools-0.60.1/oletools/rtfobj.py:272: DeprecationWarning: invalid escape sequence '\d'
DECIMAL_GROUP = b'(\d{1,250})'
oletools/oleobj.py:537
/BUILD/oletools-0.60.1/oletools/oleobj.py:537: DeprecationWarning: invalid escape sequence '\-'
sane_fname = re.sub(u'[^a-zA-Z0-9.\-_ ]', replacement, basepath)
oletools/oleobj.py:581
/BUILD/oletools-0.60.1/oletools/oleobj.py:581: SyntaxWarning: "is" with a literal. Did you mean "=="?
if idx is -1:
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================== short test summary info ===========================================
FAILED tests/olevba/test_basic.py::TestOlevbaBasic::test_xlm - AssertionError: 2 != 3
================================= 1 failed, 80 passed, 3 warnings in 43.25s ==================================
Version information:
- OS: Linux
- OS version: Fedora 37 - 64 bits
- Python version: 3.11 64 bits
- oletools version: release 0.60.1 as taken from github https://github.com/decalage2/oletools/releases/tag/v0.60.1
Additional context
Add any other context about the problem here.
xambroz commented
On the same machine with the same python version it works with oletools 0.56.2, but it doesn't with 0.60.1
Detection of XLM macros works with version 0.56.2:
$ olevba excel4_sample_macro.xlsm
olevba 0.56.2 on Python 3.11.0 - http://decalage.info/python/oletools
===============================================================================
FILE: excel4_sample_macro.xlsm
Type: OpenXML
-------------------------------------------------------------------------------
VBA MACRO xl/macrosheets/sheet1.xml
in file: xl/macrosheets/sheet1.xml - OLE stream: ''
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xm:macrosheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:xm="http://schemas.microsoft.com/office/excel/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" mc:Ignorable="x14ac xr xr2 xr3 xr6" xmlns:x14ac="http://schemas.microsoft.com/office/spreadsheetml/2009/9/ac" xmlns:xr="http://schemas.microsoft.com/office/spreadsheetml/2014/revision" xmlns:xr2="http://schemas.microsoft.com/office/spreadsheetml/2015/revision2" xmlns:xr3="http://schemas.microsoft.com/office/spreadsheetml/2016/revision3" xmlns:xr6="http://schemas.microsoft.com/office/spreadsheetml/2016/revision6" xr6:uid="{00000000-0001-0000-0000-000000000000}"><dimension ref="A1:A2"/><sheetViews><sheetView showFormulas="1" tabSelected="1" workbookViewId="0"><selection activeCell="A2" sqref="A2"/></sheetView></sheetViews><sheetFormatPr defaultRowHeight="14.6" x14ac:dyDescent="0.4"/><cols><col min="1" max="1" width="17.3046875" bestFit="1" customWidth="1"/></cols><sheetData><row r="1" spans="1:1" x14ac:dyDescent="0.4"><c r="A1" t="b"><f>ALERT("This is a sample Excel 4 macro")</f><v>0</v></c></row><row r="2" spans="1:1" x14ac:dyDescent="0.4"><c r="A2" t="b"><f>HALT()</f><v>1</v></c></row></sheetData><pageMargins left="0.7" right="0.7" top="0.75" bottom="0.75" header="0.3" footer="0.3"/></xm:macrosheet>
+----------+--------------------+---------------------------------------------+
|Type |Keyword |Description |
+----------+--------------------+---------------------------------------------+
|Suspicious|sample |May detect Anubis Sandbox |
|Suspicious|Hex Strings |Hex-encoded strings were detected, may be |
| | |used to obfuscate strings (option --decode to|
| | |see all) |
|Suspicious|XLM macrosheet |XLM macrosheet found. It could contain |
| | |malicious code |
+----------+--------------------+---------------------------------------------+
$ olevba excel4_sample_macro.xltm
olevba 0.56.2 on Python 3.11.0 - http://decalage.info/python/oletools
===============================================================================
FILE: excel4_sample_macro.xltm
Type: OpenXML
-------------------------------------------------------------------------------
VBA MACRO xl/macrosheets/sheet1.xml
in file: xl/macrosheets/sheet1.xml - OLE stream: ''
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<xm:macrosheet xmlns="http://schemas.openxmlformats.org/spreadsheetml/2006/main" xmlns:xm="http://schemas.microsoft.com/office/excel/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" mc:Ignorable="x14ac xr xr2 xr3 xr6" xmlns:x14ac="http://schemas.microsoft.com/office/spreadsheetml/2009/9/ac" xmlns:xr="http://schemas.microsoft.com/office/spreadsheetml/2014/revision" xmlns:xr2="http://schemas.microsoft.com/office/spreadsheetml/2015/revision2" xmlns:xr3="http://schemas.microsoft.com/office/spreadsheetml/2016/revision3" xmlns:xr6="http://schemas.microsoft.com/office/spreadsheetml/2016/revision6" xr6:uid="{00000000-0001-0000-0000-000000000000}"><dimension ref="A1:A2"/><sheetViews><sheetView showFormulas="1" tabSelected="1" workbookViewId="0"><selection activeCell="A2" sqref="A2"/></sheetView></sheetViews><sheetFormatPr defaultRowHeight="14.6" x14ac:dyDescent="0.4"/><cols><col min="1" max="1" width="17.3046875" bestFit="1" customWidth="1"/></cols><sheetData><row r="1" spans="1:1" x14ac:dyDescent="0.4"><c r="A1" t="b"><f>ALERT("This is a sample Excel 4 macro")</f><v>0</v></c></row><row r="2" spans="1:1" x14ac:dyDescent="0.4"><c r="A2" t="b"><f>HALT()</f><v>1</v></c></row></sheetData><pageMargins left="0.7" right="0.7" top="0.75" bottom="0.75" header="0.3" footer="0.3"/></xm:macrosheet>
+----------+--------------------+---------------------------------------------+
|Type |Keyword |Description |
+----------+--------------------+---------------------------------------------+
|Suspicious|sample |May detect Anubis Sandbox |
|Suspicious|Hex Strings |Hex-encoded strings were detected, may be |
| | |used to obfuscate strings (option --decode to|
| | |see all) |
|Suspicious|XLM macrosheet |XLM macrosheet found. It could contain |
| | |malicious code |
+----------+--------------------+---------------------------------------------+
Doesn't work with 0.60.1:
[/oletools-0.60.1/tests/test-data/excel4-macros] 2022-11-10 19:32:25 +0100
$ olevba excel4_sample_macro.xlsm
olevba 0.60.1 on Python 3.11.0 - http://decalage.info/python/oletools
===============================================================================
FILE: excel4_sample_macro.xlsm
Type: OpenXML
No VBA or XLM macros found.
$ olevba excel4_sample_macro.xltm
olevba 0.60.1 on Python 3.11.0 - http://decalage.info/python/oletools
===============================================================================
FILE: excel4_sample_macro.xltm
Type: OpenXML
No VBA or XLM macros found.
xambroz commented
It seems to be about XLMMacroDeobfuscator ... in 0.56.2 it was probably somehow bundled with oletools, while now it is not, but olevba gracefully omits to report anything about missing dependency.
I would recommed either adding XLMMacroDeobfuscator to requirements.txt or at least issue some warning upon the olevba execution that this component is missing and so the XLM detections might be missing.