Illegal instruction
drsn00ker opened this issue · 16 comments
I was getting an illegal instruction trying to execute unblob in kali. But because I've been messing with cross-compilation, I thought may be I messed up a library somewhere. So in a fresh HDD partition installed Debian. Successfully installed unblob. Again, getting an illegal instruction. (see picture) This is on an oldie but goodie core2duo Intel CPU.
You probably need more info, so please reach out how I can help identify the root cause.
My guess is that your CPU is too old to support Hyperscan pattern matching.
Hyperscan relies on Advanced Vectors Extensions (AVX) and looking at this page, it looks like Intel introduced AVX in 2011 with Sandy Bridge. The Intel Core 2 you mentioned was released in 2006 and discontinued in 2012 so I suppose there is no AVX support.
To confirm that you can attach GDB to unblob and you'll be able to see the illegal instruction in hex, which will probably translate to an AVX extension opcode.
Hyperscan provides a function to test if the platform is supported, but does not check for AVX.
hs_error_t hs_valid_platform(void)
Utility function to test the current system architecture.
Hyperscan requires the Supplemental Streaming SIMD Extensions 3 instruction set.
This function can be called on any x86 platform to determine if the system provides
the required instruction set.
This function does not test for more advanced features if Hyperscan has been built
for a more specific architecture, for example the AVX2 instruction set.
Returns
HS_SUCCESS on success, HS_ARCH_ERROR if system does not support Hyperscan.
@vlaci how is pyperscan built ? does it make sense that this SIGILL is triggered by it ?
Hyperscan provides what they call a Fat runtime introducting "the ability for the Hyperscan library to dispatch the most appropriate runtime code for the host processor". Not sure if this is used by vectorscan.
You can try installing hyperscan-dev using apt, and pyperscan from the latest git source, so it will link to the system provided hyperscan library to figure out is it using more suitable build options
Thanks for the feedback, I'm not even a script kiddie (and mostly a windows user)
I've installed libhyperscan-dev and turns out pyperscan was already satisfied. Uninstalled unblob and reinstalled. Sticking with Kali at this point. Sadly illegal instruction remains.
So is this a hyperscan issue? Should I add the issue there?
Don't want to drop my old computer into the e-waste as at 3.7 GHz is still running great on single threaded applications.
@drsn00ker can you provide the output of cat /proc/cpuinfo on your machine ? This way we can check the feature flags that are enabled or not on your CPUs. Hyperscan can support old CPUs but they need the SSSE3 feature flag to be enabled.
If the required flags are there, than it's a question of building the library differently. It's not something to report to Hyperscan.
If you could attach the coredump here also that would make debugging easier :)
Also, you can check if the OS provided hyperscan itself is working by building one of its example programs, e. g:
$ sudo apt update
$ sudo apt install wget build-essential libhyperscan-dev
$ wget https://raw.githubusercontent.com/intel/hyperscan/master/examples/simplegrep.c
$ cc -I/usr/include/hs simplegrep.c -lhs -o simplegrep
$ ./simplegrep include simplegrep.c
Scanning 8294 bytes with Hyperscan
Match for pattern "include" at offset 2279
Match for pattern "include" at offset 2298
Match for pattern "include" at offset 2318
Match for pattern "include" at offset 2337
Match for pattern "include" at offset 2357
Match for pattern "include" at offset 2377
Match for pattern "include" at offset 2398I've tested the above commands in an ubuntu:22.04 docker container
Simplegrep runs without problems. See attached pic.
Also attached the proc/cpuinfo (.txt) and the coredump (zipped)
core.unblob.1000.cc95603b3d1b485080976ee536cfd49e.43210.1686686911000000.zip
cpuinfo.txt
Here are a few test builds1 you can help us narrow down the issue.
Could you try force install them to your environment where you are testing unblob?
pip install --force <varian>/pyperscan-0.2.2-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Make sure, that it overwrites the pyperscan package installed by unblob.
Hopefully one of them will work...
Footnotes
Could you check test.zip? I'm not getting any files (just directories)
Honestly not sure how or why it worked, but all 4 variants (including hyperscan-default) fix the illegal instruction error.
Either way thanks for your efforts! I've got unblob going, now to see if it extracts the firmwares that don't extract with binwalk.
Could you do a final test please? This variant shouldn't work
Of course! Let me boot into Debian later, to try it out!
Didn't get it to install in Debian (even with pipx)
Anyway, back to Kali (after fixing grub os-prober disabled grrrr) and indeed version 5 causes the illegal instruction, so reinstalled version 1 (default). So my unblob is working again.
Back to extracting a few thousand FW from ftp.zyxel.lv to see if any of the binwalk failures give a filesystem with unblob. Wish me luck!