Cisco-Talos/pyrebox

make broken by sleuthkit

SecsAndCyber opened this issue · 18 comments

Fails to clean before attempting to build the update

pyrebox$ make clean
rm -f triggers/*.so triggers/*.o triggers/*.d exploit_detect/*.so exploit_detect/*.o exploit_detect/*.d
make[1]: Entering directory '/git/pyrebox/sleuthkit'
make[1]: *** No rule to make target 'clean'.  Stop.
make[1]: Leaving directory '/git/pyrebox/sleuthkit'
Makefile:78: recipe for target 'clean-sleuthkit' failed
make: *** [clean-sleuthkit] Error 2

Fails to build the update

/git/pyrebox$ make
make[1]: Entering directory '/git/pyrebox/qemu'
        CHK version_gen.h
  CC      i386-softmmu/pyrebox/qemu_glue_sleuthkit.o
In file included from /git/pyrebox/qemu/pyrebox/sleuthkit/tsk/libtsk.h:4:0,
                 from /git/pyrebox/qemu/pyrebox/qemu_glue_sleuthkit.c:26:
/git/pyrebox/qemu/pyrebox/sleuthkit/tsk/base/tsk_base.h:53:10: fatal error: tsk/tsk_incs.h: No such file or directory
 #include "tsk/tsk_incs.h"
          ^~~~~~~~~~~~~~~~
compilation terminated.
/git/pyrebox/qemu/rules.mak:66: recipe for target 'pyrebox/qemu_glue_sleuthkit.o' failed
make[2]: *** [pyrebox/qemu_glue_sleuthkit.o] Error 1
Makefile:478: recipe for target 'subdir-i386-softmmu' failed
make[1]: *** [subdir-i386-softmmu] Error 2
make[1]: Leaving directory '/git/pyrebox/qemu'
Makefile:72: recipe for target 'all' failed
make: *** [all] Error 2
/git/pyrebox$ ./build.sh
[*] Building The Sleuth Kit...
make: *** No targets specified and no makefile found.  Stop.
[!] Oops... build failed!

./build.sh --reconfigure looks like it might be working

New error:
/git/pyrebox/pyrebox-i386: symbol lookup error: /git/pyrebox/pyrebox-i386: undefined symbol: qemu_pread

(1) Make clean failed because the Makefile for sleuthkit had not yet been generated.
(2) Make was not working, because pyrebox requires (a) some sleuthkit header files to be generated, (b) the sleuthkit library to be compiled beforehand. This is automated by build.sh.
(3) ./build.sh was not working because it was not triggering the reconfiguration of the project (if pyrebox is already configured, for instance, if the latest commit it fetched via git pull).
(4) ./build.sh --reconfigure worked because it was reconfiguring the whole project (triggerint the configuration of sleuthkit as well).
(5) /git/pyrebox/pyrebox-i386: symbol lookup error:, was caused because pyrebox-i386 is not told which is the location of the sluthkit shared library (via LD_LIBRARY_PATH).

All these issues have been solved one by one in the latest commit (e212881) that affects the Makefile, build.sh, and pyrebox-i386 and pyrebox-x86_64 symbolic links (now converted to bash scripts).

Plus this commit: c4a93d1

0xcpu commented

@xabiugarte I've built pyrebox on Ubuntu 18.04.1 LTS with the latest commit and the error with missing qemu_pread persists. Any ideas why it fails?

./start_i386.sh 
./pyrebox-i386: symbol lookup error: ./pyrebox-i386: undefined symbol: qemu_pread
./start_x86_64.sh 
./pyrebox-x86_64: symbol lookup error: ./pyrebox-x86_64: undefined symbol: qemu_pread

Thank you!

Reopening so that I check this error

The same problem on System Ubuntu 16.04 "./pyrebox-i386: symbol lookup error: ./pyrebox-i386: undefined symbol: qemu_pread"

0xcpu commented

I solved the issue with the undefined symbol but now sleuthkit fails to open the image.

[*] Loading python component initialization script
[*] Platform: x86_64-softmmu
[*] Starting python module initialization
[*] Reading configuration from 'pyrebox.conf'
[*] Searching for KDBG...
[*] Initializing scripts...
[*] Finished python module initialization
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on 127.0.0.1:5900

[SLEUTHKIT]
Opening image - Size: 10737418240
[!] Error while opening image
Unsupported image type (512)

[SLEUTHKIT]
Opening image - Size: 3224686592
[!] Error while opening image
Unsupported image type (512)

What was the fix?

0xcpu commented

@GKRSOFT I've changed the scope of qemu_pread to the qemu_glue_sleuthkit.c translation unit, but that's not ok, that breaks the integration with sleuthkit, still not figured out where is the cause(although I guess it's in the Makefile and how pyrebox links with sleuthkit)

Hi,

Can you check if PyREBox was properly updated?

Now pyrebox-i386 and pyrebox-x86_64 are bash scripts instead of symbolic links.

But they might have not been properly updated if you did a git pull.

They should contain something like:

#!/usr/bin/env bash
BASEDIR=$(dirname "$0")
LD_LIBRARY_PATH=$BASEDIR/sleuthkit/tsk/.libs:$LD_LIBRARY_PATH $BASEDIR/qemu/x86_64-softmmu/qemu-system-x86_64 "$@"
0xcpu commented

@xabiugarte yes, they aren't updated properly

Thanks @0xcpu ,

Then I'll submit a fix for that shortly.

Commit 7f98ae7 should fix this:

git pull
make

Let me know if that works

Just realized that due to the presence of symbolic links, the fix might not work as expected.

0xcpu commented

Checked with latest changes, seems to be ok.

sudo ./pyrebox-x86_64 -smp 2 -m 2048 -monitor stdio -usb -drive file=images/win7x64.qcow2,index=0,media=disk,format=qcow2,cache=unsafe -enable-kvm

[*] Loading python component initialization script
[*] Platform: x86_64-softmmu
[*] Starting python module initialization
[*] Reading configuration from 'pyrebox.conf'
[*] Searching for KDBG...
[*] Initializing scripts...
[*] Finished python module initialization
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on 127.0.0.1:5900

[SLEUTHKIT]
Opening image - Size: 21474836480
[*] Found volume system of type DOS Partition Table at 0, number of partitions: 5
    [#] Partition 0 - Start sector: 0 - Number of sectors: 1 - Desc: Primary Table (#0)
    [#] Partition 1 - Start sector: 0 - Number of sectors: 800 - Desc: Unallocated
    [#] Partition 2 - Start sector: 800 - Number of sectors: 32000 - Desc: NTFS / exFAT (0x07)
        [+] Found file system type ntfs at 100000
    [#] Partition 3 - Start sector: 32800 - Number of sectors: 27cd000 - Desc: NTFS / exFAT (0x07)
        [+] Found file system type ntfs at 6500000
    [#] Partition 4 - Start sector: 27ff800 - Number of sectors: 800 - Desc: Unallocated

Thanks @xabiugarte !

Just made sure that symbolic links are removed if present in latest commit:

e463f8e

Issue should be solved so I am closing it.

Cheers,