SAAF-Developers/saaf

ERROR SAAFThreadPoolExecutor - Analysis failed with exception! java.util.concurrent.ExecutionException: java.lang.Error: Unresolved compilation problems

Opened this issue · 10 comments

mojtaba-esk commented

I have updated Smali and the ApkTool to the latest version but its still not working.
It works with a few of APKs but not with the most of them. I tried this one [ com.facebook.orca.apk ]
Thank you.

here is the error logs:

11 Sep 2015 10:44:32,859 [pool-1-thread-1] DEBUG AbstractStep - Start Analysis Step: Check APK
11 Sep 2015 10:44:32,859 [pool-1-thread-1] INFO FileCheckStep - Checking APK...
11 Sep 2015 10:44:32,859 [pool-1-thread-1] INFO FileCheckStep - Success.
11 Sep 2015 10:44:32,859 [pool-1-thread-1] DEBUG AbstractStep - Stop Analysis Step: Check APK
11 Sep 2015 10:44:32,859 [pool-1-thread-1] DEBUG AbstractStep - Start Analysis Step: Hash APK
11 Sep 2015 10:44:37,687 [pool-1-thread-1] DEBUG AbstractStep - Stop Analysis Step: Hash APK
11 Sep 2015 10:44:37,688 [pool-1-thread-1] DEBUG AbstractStep - Start Analysis Step: Create Folders
11 Sep 2015 10:44:37,688 [pool-1-thread-1] DEBUG SetupFileSystemStep - Setting up analysis directory for: com.facebook.orca.apk
11 Sep 2015 10:44:37,688 [pool-1-thread-1] DEBUG SetupFileSystemStep - The analysis will be stored at: /home/lroot/gholi/SAAF/bytecode
11 Sep 2015 10:44:37,688 [pool-1-thread-1] DEBUG SetupFileSystemStep - The appdirectory will be at: /home/lroot/gholi/SAAF/bytecode/com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8
11 Sep 2015 10:44:37,689 [pool-1-thread-1] DEBUG SetupFileSystemStep - The decoded content will be at: /home/lroot/gholi/SAAF/bytecode/com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8/bytecode
11 Sep 2015 10:44:37,689 [pool-1-thread-1] DEBUG SetupFileSystemStep - The apk content will be at: /home/lroot/gholi/SAAF/bytecode/com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8/apk_content
11 Sep 2015 10:44:37,689 [pool-1-thread-1] DEBUG SetupFileSystemStep - The smali files will be at: /home/lroot/gholi/SAAF/bytecode/com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8/bytecode/smali
11 Sep 2015 10:44:37,689 [pool-1-thread-1] DEBUG SetupFileSystemStep - The mainfest will be at: /home/lroot/gholi/SAAF/bytecode/com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8/bytecode/AndroidManifest.xml
11 Sep 2015 10:44:37,689 [pool-1-thread-1] DEBUG AbstractStep - Stop Analysis Step: Create Folders
11 Sep 2015 10:44:37,689 [pool-1-thread-1] DEBUG AbstractStep - Start Analysis Step: Setup Separate Logfile
11 Sep 2015 10:44:37,690 [pool-1-thread-1] DEBUG AbstractStep - Stop Analysis Step: Setup Separate Logfile
11 Sep 2015 10:44:37,690 [pool-1-thread-1] DEBUG AbstractStep - Start Analysis Step: Extract APK
11 Sep 2015 10:44:37,690 [pool-1-thread-1] INFO ExtractApkStep - Coyping apk-file to /home/lroot/gholi/SAAF/apps
11 Sep 2015 10:44:38,689 [pool-1-thread-1] INFO ExtractApkStep - Extracting content to /home/lroot/gholi/SAAF/bytecode/com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8/apk_content
11 Sep 2015 10:44:41,276 [pool-1-thread-1] INFO ExtractApkStep - Decoding extracted content to /home/lroot/gholi/SAAF/bytecode/com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8/apk_content
11 Sep 2015 10:44:41,277 [pool-1-thread-1] INFO Analysis - Storing results. Backtracking: 0 Heuristic: 0
11 Sep 2015 10:44:41,278 [pool-1-thread-1] DEBUG AnalysisEntityManager - Saving parent object of com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8
11 Sep 2015 10:44:41,278 [pool-1-thread-1] DEBUG ApplicationEntityManager - Trying to save entity: Application [applicationName=com.facebook.orca]
11 Sep 2015 10:44:41,279 [pool-1-thread-1] DEBUG ApplicationEntityManager - Entity does not have ID. Creating new record...
11 Sep 2015 10:44:41,285 [pool-1-thread-1] DEBUG AbstractXMLDataSource - Validating additional configuration...
11 Sep 2015 10:44:41,296 [pool-1-thread-1] DEBUG AbstractXMLDataSource - Additional configuration valid.
11 Sep 2015 10:44:41,357 [pool-1-thread-1] DEBUG ApplicationEntityManager - Entity was stored in the database during a previous run. Getting the ID ...
11 Sep 2015 10:44:41,358 [pool-1-thread-1] DEBUG ApplicationEntityManager - The id of the previously saved entity is: 1
11 Sep 2015 10:44:41,359 [pool-1-thread-1] DEBUG AnalysisEntityManager - Trying to save entity: com.facebook.orca_d1058e05bbb5fe6a618740885f3600706aaa54a8
11 Sep 2015 10:44:41,360 [pool-1-thread-1] DEBUG AnalysisEntityManager - Entity does not have ID. Creating new record...
11 Sep 2015 10:44:41,361 [pool-1-thread-1] DEBUG AnalysisEntityManager - Record created with ID: 51
11 Sep 2015 10:44:41,362 [pool-1-thread-1] INFO Analysis - Results stored.
11 Sep 2015 10:44:41,363 [pool-1-thread-1] ERROR SAAFThreadPoolExecutor - Analysis failed with exception!
java.util.concurrent.ExecutionException: java.lang.Error: Unresolved compilation problems:
Unhandled exception type DirectoryException
Unhandled exception type DirectoryException

at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:188)
at de.rub.syssec.saaf.SAAFThreadPoolExecutor.afterExecute(SAAFThreadPoolExecutor.java:84)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1153)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.Error: Unresolved compilation problems:
Unhandled exception type DirectoryException
Unhandled exception type DirectoryException

at de.rub.syssec.saaf.analysis.steps.extract.ApkDecoderInterface.decode(ApkDecoderInterface.java:147)
at de.rub.syssec.saaf.analysis.steps.extract.ExtractApkStep.doProcessing(ExtractApkStep.java:58)
at de.rub.syssec.saaf.analysis.steps.AbstractStep.process(AbstractStep.java:112)
at de.rub.syssec.saaf.analysis.Analysis.doPreprocessing(Analysis.java:255)
at de.rub.syssec.saaf.analysis.Analysis.run(Analysis.java:154)
at de.rub.syssec.saaf.AnalysisTask.run(AnalysisTask.java:56)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
... 2 more

11 Sep 2015 10:44:41,364 [pool-1-thread-1] ERROR SAAFThreadPoolExecutor - Something unexpected occurred, see above. Will now perform an unclean exit (DB connections are not closed etc, but ShutdownHooks are run)!

er4z0r commented

I am sorry, SAAF has not been tested in quite a while. What version of apktool are you using?

mojtaba-esk commented

Thank you for the reply.

Now I have updated it to "apktool_2.0.1", I can decompile that APK through command line, but by SAAF it just gives me errors.
If its not simple to solve it, can i somehow comment some parts of those codes and do the decompiling manually?

cheers

er4z0r commented

Sorry, I haven't worked on this in a long time and cannot get into right now. Might look into it over the weekend. @hannoL might have some insight since he has worked with the tool more recently I think

mojtaba-esk commented

Thank you I will wait for any possible respond.

hannoL commented

My student just last week has updated apktool and smali. And fixed the crashes. ;-) I will look, if I can push this branch to github in the next days. Hope that will fix your issue.

mojtaba-esk commented

Thanks a lot @hannoL, it would be great :)

hannoL commented

So I just pushed the commits of Julian as seperated feature branch:
Please try: https://github.com/SAAF-Developers/saaf/tree/feature/apktool2

mojtaba-esk commented

Thanks a lot. For that one it works, but now I am facing the similar error on another APK while when I am trying to decode it directly by command line like this:

unzip -n com.whatsapp.apk classes.dex -d src
java -jar baksmali-2.0.6.jar -o src/smali/ src/classes.dex

It works well. I can have smali codes and browse into them. But when SAAF wanna do it, it faces some sort of error. I don't know maybe it uses another version or tool to decode dex to baksmali.
I hope this issue will be solved.
Thank you so much in advance.

This is the logs:

14 Sep 2015 12:15:42,311 [pool-1-thread-1] DEBUG AbstractStep  - Start Analysis Step: Check APK
14 Sep 2015 12:15:42,311 [pool-1-thread-1] INFO  FileCheckStep  - Checking APK...
14 Sep 2015 12:15:42,312 [pool-1-thread-1] INFO  FileCheckStep  - Success.
14 Sep 2015 12:15:42,312 [pool-1-thread-1] DEBUG AbstractStep  - Stop Analysis Step: Check APK
14 Sep 2015 12:15:42,312 [pool-1-thread-1] DEBUG AbstractStep  - Start Analysis Step: Hash APK
14 Sep 2015 12:15:45,024 [pool-1-thread-1] DEBUG AbstractStep  - Stop Analysis Step: Hash APK
14 Sep 2015 12:15:45,025 [pool-1-thread-1] DEBUG AbstractStep  - Start Analysis Step: Create Folders
14 Sep 2015 12:15:45,025 [pool-1-thread-1] DEBUG SetupFileSystemStep  - Setting up analysis directory for: com.whatsapp.apk
14 Sep 2015 12:15:45,025 [pool-1-thread-1] DEBUG SetupFileSystemStep  - The analysis will be stored at: /home/lroot/gholi/SAAF/bytecode
14 Sep 2015 12:15:45,026 [pool-1-thread-1] DEBUG SetupFileSystemStep  - The appdirectory will be at: /home/lroot/gholi/SAAF/bytecode/com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c
14 Sep 2015 12:15:45,026 [pool-1-thread-1] DEBUG SetupFileSystemStep  - The decoded content  will be at: /home/lroot/gholi/SAAF/bytecode/com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c/bytecode
14 Sep 2015 12:15:45,026 [pool-1-thread-1] DEBUG SetupFileSystemStep  - The apk content  will be at: /home/lroot/gholi/SAAF/bytecode/com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c/apk_content
14 Sep 2015 12:15:45,027 [pool-1-thread-1] DEBUG SetupFileSystemStep  - The smali files will be at: /home/lroot/gholi/SAAF/bytecode/com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c/bytecode/smali
14 Sep 2015 12:15:45,027 [pool-1-thread-1] DEBUG SetupFileSystemStep  - The mainfest  will be at: /home/lroot/gholi/SAAF/bytecode/com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c/bytecode/AndroidManifest.xml
14 Sep 2015 12:15:45,027 [pool-1-thread-1] DEBUG AbstractStep  - Stop Analysis Step: Create Folders
14 Sep 2015 12:15:45,027 [pool-1-thread-1] DEBUG AbstractStep  - Start Analysis Step: Setup Separate Logfile
14 Sep 2015 12:15:45,028 [pool-1-thread-1] DEBUG AbstractStep  - Stop Analysis Step: Setup Separate Logfile
14 Sep 2015 12:15:45,028 [pool-1-thread-1] DEBUG AbstractStep  - Start Analysis Step: Extract APK
14 Sep 2015 12:15:45,028 [pool-1-thread-1] INFO  ExtractApkStep  - Coyping apk-file to /home/lroot/gholi/SAAF/apps
14 Sep 2015 12:15:45,066 [pool-1-thread-1] INFO  ExtractApkStep  - Extracting content to /home/lroot/gholi/SAAF/bytecode/com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c/apk_content
14 Sep 2015 12:15:45,545 [pool-1-thread-1] INFO  ExtractApkStep  - Decoding extracted content to /home/lroot/gholi/SAAF/bytecode/com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c/apk_content
14 Sep 2015 12:15:48,658 [pool-1-thread-1] ERROR Analysis  - Analysis for com.whatsapp failed!
de.rub.syssec.saaf.model.analysis.AnalysisException: de.rub.syssec.saaf.analysis.steps.extract.DecoderException: brut.androlib.err.UndefinedResObject: resource spec: 0x01010462
    at de.rub.syssec.saaf.analysis.steps.extract.ExtractApkStep.doProcessing(ExtractApkStep.java:60)
    at de.rub.syssec.saaf.analysis.steps.AbstractStep.process(AbstractStep.java:112)
    at de.rub.syssec.saaf.analysis.Analysis.doPreprocessing(Analysis.java:255)
    at de.rub.syssec.saaf.analysis.Analysis.run(Analysis.java:154)
    at de.rub.syssec.saaf.AnalysisTask.run(AnalysisTask.java:56)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: de.rub.syssec.saaf.analysis.steps.extract.DecoderException: brut.androlib.err.UndefinedResObject: resource spec: 0x01010462
    at de.rub.syssec.saaf.analysis.steps.extract.ApkDecoderInterface.decode(ApkDecoderInterface.java:183)
    at de.rub.syssec.saaf.analysis.steps.extract.ExtractApkStep.doProcessing(ExtractApkStep.java:58)
    ... 9 more
Caused by: brut.androlib.err.UndefinedResObject: resource spec: 0x01010462
    at brut.androlib.res.data.ResPackage.getResSpec(ResPackage.java:59)
    at brut.androlib.res.data.ResTable.getResSpec(ResTable.java:65)
    at brut.androlib.res.data.ResTable.getResSpec(ResTable.java:61)
    at brut.androlib.res.data.value.ResReferenceValue.getReferent(ResReferenceValue.java:57)
    at brut.androlib.res.data.value.ResStyleValue.serializeToResValuesXml(ResStyleValue.java:54)
    at brut.androlib.res.AndrolibResources.generateValuesFile(AndrolibResources.java:496)
    at brut.androlib.res.AndrolibResources.decode(AndrolibResources.java:252)
    at brut.androlib.Androlib.decodeResourcesFull(Androlib.java:134)
    at brut.androlib.ApkDecoder.decode(ApkDecoder.java:104)
    at de.rub.syssec.saaf.analysis.steps.extract.ApkDecoderInterface.decode(ApkDecoderInterface.java:104)
    ... 10 more
14 Sep 2015 12:15:48,665 [pool-1-thread-1] INFO  Analysis  - Storing results. Backtracking: 0 Heuristic: 0
14 Sep 2015 12:15:48,666 [pool-1-thread-1] DEBUG AnalysisEntityManager  - Saving parent object of com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c
14 Sep 2015 12:15:48,666 [pool-1-thread-1] DEBUG ApplicationEntityManager  - Trying to save entity: Application [applicationName=com.whatsapp]
14 Sep 2015 12:15:48,667 [pool-1-thread-1] DEBUG ApplicationEntityManager  - Entity does not have ID. Creating new record...
14 Sep 2015 12:15:48,674 [pool-1-thread-1] DEBUG ApplicationEntityManager  - Entity was stored in the database during a previous run. Getting the ID ...
14 Sep 2015 12:15:48,675 [pool-1-thread-1] DEBUG ApplicationEntityManager  - The id of the previously saved entity is: 5
14 Sep 2015 12:15:48,678 [pool-1-thread-1] DEBUG AnalysisEntityManager  - Trying to save entity: com.whatsapp_fdc5ff5b829edb6d2a1ffd043c116f38d73d7f5c
14 Sep 2015 12:15:48,678 [pool-1-thread-1] DEBUG AnalysisEntityManager  - Entity does not have ID. Creating new record...
14 Sep 2015 12:15:48,708 [pool-1-thread-1] DEBUG AnalysisEntityManager  - Record created with ID: 52
14 Sep 2015 12:15:48,709 [pool-1-thread-1] DEBUG AnalysisEntityManager  - Entity is unchanged. Skipping.
14 Sep 2015 12:15:48,709 [pool-1-thread-1] DEBUG ExceptionEntityManager  - Trying to save entity: de.rub.syssec.saaf.model.SAAFException: de.rub.syssec.saaf.analysis.steps.extract.DecoderException: brut.androlib.err.UndefinedResObject: resource spec: 0x01010462
14 Sep 2015 12:15:48,709 [pool-1-thread-1] DEBUG ExceptionEntityManager  - Entity does not have ID. Creating new record...
14 Sep 2015 12:15:48,711 [pool-1-thread-1] DEBUG ExceptionEntityManager  - Record created with ID: 33
14 Sep 2015 12:15:48,711 [pool-1-thread-1] INFO  Analysis  - Results stored.
14 Sep 2015 12:15:48,711 [pool-1-thread-1] INFO  Analysis  - Analysis for application com.whatsapp completed


14 Sep 2015 12:15:48,712 [pool-1-thread-1] ERROR SAAFThreadPoolExecutor  - Analysis of `com.whatsapp failed!


14 Sep 2015 12:15:48,712 [main] INFO  SAAFThreadPoolExecutor  - 
Analysis statistics.
====================
#APKs: 1
#Analyses: 1
#Analyses w/ uncritical exceptions: 0
#Critical Exceptions: 1
er4z0r commented

This problem has been cropping up from time to time. Other projects are plagued by it as well. Have a look at this issue in apktool.

mojtaba-esk commented

Thank you.