xray / data controller indicators for smartphone apps

Pipeline 1: Dynamic analysis

In this pipeline we will donwload and install apps from the app store, set up mitmproxy, interact with the apps, and do some analysis on their traffic.

capture logs (dynamic approach)

check out the repo
create a virtualenv, enter the virtualenv; pip install mitmproxy
create your own mitm-config.json based on mitm-config.sample.json to customise your environment
run the script like mitmdump -s mitm-save.py -p 8081

While you were interacting with an app on your device you should see traffic on your local console, and logs to be saved in the directory specified in your mitm-config.json file.

process the logs

create a copy of config.sample.json called config.json
you need bower, make sure you have it (or npm install -g bower)
at the top level, run bower install
go to the mitm directory
run npm install
make a 'mitm_out' directory (or whatever destination you specified in config.json)
run node parse-output.js to generate the data files
stand at the base directory, run http-server
point a browser to to localhost:8080/prototypes/p1.html

Automated MITM traffic logging

Another way is to use monkey to simulate user events. Logs traffic data using mitmproxy.

Requires Android Developer Studio tools (for monkeyrunner)

Requires rooted phone with adb root shell access, network log tool with superuser permissions.

Requires working mitmproxy setting up, as described above.

Edit logger.sh to specify packages to log
turn on network logging on the device
Activate the python virtual environment in xray directory, run mitmdump -s mitm-save.py -p 8080
Run logger.sh to start UI fuzzing on packages and logging mitmproxy traffic in the background
When the fuzzer has done its business, Ctrl+C the mitmproxy logging.
There will be a big CSV in the data directory. It can be split into smaller csvs for each app using csplit filename.csv 'endsession_apppackagename' (an ugly solution, will do for now!). Place in the data directory
export network log data from device, place in the data directory
auto-cleaner.py is work in progress, it uses the network log to filter out non-app traffic from the mitmproxy log csv file.

Pipeline 2: Static analysis

In this pipeline we will install apps from the app store, using an automated UI on an android device, pull the APKs from the device, do some static code analysis on the APKs using LibRadar, and store the results of further analysis.

Dependencies

Ensure you have JDK version 1.8.0_25

Downloading APKs via the app store

Requires root access to the android device.

Install android-developer-tools.
Get the device id adb devices
Modify install-from-store-random-motog.py with the device id.
Start ADB demon as root adb root
Run python install-from-store-random-motog.py

This should result in the APK files being written to a directory outside the repo called apks

Static analysis

Adjust the path in static/config.json to point to your APK folder
in the static directory, run node ./trie.js
this should write the LibRadar output to json files in the output directory
Todo: hostsearch.py

DKarandikar/xray