Assembly code analysis is a time-consuming process. An effective and efficient assembly code clone search engine can greatly reduce the effort of this process, since it can identify the cloned parts that have been previously analyzed. Kam1n0 is a scalable system that supports assembly code clone search. It allows a user to first index a (large) collection of binaries and then search for the code clones of a given target function or binary file. We created a promotional video on YouTube to illustrate assembly code clone search:
Kam1n0 tries to solve the efficient subgraph search problem (i.e. graph isomorphism problem) for assembly functions. Given a target function (the middle one in the figure below) it can identity the cloned subgraphs among other functions in the repository (the ones on the left and right as shown below). Kam1n0 supports rich comment format and has an IDA Pro plug-in to use its indexing and searching capabilities via IDA Pro.
Kam1n0 was developed by Steven H. H. Ding under the supervision of Benjamin C. M. Fung of the Data Mining and Security Lab at McGill University in Canada. It won the second prize at the Hex-Rays Plug-In Contest 2015. If you find Kam1n0 useful, please cite our paper:
- S. H. H. Ding, B. C. M. Fung, and P. Charland. Kam1n0: MapReduce-based Assembly Clone Search for Reverse Engineering. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD), pages 461-470, San Francisco, CA: ACM Press, August 2016.
In this repository, we release the initial version of Kam1n0 and its IDA Pro plug-in. It can run on a single workstation/server and provides a clone search service through RESTful web services. Users can connect to the server through IDA Pro. Alternatively, it can be deployed on a distributed cluster (next major release).
- [Kam1n0 Core] Added a new symbolic mode. Now it supports cross-architecture subgraph clone search on the symbolic expression level. Included libvex and z3 library. Supported architecture: x86, AMD64, MIPS32, MIPS64, PowerPC32, PowerPC64, ARM32, and ARM64.
- [Kam1n0 Core] Updated graph search algorithm. Improved scalability & accuracy. Updated default ALSH settings.
- [Kam1n0 Core] Added Visual C++ Redistributable for VS15 dependency (included in the installer for z3).
- [Web UI] In the symbolic mode, we also visualize the control flow graph with abstract syntax tree for each basic block.
- [Web UI] User can index multiple files at a time.
- [Web UI] User can directly index idb or i64 file.
- [Web UI] Fixed web UI bugs and improved usability.
- [Web UI] User can interrupt running jobs through the administration portal.
- [RESTful API] The old API is no longer working. Check out new API after installation.
- [IDA Pro plug-in for Kam1n0] Support composition analysis query.
- [Web UI] Added a web interface for clone search of an assembly function.
- [Web UI] Added a web interface for clone search of a binary file.
- [Kam1n0 Workbench] Multiple repositories can be created and managed on a single workstation.
- [Kam1n0 Core] The clone search results file can be shared and browsed on another machine without access to the repository.
- [Kam1n0 Core] Support indexing and searching of large binary files (>40 MB) without limits on system memory.
- [Kam1n0 Core] Support ARM, PowerPC, x86, and AMD64 binaries.
- [Kam1n0 Core] Support user-defined processor architectures.
- [Kam1n0 Core] Optimized index structure provides better scalability and clone search quality.
- [Kam1n0 Core] Kam1n0 no longer skips basic blocks which have less than three instructions. Now, only single instruction basic blocks are skipped, thanks to the new index structure.
- [IDA Pro plug-in for Kam1n0] [Experimental] Added an assembly fragment search functionality.
- [IDA Pro plug-in for Kam1n0] Added a tree view for browsing a large number of clones.
- The assembly code repositories and configuration files used in previous versions (<1.0.0) are no longer supported by the latest version. See the documentation on how to migrate previous repositories.
- You can index millions of functions in each repository on a single machine. The average response time for a query still stays around 1 s and the average indexing time for a function, around 20 ms.
The current release of Kam1n0 consists of two installers: The core server and IDA Pro plug-in.
| Installer | Included components | Description |
|---|---|---|
| Kam1n0-server.msi | Core engine | Main engine providing service for indexing and searching. |
| Workbench | A user interface to manage the repositories and running service. | |
| Web user interface | Web user interface for searching/indexing binary files and assembly functions. | |
| Visual C++ redistributable for VS 15 | Dependecy for z3. | |
| Kam1n0-client-idaplugin.msi | Plug-in | Connectors and user interface. |
| Cefpython | Rendering engine for the user interface. | |
| Wxpython | Rendering engine for Cefpython. |
The Kam1n0 core engine is purely written in Java. You need the following dependencies:
- [Required] The latest x64 8.x JRE/JDK distribution from Oracle.
- [Optional] The latest version of IDA Pro with the idapython plug-in installed. The Python plug-in and runtime should have already been installed with IDA Pro. Reinstall IDA Pro if necessary.
Download the Kam1n0-server.msi file from our release page. Follow the instructions to install the server. You will be prompted to select an installation path as well as the IDA Pro installation path. The latter is optional if the server does not have to deal with any disassembling. In other words, the client side uses the Kam1n0 plugin for IDA Pro. It is strongly suggested to have the IDA Pro installed with the Kam1n0 server. The current version of Kam1n0 only supports IDA Pro.
The Kam1n0 IDA Pro plug-in is written in Python for the logic and in HTML/JavaScript for the rendering. The following dependencies are needed for its installation:
- [Required] The latest version of IDA Pro with the idapython plug-in installed. The Python plug-in and runtime should have already been installed with IDA Pro. Reinstall IDA Pro if necessary.
Next, download the Kam1n0-client-idaplugin.msi installer from our release page. Follow the instructions to install the plug-in and runtime. Please note that the plug-in has to be installed in the IDA Pro plugins folder which is located at $IDA_PRO_PATH$/plugins. For example, on Windows, the path could be C:/Program Files (x86)/IDA 6.95/plugins. The installer will validate the path.
In the previous version of Kam1n0, only a single repository was supported on a workstation and the configuration files for Kam1n0 were in the same folder than the engine executable file. Starting from 1.x.x version, Kam1n0 supports multiple repositories on a workstation and each repository can support different processor architectures. Each repository is given a data folder where you can find its configuration files. More details can be found in the Kam1n0 workbench tutorial.
- Manage repositories with the Kam1n0 Workbench
- Web interface tutorial
- IDA Pro plug-in tutorial
- Working with a cluster
- Create your own processor definition
- Migrate a repository from a previous version
- CLI tutorial
The software was developed by Steven H. H. Ding under the supervision of Benjamin C. M. Fung at the McGill Data Mining and Security Lab. It is distributed under the Apache License Version 2.0. Please refer to LICENSE.txt for details.
Copyright 2017 McGill University. All rights reserved.