Pulled in useful PR's & forks to keep upto date as main repo has gone stale

Added new Templated Type view to Structure Builder

Plugin for IDA Pro

Table of Contents

About
Installation
Configuration
Features
Presentations

About

The plugin assists in the creation of classes/structures and detection of virtual tables. It also facilitates transforming decompiler output faster and allows to do some stuff which is otherwise impossible.

Note: The plugin supports IDA Pro 7.x with Python 2/3.

Installation

Copy HexRaysPyTools.py file and HexRaysPyTools directory to $IDAUSR/plugins directory.
Install the plugin requirements found in the requirements.txt file. Be sure to install these for the Python environment used by your IDA installation.
- pip install -r requirements.txt

Configuration

Can be found at $IDAUSR\cfg\HexRaysPyTools.cfg

debug_message_level. Set 10 if you have a bug and want to show the log along with the information about how it was encountered in the issue.
propagate_through_all_names. Set True if you want to rename not only the default variables for the Propagate Name feature.
store_xrefs. Specifies whether to store the cross-references collected during the decompilation phase inside the database. (Default - True)
scan_any_type. Set True if you want to apply scanning to any variable type. By default, it is possible to scan only basic types like DWORD, QWORD, void * e t.c. and pointers to non-defined structure declarations.
templated_types_file. Set to default TOML file path for templated types view. (Default - Empty, will auto load %IDA_DIR%/Plugins/HexRaysPyTools/types/templated_types.toml)

Features

Recently added

Structure reconstruction

The reconstruction process usually comprises the following steps:

Open structure builder.
Find a local variable that points to the structure you would like to reconstruct.
Apply "Scan variable". It will collect the information about the fields that were accessed in the boundaries of one function. As an option, you can apply "Deep Scan variable", which will do the same thing but will also recursively visit other functions that has the same variable as its argument.
After applying steps 2 and 3 enough times, resolve conflicts in the structure builder and finalize structure creation. All the scanned variables will get a new type. Also, cross-references will be remembered and usable anytime.

Now, a few more details.

Structure Builder (Alt + F8)

The place where all the collected information about the scanned variables can be viewed and modified. Ways of collecting information:

Right Click on a variable -> Scan Variable. Recognizes fields usage within the current function.
Right Click on a variable -> Deep Scan Variable. First, recursively touches functions to make Ida recognize proper arguments (it happens only once for each function during a session). Then, it recursively applies the scanner to variables and functions, which get the structure pointer as their argument.
Right Click on a function -> Deep Scan Returned Value. If you have the singleton pattern or the constructor is called in many places, it is possible to scan all the places, where a pointer to an object was recieved or an object was created.
API [TODO]

Structure View

Structure builder stores collected information and enables interaction:

Types with the BOLD font are virtual tables. A double click opens the list with all virtual functions, which helps to visit them. The visited functions are marked with a cross and color:

Types with the ITALIC font have been found as passed argument. It can help in finding substructures. [TODO]
Double click on field Name or Type to edit.
Double click on Offset opens a window with all the places, where this field has been extracted. Click the "Ok" button to open a selected place in the pseudocode window:

Buttons serve the following purpose:

Finalize - opens a window with an editable C-like declaration and assigns new types to all scanned variables.

Disable, Enable - are used for collision resolution.

Origin - switches the base offset which is used to produce new fields to structure (this value will be added to every offset of a newly-scanned variable, default = 0).

Array - renders a selected field as an array the size of which is automatically calculated.

Pack - creates and substitutes a substructure for selected items (collisions for these items should be resolved).

Unpack - dismembers a selected structure and adds all its fields to the builder.

Load - loads a predefined type that is loaded into IDA into the structure builder.

Remove - removes the information about selected fields.

Clear - clears all.

Recognize Shape - looks for appropriates structure for selected fields.

Resolve Conflicts (new) - attempts to disable less meaningful fields in favor of more useful ones. (char > _BYTE, SOCKET > _DWORD etc). Doesn't help to find arrays.

Templated Types View - switches the templated types view

Structure View - switches to the structure builder view

Structure Cross-references (Ctrl + X)

With HexRaysPyTools, every time the F5 button is pressed and code is decompiled, the information about addressing to fields is stored inside cache. It can be retrieved with the "Field Xrefs" menu. So, it is better to apply reconstructed types to as many locations as possible to have more information about the way structures are used.

Note: IDA 7.4 has now an official implementation of this feature, available through Shift-X hotkey.

Guessing Allocation

Warning!! Very raw feature. The idea is to help find where a variable came from so as to run Deep Scan Process at the very top level and not to skip large amounts of code.

Structures with given size

Usage:

In Pseudocode viewer, right click on a number -> "Structures with this size". (hotkey "W")
Select a library to be looked for structures.
Select a structure. The Number will become sizeof(Structure Name), and type will be imported to Local Types.

Recognition of structures by shapes

Helps find a suitable structure by the information gleaned from pseudocode after variable scanning.

Usage:

Method 1
1. Right click on a variable with -> Select "Recognize Shape".
2. Select Type Library.
3. Select structure.
4. Type of the variable will be changed automatically.
Method 2
1. Clear Structure Builder if it's currently used.
2. Right click on the variables that are supposed to be the same -> "Scan Variable".
3. Edit types (will be implemented later), disable or remove uninteresting fields, and click the "Recognize Shape" button.
4. You can select several fields and try to recognize their shapes. If found and selected, they will be replaced with a new structure.
5. After final structure selection, types of all scanned variables will be changed automatically.

Templated Types View

The templated types view allows you to easily define and propagate templated types such as the data types you find within STL.

Type List - list of the loaded types, click on the type you want to set to populate the middle form.

Selected Type - each templated typename has two fields, Type & Name. The Type field is the actual type that will be defined, this has to be a real type, as it will throw an error if it is not. The Name field is used to create an unique typename for the templated type.

Creating Type - a live output of the types you will set.

Reload Templated Types TOML - if you have edited the current TOML file you will have to reload the list.

Open Templated Types TOML - open your own custom templated types TOML following the templated types structure

Before:

After

TOML Format

["std::vector<T>"]           
base_name = "std_vector_{1}"
types = ["T"]                
struct = """                 
struct std_vector_{1}        
{{                           
   {0} *_Myfirst;            
   {0} *_Mylast;             
   {0} *_Myend;              
}};                          
"""

Class Name - pretty type name, this is used as dictionary key and in the types list (["std::vector<T>"])

Base Name - - name used for IDA's structs as we cannot use ::, format identifiers for type name, these are always odd numbers (Note: this must match the base structure name within struct string)

Types - define the types here as a list of strings, each type will have 2 format specifier tokens

Struct - this is where you define the struct, format specifiers work as the following:

1st type:
- {0} (actual type)
- {1} (pretty print type, int* -> pInt)
2nd type:
- {2} (actual type)
- {3} (pretty print type, std::string* -> pStdString)
(Note: brackets need to be doubled up {{ & }} due to str.format

Disassembler code manipulations

Containing structures

Helps find containing structure and makes code prettier by replacing pointers with CONTAINING_RECORD macro

Before:

After:

Usage:

If a variable is a structure pointer and there's an access to outside of the boundaries of that structure, then:

Right click -> Select Containing Structure.
Select Type Library.
Select appropriate Structure and Offset.
If the result does not satisfy the requirements, then Right Click -> Reset Containing Structure and go back to step 1.

Function signature manipulation

Right click first line -> "Remove Return" converts return type to void (or from void to _DWORD).
Right click on argument -> "Remove Argument" disposes of this argument.
Right click on convention -> "Convert to __usercall" switches to __usercall or __userpurge (same as __usercall but the callee cleans the stack).

Recasting (Shift+R, Shift+L), Renaming (Shift+N, Ctrl+Shift+N)

Every time you have two sides in an expression, where each side may be a local or global variable, argument or return value of the function signature, it is possible to right-click or press the hotkey to give both sides of the expression similar types. Below, there is the table of possible conversions:

Original	Shift+L	Shift+R
var = (TYPE) expr	var type -> TYPE
exp = (TYPE) var		var type -> TYPE
function(..., (TYPE) var, ...)	functions' argument -> TYPE	var type -> TYPE
(TYPE) function(...)		functions' return type -> TYPE
return (TYPE) var	functions' return type -> TYPE	var type -> TYPE
struct.field = (TYPE) var	type(field) -> TYPE
pstruct->field = (TYPE) var	type(field) -> TYPE

When you have an expression like function(..., some_good_name, ...), you can rename function parameter.

When you have an expression like function(..., v12, ...), and function has an appropriate parameter name, you can quickly apply this name to the variable.

Also possible to rename vXX = v_named into _v_named = v_named and vice versa.

And there's a feature for massive renaming functions using assert statements. If you find a function that looks like an assert, right-click the string argument with the function name and select "Rename as assert argument". All the functions where a call to assert statement has happened will be renamed (provided that there is no conflicts, either way, you'll see the warning in the output window)

Name Propagation (P)

This feature does the same recursive traversal over functions as the Deep Scan Variable does. But this time, all elements that have a connection with the selected one receive its name. It’s possible to rename it or use names of both local and global variables, as well as structure members. By default, the plugin propagates names only over default names like v1, a2. See Configuration in order to change that.

Untangling 'if' statements

Clicking if manually allows to switch then and else branches
Automatically applies the following transformations:

Before:

...
if (condition) {
    statement_1;
    statement_2;
    ...
    return another_value;
}
return value;

After:

...
if (opposite_condition) {
    return value;
}
statement_1;
statement_2;
...
return another_value;            // if 'then' branch has no return, than `return value;`

Classes

Also, it can be found at View->Open Subview->Classes. Helps to manage classes (structures with virtual tables).

!! Better to rename all functions before debugging, because Ida can mess up default names, and the information in virtual tables will be inconsistent.

Class, virtual tables, and functions names are editable. Also a function's declaration can be edited. After editting, the altered items change font to italic. Right click opens the following menu options:

Expand All / Collapse All.
Refresh - clear all and rescan local types for information again.
Rollback - undo changes.
Commit - apply changes. Functions will be renamed and recasted both in virtual tables in Local Types and disassembly code.
Set First Argument type - allows selecting the first argument for a function among all classes. If right click was used on class name, then its type will be automatically applied to the virtual table at offset 0.

You can also filter classes using Regexp either by class_name or by existence of specific functions. Simply input an expression in line edit for filtering by class_name or prepend it with "!" to filter by function name.

Structure Graph

Shows relationship between structures:

Also: dark green node is union, light green - enum.

Usage:

Open Local Types.
Select structures and right click -> "Show Graph" (Hotkey "G").
Plugin creates a graph of all structures that have relationship with selected items.
Double clicking on a node recalculates the graph for it.
Every node has a hint message that shows C-like typedef.

API

Under construction

oopsmishap/HexRaysPyTools

Pulled in useful PR's & forks to keep upto date as main repo has gone stale

Added new Templated Type view to Structure Builder

About

Installation

Configuration

Features

Structure reconstruction

Structure Builder (Alt + F8)

Structure View

Structure Cross-references (Ctrl + X)

Guessing Allocation

Structures with given size

Recognition of structures by shapes

Templated Types View

TOML Format

Disassembler code manipulations

Containing structures

Function signature manipulation

Recasting (Shift+R, Shift+L), Renaming (Shift+N, Ctrl+Shift+N)

Name Propagation (P)

Untangling 'if' statements

Classes

!! Better to rename all functions before debugging, because Ida can mess up default names, and the information in virtual tables will be inconsistent.

Structure Graph

API

Presentations