/*
IDA Pro Collabreation/Synchronization Plugin
Copyright (C) 2008 Chris Eagle <cseagle at gmail d0t com>
Copyright (C) 2008 Tim Vidas <tvidas at gmail d0t com>
This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option)
any later version.
This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
more details.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc., 59 Temple
Place, Suite 330, Boston, MA 02111-1307 USA
*/
Version 0.4.0
BACKGROUND:
IdaPro has an event notification mechanism that plugins may utilize to be
notified of many different types of changes being made to a database. The
primary idea behind CollabREate is to hook these change notifications and
broadcast changes to a central database server that records each change
and forwards the associated change parameters to other interested users
working on the same binary file. The theory being that one user's changes
will be applied to a second user's database effectively allowing multiple
users to simultaneously edit and annotate their databases.
Please not that no single master copy of a particular IDB is maintained
anywhere. A central database that records all events generated by all
registered users is the only record of changes. It is entirely possible
that the state of two databases diverges if one user were to make
modifications without publishing those modifications to the database server.
A major difficulty in this approach arises from the manner in which Ida
notifications are generated. Because of the manner in which the Ida auto
analysis subsystem operates, what appears to be a single action to the user
may in fact translated into many (tens, hundreds, or even thousands)
individual notification messages which all get pushed to the database. This
makes conflict resolution between users and events extremely difficult
because it is virtually impossible to associate a specific notification
message with a specific user action. In fact some notification messages may,
at first glance seem entirely unrelated to the user action which resulted in
the generation of those notifications.
Another challenge that arises is to supress the generation of duplicate
notifications when processing incoming notifications. The current version of
CollabREate simply de-registers itself for notifications while applying any
changes indicated by each incoming message. This seems to work well unless
the incoming messages results in the generation of more than one additional
notification in the receiving IDB. In such cases, it is extremely difficult
to determine when the last notification associated with the application of
a newly received message has be generated, and therefore difficult to
determine exactly when to re-register for receipt of local notifications.
This is suspected to be one of the primary causes of a CollabREate database
getting flooded with notifications and associated IDBs diverging in state.
In practice, this problem seems to arise from actions that generate large
numbers of notifications such as undefining large blocks of code or data
or patching large numbers of bytes.
RELEASE NOTES version 0.4.0
Removed dependency on QtNetwork. Reworked asynchronous networking
for IDA versions 5.5 and later to make use of the IDA SDK execute_sync
function. Updated MSVC build files so that a single build file accomodates
multiple versions of IDA.
RELEASE NOTES version 0.3.0
This version of the plugin is NOT compatible with older (< 0.3.0) collabreate
databases. This is a result of the addtion of 64 bit support. If you need
to use old databases, then you should continue to use version 0.2.0. We may
publish an upgrade utility that will upgrade 0.2.0 databases to 0.3.0 databases.
For the purposes of this discussion we are talking about the SQL database that
stores all of the IDA update notifications.
BUILDING THE PLUGIN
The IDA Pro plugin requires the IDA Pro SDK, which is distributed along with
Pro. You need the version of the SDK that matches the version of IDA with
which you intend to use the plugin.
First you need to unpack the archive, you should do this within
the SDK's "plugins" directory:
tar -zxf collabreate.tgz
Compiled binary plugins for most versions of Ida may
be found in the collabreate/trunk/bin directory.
BUILDING THE WINDOWS ONLY PLUGIN
The Windows only version of the plugin works with all Windows versions
of Ida from the 4.9 freeware version up to version 6.0. Specifically
the Windows version is for use with idag.exe on Windows, but will also
work with idaq.exe.
The easiest way to build the plugin on Windows is to use the Visual C++
along with the included collabreate.sln to build the plugin.
Upon successfull compilation, the plugin collabreate.plw will be located in
the <SDKDIR>/bin/plugins subdirectory of the SDK you built with
BUILDING THE QT VERSION OF THE PLUGIN
Ida ships with a limited number of link libraries for Qt. These may be found
in <SDK>/lib/Qt.w32 on Windows. On linux you may link against the .so files
that ship with your Ida install. On OS X you may link against the .dylib
files that ship with your Ida install.
In order to build the Qt version of the plugin for Ida versions 6.0 and later
you will also need the Qt source code. Ida 6.0 uses Qt version 4.6.3 whose
source is available here
ftp://ftp.qt.nokia.com/qt/source/qt-everywhere-opensource-src-4.6.3.tar.gz
or here
ftp://ftp.qt.nokia.com/qt/source/qt-everywhere-opensource-src-4.6.3.zip
Before building the plugin, your Qt source must be properly configured. See the
Qt documentation for your platform regarding configuring Qt. The Qt libraries
shipped with Ida are configured to wrap all of Qt in a QT namespace so a sample
configuration might look like:
configure -qtnamespace QT -no-openssl -no-dbus -shared -no-phonon-backend \
-no-phonon -no-audio-backend -no-multimedia -no-rtti -release -opensource \
-no-script -no-scripttools -no-webkit -platform win32-msvc2008
Change the -platform to linux-g++ or macx-g++ as appropriate
Note that the Qt libraries shipped with the Windows version of Ida were built
with Visual Studio, so the Windows version of the plugin will need to be
built using Visual Studio.
Once Qt is configured, the Qt libraries need to be built. On Windows, with
a properly configure Visual Studo command line environment, the nmake utility
may be used to compile Qt. On linux gmake is used, and on OS X make is used.
Once Qt has been confugred and built, the plugin can be built. From the
plugin's trunk directory, the first step is to create the required makefiles
using Qt's qmake utility (which should be in your path):
> qmake -o Makefile.msvc collab.pro -platform win32-msvc2008
Of course you can name the output makefile anything you like and you should
set the platform option appropriately.
To complete the build on Windows, use Visual Studio's nmake utility, again
assuming a properly configure command line build environment:
> nmake -f Makefile.msvc
You should find the compiled plugin at <SDK>/bin/plugins/collab_qt.plw
Build scripts for Windows, linux, and OS X shipw with collabREate. The command
> ./build.win32
should perform the necessary steps to build the plugin (assuming Qt and Visual
Studio environment variables are properly set)
INSTALLING THE PLUGIN
As with any IDA plugin, simply copy the compiled plugin file into IDA's
plugins directory which is probably something like C:\Program Files\IDA\plugins\
on most Windows systems.
Due to Windows' file locking, you will likely need to make sure all Ida databases
are closed before copying to this directory.
On linux the plugin should be copied to ida/plugins
On OS X the plugin should be copied to ida/idaq.app/Contents/MacOs/plugins
USING THE PLUGIN
It is intended that a new collabreate session is used for each binary loaded
into IDA Pro. Once a binary is loaded and the autoanalysis phase has completed
you can activate CollabREate by pressing the hotkey ALT-F6 (which you can change
in plugins.cfg if you need to)
Upon activation you will be presented with a series of screens prompting you to
connect to a CollabREate server, authenticate, start a new project, etc. The
exact series of steps will very upon the running mode of the server (such as if
it is connected to a backend database or not).
Typical Project Join Actions:
Connect: Connects to a collabreate server (default port is 5042)
Authenticate: Provide a User / Password to connect to the server
Choose Project: Provides a list of 'related' projects (1)
New Project: Request that the server creates a new project (2)
Existing Project: Join an existing project
Project Snapshot: Request a new project that is a Fork of an existing project
at a particular, previously saved point (2)
(1) only projects that match the MD5 of the binary loaded in IDA are displayed
(2) when starting a new project you must provide a textual description
When connecting to an existing project for the first time, you automatically
recieve all updates that have been made to the project prior to you joining.
On subsequent connections you will automatically receive any updates that have
been made since the last time you were connected to the project.
Once connected to a project, you can use IDA normally. Updates you make are
sent to the server, updates other collabreators make are reflected in your
database. The types of updates you can make depend on the version of IDA you
are using, basically 5.1 and higher have the most functionality (see table 1)
for more information.
Table 1: Collabreate capabilities by IDA version
P - publish, S - subscribe
______________________________________________________
| 4.9 | 5.0 | 5.1 | 5.2 | 5.3+ |
(incl FW) | | | |
-------------+-------+-------+-------+-------|-------|
Action | P | S | P | S | P | S | P | S | P | S |
-------------+---+---+---+---+---+---+---+---|---+---|
Undefine | X | X | X | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Make code | X | X | X | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Make data | X | X | X | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Move segment | X | X | X | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Name changed | | X | | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Function | | | | | | | | | | |
added or | | X | | X | X | X | X | X | X | X |
deleted | | | | | | | | | | |
-------------+---+---+---+---+---+---+---+---|---+---|
Function | | | | | | | | | | |
bounds | | X | | X | X | X | X | X | X | X |
changed | | | | | | | | | | |
-------------+---+---+---+---+---+---+---+---|---+---|
Byte | | | | | | | | | | |
patched | | X | | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Comment | | | | | | | | | | |
changed | | X | | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Operand | | | | | | | | | | |
type | | X | | X | X | X | X | X | X | X |
changed | | | | | | | | | | |
-------------+---+---+---+---+---+---+---+---|---+---|
Enum created | | | | | | | | | | |
or deleted | | X | | X | X | X | X | X | X | X |
-------------+---+---+---+---+---+---+---+---|---+---|
Struct | | | | | | | | | | |
created, | | X | | X | X*| X | X*| X | X?| X |
deleted, | | | | | | | | | | |
or changed | | | | | | | | | | |
-------------+---+---+---+---+---+---+---+---|---+---|
Function | | | | | | | | | | |
tail added | | X | | X | X | X | X | X | X | X |
or deleted | | | | | | | | | | |
-------------+---+---+---+---+---+---+---+---|---+---|
Segment | | | | | | | | | | |
added, | | X | | X | X | X | X | X | X | X |
deleted, | | | | | | | | | | |
or changed | | | | | | | | | | |
-------------+---+---+---+---+---+---+---+---|---+---|
Flirt | | | | | | | | | | |
function | | X | | X | | X | X | X | X | X |
identified | | | | | | | | | | |
-------------+---+---+---+---+---+---+---+---|---+---|
Xref add and | | | | | | | | | | |
delete | | X | | X | | X | | X | X | X |
-----------------------------------------------------|
* An updated IDA 5.2 kernel is required in order for full structure updates
to be properly published.
Pressing Alt-F6 after the plugin is already activated will bring up a menu of
collabreate specific commands.
Collabreation Actions:
Disconnect: Disconnect from the server.
Fork: Immediatly fork the project with the updates that IDA has
recieved and join this new project. (3)
Snapshot: Save a snapshot (point in time) (3)
Manage Req Perms: Allows the user to change the permissions requested when
joining the project
Manage Proj Perms: Allows the user (only if the user is the owner of the
project) to alter the project permissions.
(3) the user must provide a textual description
Forking creates a new projet that can have further updates, and that others can
participate in (join). Snapshots are simply 'projects at a point in time' - not
actual projects. You can not 'join' a snapshot. You must fork a new project
from a snapshot. Overall, snapshots take up much less space than forked
projects.
If the server is connected to a database backend, a unique project id is stored
in the idb file and subsequent connections to a server will automatically
connect to the correct project.