RolfRolles/HexRaysDeob

Does this compile for Linux?

eugenekolo opened this issue · 8 comments

Successfully built the .so, and placed into the IDA 7.1/plugins directory.
It can't load successfully in IDA though:

dlsym(/home/eugenek/ida-7.1/plugins/HexRaysDeob.so._PLUGIN): /home/eugenek/ida-7.1/plugins/HexRaysDeob.so: undefined symbol: PLUGIN
/home/eugenek/ida-7.1/plugins/HexRaysDeob.so: not IDA DLL file

I've never tested it on Linux, but as far as I know, my plugin doesn't make use of anything Windows-specific. Another user contributed a makefile that supposedly works on Mac; that's the "makefile" in the root that you're probably using (https://github.com/RolfRolles/HexRaysDeob/blob/master/makefile). I think it should work.

Since I've never built this plugin (or any plugin) on Linux, I'm not sure I can offer low-level advice on this one. I do have some general thoughts. Every IDA plugin must have an export called PLUGIN (which, furthermore, must be a data structure of type plugin_t. In the first line of the output that you pasted, in the first half, IDA is looking for an export called _PLUGIN. In the second half, it says that the symbol PLUGIN is undefined, i.e., not exported from the .so file.

If it were me, at this point I'd probably try listing the exports from your .so file (see this stackoverflow link). Does it have an export named PLUGIN or _PLUGIN? If not, there's a build problem of some sort, where the PLUGIN symbol isn't being exported as it needs to be. If it does have a symbol named PLUGIN or _PLUGIN, try exporting via the other name (add or remove an underscore from the name PLUGIN at the bottom of main.cpp). (It's possible this change might require other modifications to the build process.)

Hi @RolfRolles, you're right in assuming that something is wrong with the exports. In fact, it looks like nothing much was compiled into the final .so. So, I'm guessing the makefile as is doesn't quite work for Linux with gcc. Going to try clang, and then investigate the makefile. Thanks!

Please do post back if you resolve the issue, so that if other users have the same problem, they might get some hints from your solution.

I was able to build a makefile for linux and build the SO file.

I can get the msg "Microcode explorer ready to use", but it seems clicking on Edit>Plugins>Microcode Explorer doesn't do anything.

I can submit a pull request with the makefile for linux and some instructions.

I have a feeling I know what causes the behavior you're experiencing; I think it's bitten a few people and that I should just fix it. Change the 3 in this line to a 0, recompile, and try again.

Cool, that worked. I'll submit a PR of the makefile.lnx and instructions.

For posterity, I'll explain why you experienced that issue. An IDA plugin's run function takes an integer argument as input. By default, IDA passes 0 for the argument.

My plugin has several pieces of functionality that the user may wish to invoke (fixing the calls to alloca as described in the blog entry, or invoking the Microcode Explorer). I use the integer argument to run to determine which functionality the user wants.

In order to tell my plugin which functionality I'm interested in running, I've always just used the IDC script window and executed the one-line IDC script RunPlugin([path], ARGNUM); Although it's second nature to me to interact with my plugins in this way, I'll concede that this mechanism to select functionality is not very user-friendly, and is certainly not documented.

By changing the 3 to a 0, the Microcode Explorer will become the default functionality invoked by Edit->Plugins->Microcode Explorer.

Having accepted your makefile.lnx, I'm closing this issue. Thanks for your assistance.