initializing codeblock object slow
Closed this issue ยท 27 comments
I am iterating through a list of addresses and colors for functions, lines, and codeblocks. I use identical code for each with the only difference being whether sark.Function(address)/sark.Line(address) is called or sark.CodeBlock(address). The issue is that when called on my test files the line and function loops take only .05 seconds. The loop that calls codeblock takes 6.5 and has dramatically decreased the speed of my program.
This is due to the way a codeblock is generated. It currently needs to:
- Get a flowchart for the containing function
- Iterate over the flowchart to find the block
And doing this for every address takes a while. I will look into that and see what I can do.
I suspected that it was something like that. Thanks for getting back so
quickly, and thank you for creating this tool. It has certainly made my
life easier.
On Wed, Jun 29, 2016 at 7:20 AM, Tamir Bahar notifications@github.com
wrote:
This is due to the way a codeblock is generated. It currently needs to:
- Get a flowchart for the containing function
- Iterate over the flowchart to find the block
And doing this for every address takes a while. I will look into that and
see what I can do.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQCqe61641TIoImyrpF787FSQ65HHks5qQlUFgaJpZM4JAMpH
.
Thanks for the feedback ๐
Coming up with a proper solution might take me a while (as I don't like the idea of caching results, and am not sure that there is an alternative here), but you can probably find a simple solution for your needs.
My approach (not knowing exactly what your requirements are) would be to iterate over all functions using sark.functions()
, get the flowcharts using sark.FlowChart(function)
, and then iterating the blocks using for block in flowchart
. This should (hopefully) be a lot faster. I hope this suits your needs.
All right, I will implement that and let you know. Thanks.
On Wed, Jun 29, 2016 at 11:23 AM, Tamir Bahar notifications@github.com
wrote:
Thanks for the feedback ๐
Coming up with a proper solution might take me a while (as I don't like
the idea of caching results, and am not sure that there is an alternative
here), but you can probably find a simple solution for your needs.My approach (not knowing exactly what your requirements are) would be to
iterate over all functions using sark.functions(), get the flowcharts
using sark.FlowChart(function), and then iterating the blocks using for
block in flowchart. This should (hopefully) be a lot faster. I hope this
suits your needs.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQAa5FADlM2Viog3TWiUsrTp49SbOks5qQo3xgaJpZM4JAMpH
.
So I took your advice and used this
def gen_block_dict(self):
block_dict = {}
for func in sark.functions():
flow_chart = sark.FlowChart(func.ea)
for block in flow_chart:
block_dict[block.startEA] = block
return block_dict
which decreased the time it took by a factor of 50!
previously I had called
sark.CodeBlock(EA)
which was modified to
b_dict = self.gen_block_dict()
sblock = b_dict[EA]
Anyway, may I suggest that you add a blocks() function similar to the
lines() and function() functions?
Thanks
Fraser Hood
On Wed, Jun 29, 2016 at 12:15 PM, Fraser Hood fraserhood@gmail.com wrote:
All right, I will implement that and let you know. Thanks.
On Wed, Jun 29, 2016 at 11:23 AM, Tamir Bahar notifications@github.com
wrote:Thanks for the feedback ๐
Coming up with a proper solution might take me a while (as I don't like
the idea of caching results, and am not sure that there is an alternative
here), but you can probably find a simple solution for your needs.My approach (not knowing exactly what your requirements are) would be to
iterate over all functions using sark.functions(), get the flowcharts
using sark.FlowChart(function), and then iterating the blocks using for
block in flowchart. This should (hopefully) be a lot faster. I hope this
suits your needs.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQAa5FADlM2Viog3TWiUsrTp49SbOks5qQo3xgaJpZM4JAMpH
.
I began to implement the function, and found a much better solution.
Just use sark.FlowChart(bounds=(start, end))
. It gives a flowchart with every block in the range. Fast and easy. It will also include non-function blocks, so be aware of that.
This is actually really cool, I'm happy to have come across it.
I will add it to Sark, though.
Added the function in https://github.com/tmr232/Sark/tree/codeblocks, let me know if it works for you.
thanks
On Thu, Jun 30, 2016 at 8:07 AM, Tamir Bahar notifications@github.com
wrote:
Added the function in https://github.com/tmr232/Sark/tree/codeblocks, let
me know if it works for you.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQKIaEOvP7Dg5Z4CYpIwJmuODkyvPks5qQ7GdgaJpZM4JAMpH
.
So i implemented it like so:
for block in sark.codeblocks():
print block.color
block_dict[block.startEA] = block
and what I found is that the block.color is always None, but it picks up
the line colors
for block in sark.codeblocks():
print block.color
block_dict[block.startEA] = block
for line in block.lines:
print line.color
block_dict[line.ea] = block
just fine. Let me know if I am implementing this wrong.
On Thu, Jun 30, 2016 at 10:09 AM, Fraser Hood fraserhood@gmail.com wrote:
thanks
On Thu, Jun 30, 2016 at 8:07 AM, Tamir Bahar notifications@github.com
wrote:Added the function in https://github.com/tmr232/Sark/tree/codeblocks,
let me know if it works for you.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQKIaEOvP7Dg5Z4CYpIwJmuODkyvPks5qQ7GdgaJpZM4JAMpH
.
Block color and line color are two different things. You can see the implementation here - https://github.com/tmr232/Sark/blob/master/sark/codeblocks.py#L44
What are you trying to do?
No I realize that. The issue is that one of my blocks is colored, and yet
the print block.color statement never prints anything other than None.
On Thu, Jun 30, 2016 at 12:07 PM, Tamir Bahar notifications@github.com
wrote:
Block color and line color are two different things. You can see the
implementation here -
https://github.com/tmr232/Sark/blob/master/sark/codeblocks.py#L44What are you trying to do?
โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQNifTj4nDUf9W7OODg4Zv6-AiySgks5qQ-mmgaJpZM4JAMpH
.
I brought up the lines because they are printing the color as I would
expect them to.
On Thu, Jun 30, 2016 at 12:13 PM, Fraser Hood fraserhood@gmail.com wrote:
No I realize that. The issue is that one of my blocks is colored, and yet
the print block.color statement never prints anything other than None.On Thu, Jun 30, 2016 at 12:07 PM, Tamir Bahar notifications@github.com
wrote:Block color and line color are two different things. You can see the
implementation here -
https://github.com/tmr232/Sark/blob/master/sark/codeblocks.py#L44What are you trying to do?
โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQNifTj4nDUf9W7OODg4Zv6-AiySgks5qQ-mmgaJpZM4JAMpH
.
So to clarify, this
for func in sark.functions():
flow_chart = sark.FlowChart(func.ea)
for block in flow_chart:
print block.color
prints out the color that I expect, where as
for block in sark.codeblocks():
print block.color
this does not.
On Thu, Jun 30, 2016 at 12:14 PM, Fraser Hood fraserhood@gmail.com wrote:
I brought up the lines because they are printing the color as I would
expect them to.On Thu, Jun 30, 2016 at 12:13 PM, Fraser Hood fraserhood@gmail.com
wrote:No I realize that. The issue is that one of my blocks is colored, and yet
the print block.color statement never prints anything other than None.On Thu, Jun 30, 2016 at 12:07 PM, Tamir Bahar notifications@github.com
wrote:Block color and line color are two different things. You can see the
implementation here -
https://github.com/tmr232/Sark/blob/master/sark/codeblocks.py#L44What are you trying to do?
โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQNifTj4nDUf9W7OODg4Zv6-AiySgks5qQ-mmgaJpZM4JAMpH
.
Is the color of the block set in the graph view? If it is, it sounds like a bug in the color extraction.
Yes it is.
On Thu, Jun 30, 2016 at 2:40 PM, Tamir Bahar notifications@github.com
wrote:
Is the color of the block set in the graph view? If it is, it sounds like
a bug in the color extraction.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQMNvaTb7E6uWU7s2JHtQxWcuy1ljks5qRA2KgaJpZM4JAMpH
.
Can you show a screenshot of the graph overview?
And which version of IDA are you using?
Sorry to bother you, but I was wondering if there was any news on the issue.
On Thu, Jun 30, 2016 at 3:53 PM, Fraser Hood fraserhood@gmail.com wrote:
On Thu, Jun 30, 2016 at 3:26 PM, Tamir Bahar notifications@github.com
wrote:Can you show a screenshot of the graph overview?
And which version of IDA are you using?โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQPs8tOoG6D1jbreYJ4JReH50n0G5ks5qRBhLgaJpZM4JAMpH
.
OK, solved it.
You need to use the function's flowchart, and not just any chart. Should work now. I wonder how it affects performance.
I will let you know.
On Wed, Jul 6, 2016 at 9:25 AM, Tamir Bahar notifications@github.com
wrote:
OK, solved it.
You need to use the function's flowchart, and not just any chart. Should
work now. I wonder how it affects performance.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQDRjUhEZ59R3eIuNyJEK2l38bJJfks5qS6zQgaJpZM4JAMpH
.
So I am still having a similar issue albeit not exactly the same. The
codeblock colors are not updating properly but the line colors are. That
is, if I set a codeblock to lime green (#00ff00) save that information and
then set it back to white, when I try to change it back to green from the
saved info, it changes the line colors (is what it looks like) to green but
the codeblock stays as white. Ill include some pics to illustrate.
On Wed, Jul 6, 2016 at 9:54 AM, Fraser Hood fraserhood@gmail.com wrote:
I will let you know.
On Wed, Jul 6, 2016 at 9:25 AM, Tamir Bahar notifications@github.com
wrote:OK, solved it.
You need to use the function's flowchart, and not just any chart. Should
work now. I wonder how it affects performance.โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQDRjUhEZ59R3eIuNyJEK2l38bJJfks5qS6zQgaJpZM4JAMpH
.
Can you also say what version of IDA are you using? And on which OS?
Version 6.9.160222 (64-bit)
Windows 10
On Wed, Jul 6, 2016 at 11:19 AM, Tamir Bahar notifications@github.com
wrote:
Can you also say what version of IDA are you using? And on which OS?
โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQPHIj_7TMHfb4RrtC1d8E0jZXKvlks5qS8eGgaJpZM4JAMpH
.
Same for me, and the new blocks code seems to work. Weird.
Yeah I noticed that it seemed like it was retrieving the colors correctly
when I printed it out.
On Thu, Jul 7, 2016 at 2:56 AM, Tamir Bahar notifications@github.com
wrote:
Same for me, and the new blocks code seems to work. Weird.
โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQFtB3HPFOO4ELr4n34ZV5MMIk0y_ks5qTKMMgaJpZM4JAMpH
.
So are there any remaining issues, or can I consider it solved?
Ill let you know. Been really busy.
On Wed, Jul 13, 2016 at 8:03 AM, Tamir Bahar notifications@github.com
wrote:
So are there any remaining issues, or can I consider it solved?
โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQLLGqFfSzgp4nd6wMg0dUNdWF0ADks5qVNQUgaJpZM4JAMpH
.
Works great! Awesome!
On Thu, Jul 14, 2016 at 9:29 AM, Fraser Hood fraserhood@gmail.com wrote:
Ill let you know. Been really busy.
On Wed, Jul 13, 2016 at 8:03 AM, Tamir Bahar notifications@github.com
wrote:So are there any remaining issues, or can I consider it solved?
โ
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#44 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AIkBQLLGqFfSzgp4nd6wMg0dUNdWF0ADks5qVNQUgaJpZM4JAMpH
.