Benchmark guide

Question

Benchmark guide

Bil17t opened this issue 5 years ago · 5 comments

First, congrats on such a successful framework! I would like to test the speed of daBNN on my platform. I can see the benchmark utility, but does it include actual inference on imagenet pictures? Any more detailed instruction?

Answer 1 · 2019-10-17T15:07:21.000Z

Thanks for your attention! You might want to check out https://github.com/JDAI-CV/dabnn-example . I'll also add some detailed document soon.

…

On Thu, Oct 17, 2019, 10:56 PM Bil17t ***@***.***> wrote: First, congrats on such a successful framework! I would like to test the speed of daBNN on my platform. I can see the benchmark utility, but does it include actual inference on imagenet pictures? Any more detailed instruction? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#62?email_source=notifications&email_token=ACYRZH57JOACKFETACHJ7BTQPB4LLA5CNFSM4JB24ZN2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HSPT4EA>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACYRZHZMQZWCT4H6WERAQJTQPB4LLANCNFSM4JB24ZNQ> .

Answer 2 · 2019-10-18T02:43:37.000Z

Which do you want to do, benchmark some models on your side, or do actual inference? Thanks!

Answer 3 · 2019-10-18T12:02:19.000Z

I have tested the speed. It would be better if we can verify the accuracy as well. It will also be easier to deploy other models.

Answer 4 · 2019-10-21T08:48:52.000Z

For verifying the accuracy, I think you can modify the code based on https://github.com/JDAI-CV/dabnn-example and check the output currently.

Answer 5 · 2019-10-21T09:29:26.000Z

And if you want to verify the accuracy more carefully, you can check out 'binaries/run.cpp' about how to send a given input tensor to the model and retrieve the output.