my predict method i taking too much time even i am using GPU with 8 GB ram why is it taking too much time
DeveloperRachit opened this issue · 27 comments
my predict method i taking too much time even i am using GPU with 8 GB ram why is it taking too much time
i am using retinanet_resnet152_500_classes_0.4991.h5
that pretrained model
Can you post exact time?
yes it's taking 120 sec to predict a images objects
- You need to ensure GPU is used
- Long time is possible for first recognition - because of long model initialization. Try to recognize several images and check how much time it requires for each.
i am sure it's using GPU Memory full after using GPU why is it taking too much time
for every image it's taking 120 sec i tried many images
Did you load model before each image? Or use the same?
if you want i could show you my code also
this is for loading model
model_path = "/data/sample-apps/deep_dive_demos/open_images_detection/preprocessing/retinanet_resnet152_lt.h5" model = models.load_model(model_path, backbone_name='resnet152')
it's taking toommuch time to load
then i am using
boxes, scores, labels =model.predict(np.expand_dims(image, axis=0))
it's taking 60sec
so whole time taking for both 120sec
yes i am loading before each images
actually i am using single single images not for multiple
i simple made a userinterface where user will upload image and click onn detect when that person click on detect firstly it will load model and then i will predict for every uploading images is doing same
my api after uploading images load model and then predict objects
That's strange. It shouldn't take more than a second. Which tensorflow and keras version you use?
Did you try resnet101 and resnet50?
no i tried resnet152
tensorflow-gpu==1.14.0
Keras==2.3.1
i am using these versions
Looks fine. Do you use model for inference?
Try model based on resnet50 and check timing.
i tried on resnet50 but taking 120 sec for it also
i am using model for inferecce.
Sorry don't know what it can be (
Did you try script with inference example?
https://github.com/ZFTurbo/Keras-RetinaNet-for-Open-Images-Challenge-2018/blob/master/retinanet_inference_example.py
yes i tried
During this 60 seconds for inference can you check GPU usage?
Thu Apr 23 15:11:45 2020 +-----------------------------------------------------------------------------+ | NVIDIA-SMI 410.93 Driver Version: 410.93 CUDA Version: 10.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Quadro P4000 Off | 00000000:81:00.0 Off | N/A | | 53% 55C P0 35W / 105W | 7901MiB / 8119MiB | 11% Default | +-------------------------------+----------------------+----------------------+ +-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | 0 7759 C /usr/bin/python3 5779MiB | | 0 9620 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10424 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10428 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10430 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10436 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10438 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10440 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10441 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10442 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10444 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 10447 C /usr/local/AV/bin/ffmpeg 251MiB | | 0 11329 C /usr/local/AV/bin/ffmpeg 137MiB | | 0 14191 C /usr/local/AV/bin/ffmpeg 204MiB | | 0 25039 C /usr/local/AV/bin/ffmpeg 139MiB | +-----------------------------------------------------------------------------+
it's taking 40 sec to load model and rest 25 sec to predict.
is there any way to reduce time of model loading either we can do it only once to loading model?
I don't think it's possible to reduce load model time. But you can load model once and keep it in memory while processing images.
i am using my api when i load my model once it give me error when i pass image to predict method
ValueError: Tensor Tensor("filtered_detections/map/TensorArrayStack/TensorArrayGatherV3:0", shape=(?, 500, 4), dtype=float32) is not an element of this graph.
could you help me how to load model once in memory with tf session ?