Attempt to run X-Decoder demo in google colab

Question

Attempt to run X-Decoder demo in google colab

gdmcdonald opened this issue 2 years ago · 6 comments

Hi there! The hugging face demo is super cool! Thanks for sharing! :)

Today I tried running inference in google colab.

I installed following your instructions with:

!git clone https://github.com/microsoft/X-Decoder.git
!pip3 install torch==1.13.1 torchvision==0.14.1 --extra-index-url https://download.pytorch.org/whl/cu113
!python -m pip install 'git+https://github.com/MaureenZOU/detectron2-xyz.git'
!pip install git+https://github.com/cocodataset/panopticapi.git
!curl -O https://projects4jw.blob.core.windows.net/x-decoder/release/xdecoder_focalt_last_novg.pt
!python -m pip install -r X-Decoder/requirements.txt

And then ran with:

%cd /content/X-Decoder
!python demo/demo_captioning.py evaluate --conf_files configs/xdecoder/svlp_focalt_lang.yaml  --overrides WEIGHT ../xdecoder_focalt_last_novg.pt

And got the following segmentation fault error:

/content/X-Decoder
$UNUSED$ criterion.empty_weight, Ckpt Shape: torch.Size([134])
/usr/local/lib/python3.8/dist-packages/torchvision/transforms/transforms.py:329: UserWarning: Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. Please use InterpolationMode enum.
  warnings.warn(
2023-01-03 10:20:33.905664: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-03 10:20:35.257457: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib/python3.8/dist-packages/cv2/../../lib64:/usr/lib64-nvidia
2023-01-03 10:20:35.257602: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/lib/python3.8/dist-packages/cv2/../../lib64:/usr/lib64-nvidia
2023-01-03 10:20:35.257631: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
[c639da4ebb3c:07861] *** Process received signal ***
[c639da4ebb3c:07861] Signal: Segmentation fault (11)
[c639da4ebb3c:07861] Signal code: Address not mapped (1)
[c639da4ebb3c:07861] Failing at address: 0x7f282066f20d
[c639da4ebb3c:07861] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7f2822f11980]
[c639da4ebb3c:07861] [ 1] /lib/x86_64-linux-gnu/libc.so.6(getenv+0xa5)[0x7f2822b50775]
[c639da4ebb3c:07861] [ 2] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(_ZN13TCMallocGuardD1Ev+0x34)[0x7f28233bbe44]
[c639da4ebb3c:07861] [ 3] /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0xf5)[0x7f2822b51605]
[c639da4ebb3c:07861] [ 4] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(+0x13cb3)[0x7f28233b9cb3]
[c639da4ebb3c:07861] *** End of error message ***

Do you have any idea what might be going on?

Answer 1 · 2023-01-03T13:59:24.000Z

The problem seems to come from tensorflow, as tensorboard is only used in training, feel free to uninstall this package and see if the env problem could be bypass.

Answer 2 · 2023-03-30T19:00:44.000Z

@gdmcdonald Could you share a public version of your colab notebook. I too need to use XDecoder for inference

Answer 3 · 2023-03-30T19:18:33.000Z

If you want to try inference code, there are two other options than colab notebook:

Directly use the script here: https://github.com/microsoft/X-Decoder/tree/main/demo
Use Hugging Face demo or its source code: https://huggingface.co/spaces/xdecoder/Demo/tree/main

Answer 4 · 2023-03-30T19:36:10.000Z

Hey @MaureenZOU
Thank you for the inputs, I'll definitely try option1. Since we are using colab for inference, I was thinking of directly using an existing implementation

Answer 5 · 2023-03-30T19:48:22.000Z

I see, feel free to drop any further question

Answer 6 · 2023-03-31T15:53:52.000Z

Hey @MaureenZOU
Thank you for the amazing work. I am attaching my colab notebook here in case anyone else has the same idea.

We are planning to run inference on google street view images for our project and use it as an annotation tool. Any advice on the pertained weights or configuration I should use?