jianzongwu/betrayed-by-captions

Demo notebook fails at inferenceDetector

Opened this issue · 1 comments

The demo notebook fails with a cuda error, possibly relating to incorrect matrix shapes during multiplications. I'm using the exact same torch/cuda/mmcv/mmdet versions as listed in the README, on a Quaddro RTX 8000 GPU.

The stack trace I get after running the cell

result = inference_detector(model, img, with_caption=True, logging=True)[0]

is

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[8], [line 2](vscode-notebook-cell:?execution_count=8&line=2)
      [1](vscode-notebook-cell:?execution_count=8&line=1) # Predict segmentation results, as well as image captions
----> [2](vscode-notebook-cell:?execution_count=8&line=2) result = inference_detector(model, img, with_caption=True, logging=True)[0]
...
--> [360](~/miniforge3/envs/cgg/lib/python3.8/site-packages/torch/functional.py:360) return _VF.einsum(equation, operands)

RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling `cublasSgemmStridedBatched( handle, opa, opb, m, n, k, &alpha, a, lda, stridea, b, ldb, strideb, &beta, c, ldc, stridec, num_batches)

Have you encountered this before? And any way to fix this?

Can you download your checkpoints and can you share them