FoundationVision/GLEE

How to run the demo locally and correctly?

Opened this issue · 6 comments

ifsheldon commented

Hi! I've read README and tried to run the demo on my server, but I think there're a lot of code that is out of sync or missing. And the guides are incomplete.

Here are my steps:

  1. I installed the dependencies according to INSTALL.md
  2. I ran the app.py in this repo
  3. A lot of models are missing:
    • I need GLEE models so I downloaded them on huggingface demo directory
    • Misc models are missing, then I found the ones mentioned in TRAIN.md and downloaded them
  4. There are bugs in app.py on lines like (outputs,_) = GLEEmodel(...), which should be ((outputs, _), _, _) = GLEEmodel(...)
  5. Then I ran the app.py in this repo again, but the results are just random like below.
image

Did I do anything wrong? Should I just clone the huggingface repo instead?

knightdby commented

+1

  • 👍1
heroinlin commented

@ifsheldon @knightdby
check out to huggingface repo, and it work well!
image
image

cteant commented

The problem is that in app.py when loading the glee model, the key does not match and the model is not correctly loaded. Use the following code in L88 in app.py

GLEEmodel_r50.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_r50.items()}, strict=False)

Also in L98 in app.py

GLEEmodel_swin.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_swin.items()}, strict=False)
  • 👍5
2anchao commented

The problem is that in app.py when loading the glee model, the key does not match and the model is not correctly loaded. Use the following code in L88 in app.py

GLEEmodel_r50.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_r50.items()}, strict=False)

Also in L98 in app.py

GLEEmodel_swin.load_state_dict({k.replace('glee.', ''): v for k, v in checkpoints_swin.items()}, strict=False)

nice

victoriazinkovich commented

When testing app.py with the model GLEE_Lite_joint.pth, I also encountered an error at line 127. The issue was with the image transposition. To fix this, change the following line:

ori_image = torch.as_tensor(np.ascontiguousarray(copyed_img.transpose(2, 0, 1)))

to:

ori_image = torch.as_tensor(np.ascontiguousarray(np.transpose(copyed_img, (2, 0, 1))))

This resolves the TypeError: Image.transpose() takes 2 positional arguments but 4 were given error.