🍇 [Read our arXiv Paper] 🍎 [Try our Demo] 🍊 [Walk through Project Page]
We introduce FIND that can INterfacing Foundation models' embeDDings in an interleaved shared embedding space. Below is a brief introduction of all the generic and interleave tasks we can do!
- [2023.12.3] 🔥 We have a poster session@NeurIPS for SEEM, feel free to visit us during 5:00-7:00pm (CT)!
- [2023.12.2] We have released all the training, evaluation, and demo code!
- Demo Code
- Model Checkpoint
- Comprehensive User Guide
- Dataset
- Training Code
- Evaluation Code
| entity_train2017.json | entity_val2017.json | entity_val2017_long.json |
|---|---|---|
| download | download | download |
| COCO-Entity | COCO-Entity-Long | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| cIoU | AP50 | IR@5 | IR@10 | cIoU | AP50 | IR@5 | IR@10 | ||
| ImageBIND (H) | - | - | - | 51.4 | 61.3 | - | - | 58.7 | 68.9 |
| Grounding-SAM (H) | - | 58.9 | 63.2 | - | - | 56.1 | 62.5 | - | - |
| Focal-T | ckpt | 74.9 | 79.5 | 43.5 | 57.1 | 73.2 | 77.7 | 49.4 | 63.9 |
| Focal-L | ckpt |
- Example Output
