This project is still under development.Please look forward to it...
Install Segment Anything:
python -m pip install -e segment_anything
Install Grounding DINO:
python -m pip install -e GroundingDINO
Install diffusers:
pip install --upgrade diffusers[torch]
The following optional dependencies are necessary for mask post-processing, saving masks in COCO format, the example notebooks, and exporting the model in ONNX format. jupyter
is also required to run the example notebooks.
pip install opencv-python pycocotools matplotlib onnxruntime onnx ipykernel
More details can be found in install segment anything and install GroundingDINO
- Download the checkpoint for Segment Anything and Grounding Dino:
cd Grounded-Segment-Anything
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth
- Run Demo
export CUDA_VISIBLE_DEVICES=0
python scene_graph_gen.py \
--config GroundingDINO/groundingdino/config/GroundingDINO_SwinT_OGC.py \
--grounded_checkpoint groundingdino_swint_ogc.pth \
--sam_checkpoint sam_vit_h_4b8939.pth \
--input_image assets/demo.jpg \
--output_dir "outputs" \
--box_threshold 0.25 \
--text_threshold 0.2 \
--iou_threshold 0.5 \
--device "cpu"
then,remove the file "./outputs/scenegraph.json" to "./SGDraw"
- Run Visualization
cd SGDraw
python -m http.server
Open in browser "localhost:8000"
open "Object-Oriented_SG_Drawing.html" file
click on the button "Load JSON" and choose the file "scenegraph.json"