Error when run interactive demo
STAR-811 opened this issue · 10 comments
Hello, thanks for your work.
I installed the checkpoints and set up the environment but cannot run the demo.
python -m demo.demo frames_path='/home/bbnclxy/ext16t/lyx-star/sam-pt-main/data/demo_data/bees/' query_points_path=null longest_side_length=1024 frame_stride=1 max_frames=-1
it reports error:
I would appreciate it if you could help me.
Hej, it looks like you're encountering an error when Hydra instantiates the model. To help us get a clearer understanding of the problem, could you set the HYDRA_FULL_ERROR environment variable to 1 to display the complete stack trace, which Hydra typically hides during the model instantiation process? You can do this by running export HYDRA_FULL_ERROR=1 in your terminal anytime before executing the python command. After setting this variable, you can run your command again and share the output with us. Thanks!
Hej @STAR-811, have you perhaps resolved your issue? Let us know if you need further guidance or if there are any updates.
@m43 hey,I also encountered the same problem, but after i set HYDRA_FULL_ERROR=1 in my terminal,This mistake still exists，like that
I have resolved the above issue because some third-party libraries from Github have not been fully installed.
@m43 hey,I also encountered the same problem, but after i set HYDRA_FULL_ERROR=1 in my terminal,This mistake still exists，like that
Hi, may I ask what's your solution to the problem? I've encountered the same problem but haven't found the reason/solution yet. Thank you so much!
Hej @ttanad, you can set the HYDRA_FULL_ERROR environment variable to 1 to see the full error. Has detectron2 been successfully installed on your system, i.e., can you run python -c 'import detectron2' without error?
Hej @ttanad, you can set the
HYDRA_FULL_ERRORenvironment variable to1to see the full error. Hasdetectron2been successfully installed on your system, i.e., can you runpython -c 'import detectron2'without error?
Thank you so much for your response! Actually, I hadn't successfully installed the detectron2 since I saw another Issue post that said it's also okay for the demo even if skipping the detectron2. And I struggled for a long time but couldn't properly install detectron2 by your instruction lines and environment. Can you kindly suggest some solution for this case?
Here are some things to try:
- Run
export HYDRA_FULL_ERROR=1in your console and then run the demo. Paste the complete output log here or provide a pastebin link if the output is too long. - Remove
git+https://github.com/facebookresearch/detectron2@v0.6fromrequirements.txtand re-runpip install -r requirements.txt. Try to run the demo again - will the demo work now? - Double check you have the minimal set of checkpoints downloaded as described here.
Thank you so much for your suggestions, after properly downloading all the minimal sets, it started working and the GUI page popped out for me to choose the points. But it seems that I have some problems exiting the page with esc and seeing the result. May I ask what should I expect to see after choosing the points? Is it only the point coordinates or it's expected to see the video as a result?
(sam-pt) C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt>python -m demo.demo frames_path='${hydra:runtime.cwd}/data/demo_data/bees/' query_points_path=null longest_side_length=1024 frame_stride=1 max_frames=-1
C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
ret = run_job(
Setting seed to 72
model:
point_tracker:
target: sam_pt.point_tracker.pips.PipsPointTracker
checkpoint_path: ${hydra:runtime.cwd}/models/pips_ckpts/reference_model
stride: 4
s: 8
initial_next_frame_visibility_threshold: 0.9
sam_predictor:
sam_model:
image_encoder:
target: segment_anything.modeling.image_encoder.ImageEncoderViT
depth: 32
embed_dim: 1280
img_size: ${ ..image_size }
mlp_ratio: 4
norm_layer:
partial: true
target: torch.nn.LayerNorm
eps: 1.0e-06
num_heads: 16
patch_size: ${ ..vit_patch_size }
qkv_bias: true
use_rel_pos: true
global_attn_indexes:
- 7
- 15
- 23
- 31
window_size: 14
out_chans: ${ ..prompt_embed_dim }
prompt_encoder:
target: segment_anything.modeling.prompt_encoder.PromptEncoder
embed_dim: ${ ..prompt_embed_dim }
image_embedding_size:
- ${ ...image_embedding_size }
- ${ ...image_embedding_size }
input_image_size:
- ${ ...image_size }
- ${ ...image_size }
mask_in_chans: 16
mask_decoder:
target: segment_anything.modeling.mask_decoder.MaskDecoder
num_multimask_outputs: 3
transformer:
target: segment_anything.modeling.transformer.TwoWayTransformer
depth: 2
embedding_dim: ${ ...prompt_embed_dim }
mlp_dim: 2048
num_heads: 8
transformer_dim: ${ ..prompt_embed_dim }
iou_head_depth: 3
iou_head_hidden_dim: 256
target: sam_pt.modeling.sam.SamHydra
checkpoint: ${hydra:runtime.cwd}/models/sam_ckpts/sam_vit_h_4b8939.pth
prompt_embed_dim: 256
image_size: 1024
vit_patch_size: 16
image_embedding_size: 64
pixel_mean:
- 123.675
- 116.28
- 103.53
pixel_std:
- 58.395
- 57.12
- 57.375
target: segment_anything.predictor.SamPredictor
target: sam_pt.modeling.sam_pt.SamPt
sam_iou_threshold: 0.7
iterative_refinement_iterations: 12
positive_point_selection_method: kmedoids
negative_point_selection_method: mixed
positive_points_per_mask: -1
negative_points_per_mask: -1
add_other_objects_positive_points_as_negative_points: true
point_tracker_mask_batch_size: 5
use_patch_matching_filtering: false
patch_size: 3
patch_similarity_threshold: 0.01
use_point_reinit: false
reinit_point_tracker_horizon: 8
reinit_horizon: 8
reinit_variant: reinit-on-horizon-and-sync-masks
logging:
wandb:
entity: null
project: demo
tensorboard: true
log_code_path: ${hydra:runtime.cwd}/sam_pt
debug: false
exp_id: debug_72_2023.09.27_00.06.50
frames_path: ${hydra:runtime.cwd}/data/demo_data/bees/
query_points_path: null
longest_side_length: 1024
frame_stride: 1
max_frames: -1
seed: 72
annot_size: 16
annot_line_width: 6
wandb: Currently logged in as: ttanad664. Use wandb login --relogin to force relogin
wandb: wandb version 0.15.11 is available! To upgrade, please run:
wandb: $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.15.3
wandb: Run data is saved locally in C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\outputs\2023-09-27\00-06-50\wandb\run-20230927_000652-rxqaxdhh
wandb: Run wandb offline to turn off syncing.
wandb: Syncing run debug_72_2023.09.27_00.06.50
wandb: View project at https://wandb.ai/ttanad664/demo
wandb: View run at https://wandb.ai/ttanad664/demo/runs/rxqaxdhh
C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\torch\nn\functional.py:3499: UserWarning: The default behavior for interpolate/upsample with float scale_factor changed in 1.6.0 to align with other frameworks/libraries, and now uses scale_factor directly, instead of relying on the computed output size. If you wish to restore the old behavior, please set recompute_scale_factor=True. See the documentation of nn.Upsample for details.
warnings.warn(
How many positive points? 2
How many negative points? 1
Added positive point 1 at (272, 215)
Added positive point 2 at (224, 201)
Added negative point 3 at (401, 236)
Added query points for mask 1, you can now select points for the next mask or press esc to exit.
Added positive point 1 at (274, 174)
Added positive point 2 at (251, 163)
Added negative point 3 at (361, 271)
Added query points for mask 2, you can now select points for the next mask or press esc to exit.
Added positive point 1 at (907, 348)
Added positive point 2 at (911, 326)
Added negative point 3 at (802, 340)
Added query points for mask 3, you can now select points for the next mask or press esc to exit.
Added positive point 1 at (938, 332)
Added positive point 2 at (891, 339)
Added negative point 3 at (767, 334)
Added query points for mask 4, you can now select points for the next mask or press esc to exit.
Added positive point 1 at (904, 277)
Added positive point 2 at (984, 286)
Added negative point 3 at (917, 115)
Added query points for mask 5, you can now select points for the next mask or press esc to exit.
Traceback (most recent call last):
File "C:\Users\ttana.conda\envs\sam-pt\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\ttana.conda\envs\sam-pt\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 372, in
main()
File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra\main.py", line 94, in decorated_main
_run_hydra(
File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 394, in _run_hydra
_run_app(
File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 457, in _run_app
run_and_report(
File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 220, in run_and_report
return func()
File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\utils.py", line 458, in
lambda: hydra.run(
File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra_internal\hydra.py", line 119, in run
ret = run_job(
File "C:\Users\ttana.conda\envs\sam-pt\lib\site-packages\hydra\core\utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 45, in main
rgbs, positive_points_per_mask, query_points = load_data(cfg)
File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 96, in load_data
return data_loader(
File "C:\Users\ttana\OneDrive - HKUST Connect\Data\Point_tracking\sam-pt\demo\demo.py", line 341, in load_demo_data_interactive
k = cv2.waitKey(1)
KeyboardInterrupt
wandb: Waiting for W&B process to finish... (failed 255). Press Ctrl-C to abort syncing.
wandb:
wandb: Run summary:
wandb: work_dir C:\Users\ttana\OneDr...
wandb:
wandb: View run debug_72_2023.09.27_00.06.50 at: https://wandb.ai/ttanad664/demo/runs/rxqaxdhh
wandb: Synced 6 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: .\wandb\run-20230927_000652-rxqaxdhh\logs
After you press escape, you should get a message saying Below is the query point data... and then the inference would run, followed by displaying the segmentation result in the same cv2 window. Since cv2 does not recognize you clicking escape, make sure you have the cv2 window in focus when clicking escape (e.g., clicking on the titlebar puts a window into focus). If this does not work, I'd recommend you consider using a different termination criteria, here is the entry point.