"SA-DETR: Saliency Attention-based DETR for Salienct Object Detection"
- PyTorch >=1.5.0
- Requirements
pip install -r requirements.txt
Fan, Deng-Ping, et al. "Salient objects in clutter." IEEE Transactions on Pattern Analysis and Machine Intelligence 45.2 (2022): 2344-2366.
python main_SOC.py \
--masks \
--no_aux_loss \
--output_dir "output_path" \
--epochs 200 \
--frozen_weights detr-r50-e632da11.pth (or --frozen_weights detr-r101-2c7b67e5.pth --backbone resnet101)
[--resume "output_checkpoint_path" --lr "lr" --lr_drop "lr_drop"]
python main_SOC.py --masks --no_aux_loss --eval
python pred_SOC.py --masks --no_aux_loss --eval
Perazzi, Federico, et al. "Saliency filters: Contrast based filtering for salient region detection." 2012 IEEE conference on computer vision and pattern recognition. IEEE, 2012.
-
$N$ : Pixel numbers -
$Sal$ : Saliency(Output) map -
$G$ : GT map
Fan, Deng-Ping, et al. "Structure-measure: A new way to evaluate foreground maps." Proceedings of the IEEE international conference on computer vision. 2017.
-
$\alpha$ : Balanced parameter, [0, 1], (0.5 default) -
$S_o$ : Object-aware structural similarity -
$S_r$ : Region-aware structure similarity
Fan, Deng-Ping, et al. "Enhanced-alignment measure for binary foreground map evaluation." arXiv preprint arXiv:1805.10421 (2018).
-
$w, h$ : Width, height of map -
$\phi_{FM}$ : Enhanced alignment matrix of forground map
![스크린샷 2024-02-01 오후 9 44 51](https://private-user-images.githubusercontent.com/19163372/301507698-fa9494b0-af47-4389-b5cb-a94b985128ef.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA2ODY0MzcsIm5iZiI6MTcyMDY4NjEzNywicGF0aCI6Ii8xOTE2MzM3Mi8zMDE1MDc2OTgtZmE5NDk0YjAtYWY0Ny00Mzg5LWI1Y2ItYTk0Yjk4NTEyOGVmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzExVDA4MjIxN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTVhZGU0NjI4ZGI1ZGNjMzVmMDc4ZDFmODQ4Y2Q0MWU0NTNmZDdkODI3ZmFiYWRkN2I4ZGJkZmM1NGQwM2VmOTImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.GG7H2zys_nzpQ486cI7xqPZgmhKT0wOG-pRhp7VTExU)
![스크린샷 2024-02-01 오후 9 46 11](https://private-user-images.githubusercontent.com/19163372/301508052-693434ef-a739-456f-8958-29c162dc0c66.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA2ODY0MzcsIm5iZiI6MTcyMDY4NjEzNywicGF0aCI6Ii8xOTE2MzM3Mi8zMDE1MDgwNTItNjkzNDM0ZWYtYTczOS00NTZmLTg5NTgtMjljMTYyZGMwYzY2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA3MTElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwNzExVDA4MjIxN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTVhMTU1N2Q5Y2FkYzBiM2YxMjYyYTllNDAwNmI1MWNjOGY2ZTA0Y2Q5MTU0ZGU1NDZhNmYwZDZjMDM3ODdhNjkmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.NBRa8RjFKWu1h3E_kUr6pBt_BReHcjYxZ57gyA8M9J4)
Ablation studies of Saliency Module(SM)
- Without SM, salient objects are not detected, or other objects are detected as salient.
- With SM, each attention map recognizes the shape of an object well, resulting in an accurate object-level mask.