SiLK is a self-supervised framework for learning keypoints. As described in our paper (by Pierre Gleize, Weiyao Wang and Matt Feiszli), SiLK focuses on simplicity and flexibility, while also providing state-of-art and competitive results on existing benchmarks.
Pre-trained models are also provided.
- conda should be installed in order to setup the silk environment.
conda env create -f environment.yml
conda activate silkAn example of using SiLK for inference is provided in silk-inference.py. The pre-trained models and inference parameters are specified in common.py.
This can be run with:
python silk-inference.pyThe output will be output to matches.png.
A very important parameter is the SILK_DEFAULT_OUTPUT. It specifies the output(s) that is required from the model. When running the model, it will output a tuple of the same size as SILK_DEFAULT_OUTPUT, with the corresponding output at each position.
Valid output names for SILK_DEFAULT_OUTPUT are :
features: shared feature map being fed to the keypoint head and descriptor head.logits: dense output of the keypoint head (pre-sigmoid)probability: dense output of the keypoint head (post-sigmoid)raw_descriptors: unnormalized dense descriptorssparse_positions: positions of keypoints (with confidence value)sparse_descriptors: descriptors of each keypointsdense_positions: positions of all pixelsdense_descriptors: descriptors of all pixels
Remark : SiLK will ONLY run the minimum required computation to obtain the specified output. Any irrelevant computation will be avoided, in order to keep the computation effcient. For example, if one specifies to output "dense_descriptors" only, the keypoint scoring branch of the model will never be computed.
The results below have been computed using our VGG-4 backbone (checkpoint pvgg-4.ckpt for tab 2,3,6 and coco-rgb-aug for tab 4,5). To reproduce the IMC2022 results, we also provide the Kaggle notebook.
We provide a documentation, but it is non-exhaustive. Please create a new issue if clarification is required regarding some part of the code. We will add documentation if required by the community.
Our documentation can be found here.
See the CONTRIBUTING file for how to help out.
SiLK is licensed under a GNU General Public License (Version 3), as specified in the LICENSE file.