/sbir_state_of_the_art

Collection of subarasii and reproducible SBIR works.

MIT LicenseMIT

Sketch-based Image Retrieval Algorithm

Collection of subarasii and reproducible SBIR works.

Criteria: works must have codes available, and the reproducible results demonstrate state-of-the-art.

This collection is inspired by the image denoising collection. It makes me have an idea, why don't I summarize the SBIR works just like he did.

Feature descriptor

Global

  • GF-HoG [Web] [Web] [Code] [PDF]
    • Gradient field descriptor for sketch based retrieval and localization (ICIP, 2010), Rui Hu et al.
    • A Bag-of-Regions Approach to Sketch Based Image Retrieval (ICIP, 2011), Rui Hu et al.
    • A Performance Evaluation of Gradient Field HOG Descriptor for Sketch Based Image Retrieval (COMPUT VIS IMAGE UND, 2013), Rui Hu et al.
  • Color GF-HoG [Web] [Code] [PDF]
    • Scalable Sketch-based Image Retrieval using Color Gradient Features (ICIP, 2015), Tu Bui et al.

Local

  • SHOG [Code(msvs)] [Code(origin)]
    • Sketch-based image retrieval: Benchmark and bag-of-features descriptors (IEEE T VIS COMPUT GR, 2011), Eitz M et al.
    • A descriptor for large scale image retrieval based on sketched feature lines (SBM, 2009), Eitz M et al.
    • How do humans sketch objects? (TOG, 2012), Eitz M et al.

Deep Learning

Modelling

1: Fine-grained retrieval

Hash Coding

  • Multi-column weight sharing
    • Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval (CVPR, 2017), Li L et al. [Web] [Code(Caffe)] [PDF]

Benchmark and Dataset

  • Flickr15K [Web] [Image(source and groundtruth)] [Image(resized)] [Sketch] [Image(source and groundtruth)] [Image(resized)] [Sketch] [groundtruth] [PDF]
    • A Performance Evaluation of Gradient Field HOG Descriptor for Sketch Based Image Retrieval (CVIU 2013), Rui Hu et al.
    • 14,660 images labelled into 33 categories based on shape only
    • 10 non-expert sketchers(5 males, 5 females)
    • 330 free-hand sketches
  • Flickr25K [Web] [Image] [PDF]
    • Compact Descriptors for Sketch-based Image Retrieval using a Triplet loss Convolutional Neural Network (CVIU 2017), Tu Bui et al.
    • 25,000 images labelled into 250 categories
    • images only
    • can be used associated with TU-Berlin, shown as follows
  • TU-Berlin [Web] [Sketch] [PDF]
    • How Do Humans Sketch Objects? (Siggraph 2012), Eitz M et al.
    • 20,000 unique sketches evenly distributed over 250 object categories
    • sketch only
  • Image100k [Web] [Image] [Benchmark/Sketch] [PDF]
    • Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors (IEEE T VIS COMPUT GR 2012), Eitz M et al.
    • [Benchmark] has 1,240 images labelled into 31 categories and 31 corresponding query sketches for testing
    • [Image] has 101,240 images for training
    • 28 participants(23 males, 5 females)
  • GOLD [Web] [Image(resized)] [PDF]
    • Sketch-Based Image Retrieval by Salient Contour Reinforcement (IEEE T MULTIMEDIA 2016), Zhang Y et al.
    • contains more than 22,000 Flickr Crawled images together with their Geotags
    • extend image set, image only
  • Sketchy [Web] [Image&Sketch] [PDF]
    • The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies (TOG, 2016), Sangkloy P et al.
    • sampled from 125 categories and acquire 75,471 sketches of 12,500 objects
    • a benchmark contained
  • SketchyScene [Web] [PDF]
    • SketchyScene: Richly-Annotated Scene Sketches (ECCV, 2018), Changqing Z et al.