CBIR-GAN: A Triplet Generative Adversarial Network for Content-based Image Retrieval
Image embedding algorithm in SnapMod project : https://www.snapmode.ir/
Unfortunately, due to the commercial nature of the product, I can not share its code. Please Find the paper at this URL: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4057354
Content-based image retrieval systems have become popular in various fields such as computer vision and artificial intelligence. Deep neural networks, especially CNNs, have frequently been employed in visual representations; however, they require large amounts of labeled data, which are hard, costly, and sometimes impossible to obtain. Moreover, since these methods rely only on semantically discriminative representations, they fail to yield significant outputs in instance-level image retrieval systems. Therefore, this paper proposes a triplet generative adversarial network (GAN) based on the idea of integrating deep metric learning methods with GANs to benefit from the advantages of both at the same time. In this model, three generator networks that use a triplet loss function are responsible for learning a similarity measure over objects and embedding images in an appropriate vector space. In these networks, a CNN-based perceptual loss function is also employed to force the generators to adhere a certain type of structural features in intermediate layers. Since only triplets must be used as network inputs in the proposed method, the learning process is performed in a semi-supervised way. According to the results of comprehensive experiments conducted on four datasets in comparison with several state-of-the-art methods, the proposed method was efficient in terms of precision and computational complexity. For the real-life implementation of the proposed method, a distributed large-scale fashion image retrieval platform, called SnapMode1, has been developed through big data tools such as Apache Storm, Kafka, Solr, and Milvus.
The schematic view of the CBIR-GAN
Evaluation Results of Baseline methods on Different Datasets
Generator and Discriminator Networks
Comparing the Efficiency of the Proposed Model in Image Generation Task
Qualitative Results Comparing Our Proposed Model VS Other State-Of-The-Art CNN Models on Deepfashion2
Examples of Images retrieved by Proposed Model from Eight Iranian Fashion Website
