Achieving high performance on sketch-based image retrieval without real sketches for training

Simple item page
dc.coverageDOI: 10.1016/j.patrec.2025.04.018
dc.creatorSaavedra, Jose M.
dc.creatorStears, Christopher
dc.creatorCampos, Waldo
dc.date2025
dc.date.accessioned2025-11-18T19:44:20Z
dc.date.available2025-11-18T19:44:20Z
dc.description<p>Sketch-based image retrieval (SBIR) has become an attractive area in computer vision. Along with the advances in deep learning, we have seen more sophisticated models for SBIR that have shown increasingly better results. However, these models are still based on supervised learning strategies, requiring the availability of real sketch-photo pairs. Having a paired dataset is impractical in real environments (e.g. eCommerce), which can limit the massification of this technology. Therefore, based on advances in foundation models for extracting highly semantic features from images, we propose S3BIR-DINOv2, a self-supervised SBIR model using pseudo-sketches to address the absence of real sketches for training, learnable vectors to allow the model to hold only one encoder for processing the underlying two image modalities, contrastive learning and an adapted DINOv2 as the visual encoder. Our experiments show our model performs outstandingly in diverse public datasets without requiring real sketches for training. We reach an overall mAP of 61.10% in Flickr15K and 44.37% in the eCommerce dataset.</p>eng
dc.identifierhttps://investigadores.uandes.cl/en/publications/f0e2ba90-753e-49d4-ad73-30f3f561c72d
dc.identifier.urihttps://repositorio.uandes.cl/handle/uandes/53347
dc.languageeng
dc.rightsinfo:eu-repo/semantics/restrictedAccess
dc.sourcevol.193 (2025) p.94-100
dc.subjectBimodal representation learning
dc.subjectSelf-supervision
dc.subjectSketch-based image retrieval
dc.titleAchieving high performance on sketch-based image retrieval without real sketches for trainingeng
dc.typeArticleeng
dc.typeArtículospa
Files
Collections
PURE