Recommender systems leveraging multimedia content

A table of publications on multimedia recommender systems, in particular recommender systems that leverage multimedia content. This page will be periodically updated to include the most recent works. Please contact us if your work is not in the list.

The table serves as overview and extension of the works discussed in the following survey. Please consider citing it if you used the survey.

@article{deldjoo2020recommender,
  title={Recommender systems leveraging multimedia content},
  author={Deldjoo, Yashar and Schedl, Markus and Cremonesi, Paolo and Pasi, Gabriella},
  journal={ACM Computing Surveys (CSUR)},
  volume={53},
  number={5},
  pages={1--38},
  year={2020},
  publisher={ACM New York, NY, USA}
}

Papers

Audio

Sounds

Year Authors Title Venue Target Item Paper Code Data/Source
2019 Smith et al. Towards a Hybrid RecommendationSystem for a Sound Library. IUI sound_track Link -- EarSketch
WordNet
FreeSound
2016 Ostuni et al. Sound and Music Recommendation with Knowledge Graphs. TIST sound_track Link -- Songfacts
Last.fmFreeSound
2015 Oramas et al. A Semantic Hybrid Approach for Sound Recommendation. WWW sound_track Link -- DBpedia
[WordNet]

Music/Artist:

Year Authors Title Venue Target Item Link Code Data
2019 Vall et al. Feature-Combination Hybrid Recommender Systems for Automated Music Playlist Continuation. UMUAI music_artist Link Link DS_name
2017 Oramas et al. A Deep Multimodal Approach for Cold-start Music Recommendation.. DLRS music_artist Link Link Million Song Dataset (MSD)
2015 Liang et al. Content-Aware Collaborative Music Recommendation Using Pre-trained Neural Networks. ISMIR music_artist Link Link1
Link2
Million Song Dataset (MSD)
2014 Shah et al. ADVISOR: Personalized Video Soundtrack Recommendation by Late Fusion with Heuristic Rankings. ACM MM music_artist Link -- Last.fm
FourSquare (NA)
GeoVid app
2014 Wang et al. Improving Content-based and Hybrid Music Recommendation Using Deep Learning. ACM MM music_artist Link Echo Nest Taste
7digital
2013 Van den Oord et al. Deep Content-based Music Recommendation. NIPS music_artist Link -- Million Song Dataset (MSD)
2013 Kaminskas et al. Location-aware music recommendation using auto-taggingand hybrid matching. RecSys music_artist Link --
2011 McFee et al. The Natural Language of Playlists. ISMIR music_artist Link MSD
Echo Nest
Last.fm
Art of the Mix(AotM)
2010 Bu et al. Music recommendation by unified hypergraph: combining social media information and music content. ACM MM music_artist Link -- Last.fm
2009 Shao et al. Music recommendation based on acoustic features and user access patterns. ACM Tr. ASLP music_artist Link -- New Wisdom (NA)
2008 Yoshii et al. An Efficient Hybrid Music Recommender System Using an Incrementally Trainable Probabilistic Generative Model. IEEE Tr. ASLP music_artist Link -- DS_name

Image-based Recommendation

target: product (leveraging visual content of the associated images)

Year Authors Title Venue Target Item Link Code Data
2018 Niu et al. Neural Personalized Ranking for Image Recommendation. WSDM social-media Link Flickr YFCC100M
2018 Wen et al. Visual Background Recommendation for Dance Performances Using Deep Matrix Factorization. TOMCCAP birthday-and-party Link -- Pinterest
2017 Peska et al. Towards Recommender Systems for Police Photo Lineup. DLRS* people Link Link Data
2017 Rawat et al. ClickSmart: A Context-Aware Viewpoint Recommendation System for Mobile Photography. IEEE Tr. CSVT arrow Link -- FlickerAPI
2016 Nguyen et al. NowAndThen: a social network-based photo recommendation tool supporting reminiscence. MUM arrow Link -- Oxford Building DS
2015 Geng et al. Learning Image and User Features for Recommendation in Social Networks. ICCV pinterest Link Link Data (Pinterest)
2014 Wu et al. Learning to personalize trending image search suggestion. SIGIR bacteria Link Search Logs
2013 Albanese et al. A multimedia recommender system TOIT design Link -- Uffizi Gallery (NA)
ImdbAPI
2013 Bartolini et al. Recommending multimedia objects in cultural heritage applications. ICIP design Link -- Manual-buit repo about Paestum (NA)
2011 Bourke et al. The social camera: a case-study in contextual image recommendation. IUI arrow Link -- NA
2010 Cui et al. Multiple feature fusion for social media applications. SIGMOD social-media Link FlickerAPI
2010 Albanese et al. A multimedia recommender integrating object features and user behavior. MTAP design Link -- Uffizi Gallery (NA)
2008 Boutemedjet et al. A graphical model for context-aware visual content recommendation. IEEE Tr. MM interface(1) Link -- WashingtonUni. ImageDB
2006 Boutemedjet et al. A generative graphical model for collaborative filtering of visual content. ICDM interface(1) Link -- WashingtonUni. ImageDB

target: product (leveraging visual content of the associated images)

Year Authors Title Venue Target Item Code Data
2018 Yu et al. Aesthetic-based clothing recommendation. outfit (1) WWW (Instagram)
ImageNet
2018 Sun et al. Personalized clothing recommendation combining user social circle and fashion style consistency. outfit (1) MTAP (Instagram)
ImageNet
2017 Alashkar et al. Rule-based facial makeup recommendation system. concealer_64 FC 2017 (Instagram)
ImageNet
2017 Alashkar et al. Examples-Rules Guided Deep Neural Network for Makeup Recommendation. concealer_64 AAAI (Instagram)
ImageNet
2017 Elsweiler et al. Exploiting food choice biases for healthier
recipe recommendation.
recipe_64 SIGIR (Instagram)
ImageNet
2017 Yang et al. Yum-me: a personalized nutrient-based meal recommender system. PIZZA_64 TOIS (Instagram)
ImageNet
2017 Wang et al. What your images reveal: Exploiting visual contents for point-of-interest recommendation. POI_64 WWW (Instagram)
ImageNet
2017 Chu et al. A hybrid recommendation system considering visual information for predicting favorite restaurants. food-and-restaurant_64 WWW source NA
2016 He et al. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. outfit (1) WWW Code AmazonDS
2016 Chi et al. UbiShop: Commercial item recommendation using visual part-based object representation. outfit (1) MTAP ImageNet
2016 Gu et al. iGlasses: A Novel Recommendation System for Best-fit Glasses. glasses_64 SIGIR (Instagram)
ImageNet
2015 McAuley et al. Image-based recommendations on styles and substitutes. outfit (1) SIGIR Code AmazonDS
2015 Farseev et al. Harvesting Multiple Sources for User Profile Learning: aBig Data Study. POI_64 ICMR NUS-MSS
2014 Chung et al. Effect of facial makeup style recommendation on visual sensibility. concealer_64 MTAP (Instagram)
ImageNet
2014 Liu et al. Personalized geo-specific tag recommendation for photos on social websites. tag_64 Tr. MM Flicker API
2011 Lin et al. Image annotation based on recommendation model. tag_64 SIGIR 1.Ground Truth DB of Wash. Uni.
2.MIRFlicker

Video recommendation

target: movie-related content

Year Title Target Item Venue Code Data
2019 Movie Genome: Alleviating New Item Cold Start in Movie Recommendation. cinema MTAP Code MMTF-14K
2018 Exploiting Visual Contents in Posters and Still Frames for Movie Recommendation. cinema IEEE Access IMDB
2018 Using visual features and latent factors for movie recommendation. cinema CBRecSys
2018 Using Visual Features based on MPEG-7 and Deep Learning for Movie Recommendation. cinema IJMIR Mise-en-Scène DS: MPEG-7 + CNN
2018 Audio-visual encoding of multimedia content for enhancing movie recommendations. cinema RecSys MMTF-14K
2016 Latent factor representations for cold-start video recommendation. cinema RecSys 1:Video Emotion DS (Av. upon Req.)
2: Amazon Product
2016 Content-based video recommendation system based on stylistic visual features. cinema JoDS
2013 Affective recommendation of movies based on selected connotative features. cinema RecSys Gratest film scenes
2013 VideoTopic: Content-Based Video Recommendation Using a Topic Model. cinema MTAP IMDB + YouTube

User-generated video

Year Authors Title Target Item Venue Code Data/Resources
2018 Ma et al. LGA: latent genre aware micro-video recommendation on social media. elearning-1 MTAP -- Twitter Streaming API
2014 Cui et al. What videos are similar with you?: Learning a common attributed representation for video recommendation elearning-1 ACM MM -- Tencent Weibo*
2013 Deng et al. Personalized video recommendation based on cross-platform user modeling. elearning-1 ICME -- Google+,YouTube
2012 Zhao et al. On video recommendation over social network. elearning-1 MMM -- Facebook /
Aspel+WordNet
2012 Zhao et al. Personalized video recommendation based on viewing history with the study on youtube. elearning-1 ICIMCS -- YouTube
2011 Mei et al. Contextual video recommendation by multimodal relevance and user feedback. elearning-1 ACM TOIS -- MSN Video
2007 Yang et al. Online video recommendation based on multimodal fusion and relevance feedback. elearning-1 CIVR -- MSN Soapbox (NA)

*Tencent Weibo is a Twitter-style social media in China.

Contact

For questions, please feel free to contact us at deldjooy@acm.org.

Authors: