Resources (conference/journal publications, references to datasets) for harmful memes detection.
This repo contains relevant resources Automatic Harmful Memes Detection. We list a comprehensive and up-to-date information for harmful meme detection.
-
Harmful Memes:
-
Hateful Memes: Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, Davide Testuggine, The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes, 2020. [Paper] [Dataset]
-
Fine-grained Propaganda Memes: Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov and Giovanni Da San Martino, Detecting Propaganda Techniques in Memes, 2021 [Paper] [Dataset]
-
Troll Memes: Shardul Suryawanshi, Bharathi Raja Chakravarthi, Pranav Varma, Mihael Arcan, John P. McCrae and Paul Buitelaar, A Dataset for Troll Classification of TamilMemes, 2020. [Paper] [Dataset]
| Title | Types | Task | Dataset | Task Type | Approach | AUC | Acc | F1 | Details |
|---|---|---|---|---|---|---|---|---|---|
| Detecting Harmful Memes and Their Targets | Harm | Harmful vs Non-harmful | HarMeme (Covid-19) | B | VisualBERT (Pretrained using MS COCO) | 0.81 | 0.8 | ||
| Very harmful vs Partially-harmful vs Non-harmful | M | 0.74 | 0.54 | ||||||
| Target Identification of Harmful Memes | M | 0.76 | 0.66 | ||||||
| MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targets | Harm | Harmful vs Non-harmful | Harm-C | B | MOMENTA: CLIP, VGG-19, DistilBERT, Cross-modal attention fusion (CMAF) | 0.84 | 0.83 | ||
| Very harmful vs Partially-harmful vs Non-harmful | M | 0.77 | 0.55 | ||||||
| Target Identification of Harmful Memes | M | 0.78 | 0.7 | ||||||
| Harmful vs Non-harmful | Harm-P | B | 0.9 | 0.88 | |||||
| Very harmful vs Partially-harmful vs Non-harmful | M | 0.87 | 0.67 | ||||||
| Target Identification of Harmful Memes | M | 0.79 | 0.69 | ||||||
| Racist or Sexist Meme? Classifying Memes beyond Hateful | Hate | Protected category (PC) identification | FBHM | ML | CIMG + CTXT + LASER + LaBSE |
0.96 | |||
| Detecting attack type (AT) | ML | 0.97 | |||||||
| “Subverting the Jewtocracy”: Online Antisemitism Detection Using Multimodal Deep Learning | Hate | Antisemitic content detection | GAB | B | MFAS | 0.91 | |||
| 0.71 | |||||||||
| Antisemitism category classification | GAB | M | 0.67 | ||||||
| 0.68 | |||||||||
| Detecting Sexist MEME On The Web: A Study on Textual and Visual Cues | Hate | Automatic detection of sexist memes. | The MEME | B | Late fusion | 0.76 | Multi-modal Late-fusion Hand-crafted visual dec\scriptiors used: Low-level greyscale, colored, photographic and semantic features. Bag-of-words approached used as textual features. SVM |
||
| Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset |
Hate | Hateful Meme Detection | FBHM | CLIP (Linear Probe) | 0.56 | ||||
| B | 0.57 | ||||||||
| Hate Speech in Pixels: Detection of Offensive Memes towards Automatic Moderation | Hate | Hateful Meme Detection | BERT, VGG-16, MLP | 0.83 | |||||
| Disentangling Hate in Online Memes | Hate | Hateful Meme Detection | FBHM | B | DisMultiHate (BERT, Faster-RCNN, Disentangled representations) | 0.83 | 0.76 | ||
| MultiOFF | 0.65 | ||||||||
| Exploring Hate Speech Detection in Multimodal Publications | Hate | Hatespech detection in multimodal publications | MMHS150K | B | FCM (Feature concatenation model), Inception-V3, LSTM | 0.73 | 0.68 | 0.70 | |
| AOMD: An Analogy-aware Approach to Offensive Meme Detection on Social Media | Offensive | Offensive (analogy) meme detection | GAB | Analogy-aware Multi-modal Representation Learning (Faster R-CNN, ResNet50, Glove-based LSTM, BERT) MLP |
0.69 | 0.56 | |||
| B | 0.72 | 0.49 | |||||||
| KnowMeme: A Knowledge-enriched Graph Neural Network Solution to Offensive Meme Detection | Offensive | Offensive meme detection | YOLO V4, ConceptNET, GNN | 0.73 | 0.49 | Knowledge-aware Multimodal Entity Extraction (KMEE): YOLO V4 Knowledge-enriched Information Network Construction: ConceptNET Supervised Offensive Meme Classification: GNN |
|||
| GAB | B | 0.7 | 0.55 | ||||||
| An approach to detect offence in Memes using Natural Language Processing(NLP) and Deep learning | Offensive | Offensive meme detection | Offensive + Intensity dataset | CNN, FastText, LSTM - Sigmoid | 0.96 | ||||
| Offense intensity prediction | M | CNN, FastText, LSTM - Softmax | 1 | ||||||
| Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text | Offensive | Offensive content detection | MultiOFF | B | Early fusion: Stacked LSTM/ BiLSTM/CNN-Text + VGG16 | 0.5 | Early fusion technique (Stacked LSTM/ BiLSTM/CNN-Text + VGG16) | ||
| Detecting Propaganda Techniques in Memes | Propaganda | Detecting the type of propaganda techniques used in memes |
VisualBERT (Pretrained using MS COCO) | 0.48 | micro F1 | ||||
| MinD at SemEval-2021 Task 6: Propaganda Detection using Transfer Learning and Multimodal Fusion | Propaganda | Propaganda technique detection (Unimodal: Text) | ML | Ensemble: BERT, RoBERTa, XLNet, ALBERT, DistilBERT, DeBERTa, Embeddings, Char n-gram | 0.59 | micro F1, Team name: MinD | |||
| Volta at SemEval-2021 Task 6: Towards Detecting Persuasive Texts and Images using Textual and Multimodal Ensemble | Propaganda | Propaganda technique and span detection (Unimodal: Text) | ML | RoBERTa | 0.48 | micro F1, Team name: Volta | |||
| Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification | Propaganda | Propaganda technique detection (Multimodal: Meme) | ML | RoBERTa, Embeddings | 0.58 | micro F1, Team name: Alpha | |||
| Detection of Cyberbullying Incidents on the Instagram Social Network | Cyber-bullying | Detecting incidents of cyber-bullying | B | SVD +(Unigram, 3-gram), kernelPCA+(meta data, image categories) + lin. SVM | 0.87 | ||||
| A Dataset for Troll Classification of TamilMemes | Cyber-bullying | Detecting Troll memes | TamilMemes | B | ResNET (Training: TamilMemes) | 0.52 | macro F1 | ||
| ResNET (Training: TamilMemes + ImageNet) | 0.52 | ||||||||
| MobileNet (Training: TamilMemes + ImageNet + Flickr1k) | 0.47 | ||||||||
| ResNET (Training: TamilMemes + ImageNet + Flickr30k) | 0.52 | ||||||||
| Multimodal Sentiment Analysis To Explore the Structure of Emotions | - | Multimodal emotiom detection | Tumblr | M | Early fusion: Inception V3, LSTM | 0.72 | |||
| Multimodal Classification for Analysing Social Media | Common space fusion: InceptionNet, fastText, SVM | 0.87 | 0.85 | ||||||
| Flickr | 0.93 | 0.91 |