leyaoliatan/Sentiments-in-Tweets-with-Emojis

Understanding the meaning of emojis with tweets from twitter api. Using emojis to better forecast the sentiment classes in text.

Sense Exploring of Emojis with Word2Vec Model and Using Emojis to Modify Short-text Sentiments Classification

step1-Data collection

- period: 2022-0415-2022-0528
- filter: retweet or media
- querry: tweeets that contain at least one concerned emoji

Our dataset consists of Tweets which contain at least one of the following emoji: The baseline selection of common-used emoji refers to Ian D. Wood, Sebastian Ruder, 2016

The tag of emoji is besed on human intuition accroding to the original paper, instead of meaning in context.

step2-Data cleaning

2.1-First EDA

2.2-Data cleaning

For English tweets, we follow the following steps:

1. remove url
2. remove user names
3. remove punctuations
4. remove stopwords
5. lower the words
6. lemmatization
7. keep only english characters and emojis

For Chinese tweets, we:

1. remove url
2. remove user names
3. cut the words
4. remove stopwords (en & zh)
5. transform to simplified Chinese
6. remove punctuations (en & zh)
7. remove sensitive Chinese words
8. remove non-chinese words

2.3-Extract pure emojis and text

And, we generate 3 new coloumns which:

• remove all emojis

---

• include all emojis

---

• only include concerned emojis

For English tweets, we follow the following steps:

1. remove url
2. remove user names
3. remove punctuations
4. remove stopwords
5. lower the words
6. lemmatization

And, we generate 3 new coloumns which:

    - 1) remove all emojis
    - 1) remove non-english words
    - 2) all emojis
    - 3) only concerned emojis

For Chinese tweets, we follow the following steps:

1. remove url
2. remove user names
3. cut the words
4. remove stopwords (en & zh)
5. transform to simplified Chinese
6. remove punctuations (en & zh)
7. remove sensitive Chinese words
8. remove english words

And, we generate 3 new coloumns which:

    - 1) remove all emojis
    - 2) all emojis
    - 3) only concerned emojis

step3-EDA & Word embedding

3.1 data exploring

1. plot the most frequent emojis

3.2 word embedding

1. Word2Vec word embedding with only emoji
2. word embedding with both emoji and words

step4-Sentiment Classification

4.1 Method 1: only text

4.2 Method 2: text + emojis' description names

4.3 Method 3: text + emojis' most similar text tokens

4.4 method 4: text + pre-trained word embedding model