Analyzing 1958 - 2018 Billboard Chart Song Lyrics

Highlighting popular and trending words in song lyrics using word clouds.

Skills and knowledge: Python, Jupyter Notebook, Pandas, Data scraping, Data cleansing, Data analysis

import requests
from bs4 import BeautifulSoup
import pandas as pd
import numpy as np

from wordcloud import WordCloud

import matplotlib.pyplot as plt
% matplotlib inline

1. Collect lists of songs from different years:

# 1. Collect lists of songs from different years.

# https://www.billboard.com/charts/hot-100/1958-08-01

prepare_df = []

start = "https://www.billboard.com/charts/hot-100/"
year = 1958
end = "-08-01"

while year <= 2018:
    
    url = start + str(year) + end
    response = requests.get(url)

    soup = BeautifulSoup(response.text, 'html.parser')
    songs = soup.find_all("div", class_="chart-list-item")

    index = 0

    while index < len(songs):
        for song in songs:
            artist = songs[index]['data-artist']
            title = songs[index]['data-title']
            
            prepare_df.append({'year': year,
                               'artist': artist,
                               'title': title,
                               'lyric_url': "",
                               'lyric': ""})
            
            print("\r" + str(year) + " / " + title + " "*100, end='')
            index += 1
    
    year += 10
        
df = pd.DataFrame(prepare_df, columns = ['year', 'artist', 'title', 'lyric_url', 'lyric'])
    
df
2018 / Can't Take A Joke                                                                                                    
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
year artist title lyric_url lyric
0 1958 Ricky Nelson Poor Little Fool
1 1958 Perez Prado And His Orchestra Patricia
2 1958 Bobby Darin Splish Splash
3 1958 Elvis Presley With The Jordanaires Hard Headed Woman
4 1958 Kalin Twins When
5 1958 Duane Eddy His Twangy Guitar And The Rebels Rebel-'rouser
6 1958 The Coasters Yakety Yak
7 1958 Jack Scott My True Love
8 1958 The Johnny Otis Show Willie And The Hand Jive
9 1958 Peggy Lee Fever
10 1958 Frankie Avalon Ginger Bread
11 1958 Jimmy Clanton And His Rockets Just A Dream
12 1958 Patti Page Left Right Out Of Your Heart (Hi Lee Hi Lo Hi ...
13 1958 Pat Boone If Dreams Came True
14 1958 Jerry Butler and The Impressions For Your Precious Love
15 1958 The Danleers One Summer Night
16 1958 Jody Reynolds Endless Sleep
17 1958 The Elegants Little Star
18 1958 Doris Day Everybody Loves A Lover
19 1958 Bobby Freeman Do You Want To Dance
20 1958 Johnny Cash And The Tennessee Two Guess Things Happen That Way
21 1958 Johnny Mathis A Certain Smile
22 1958 The Olympics Western Movies
23 1958 Sheb Wooley The Purple People Eater
24 1958 Chuck Willis What Am I Living For
25 1958 Poni-Tails Born Too Late
26 1958 The Crickets Think It Over
27 1958 Jimmie Rodgers Secretly
28 1958 The Four Lads Enchanted Island
29 1958 Dean Martin Angel Baby
... ... ... ... ... ...
670 2018 Morgan Evans Kiss Somebody
671 2018 Becky G + Natti Natasha Sin Pijama
672 2018 Lil Dicky Featuring Chris Brown Freaky Friday
673 2018 Lauren Daigle You Say
674 2018 lovelytheband Broken
675 2018 Dylan Scott Hooked
676 2018 Ciara Level Up
677 2018 TK Kravitz Featuring Jacqueez Ocean
678 2018 Kygo Featuring Miguel Remind Me To Forget
679 2018 Khalid, Ty Dolla $ign & 6LACK OTW
680 2018 Daniel Caesar Featuring H.E.R. Best Part
681 2018 G-Eazy Featuring Yo Gotti & YBN Nahmir 1942
682 2018 Post Malone Featuring Nicki Minaj Ball For Me
683 2018 Carrie Underwood Cry Pretty
684 2018 Martin Garrix Featuring Khalid Ocean
685 2018 Weezer Africa
686 2018 Cole Swindell Break Up In The End
687 2018 twenty one pilots Jumpsuit
688 2018 The Weeknd Call Out My Name
689 2018 Dennis Lloyd Nevermind
690 2018 Kanye West All Mine
691 2018 Migos Narcos
692 2018 Billie Eilish You Should See Me In A Crown
693 2018 Eric Church Desperate Man
694 2018 Wiz Khalifa Featuring Swae Lee Hopeless Romantic
695 2018 A$AP Rocky Featuring Skepta Praise The Lord (Da Shine)
696 2018 NF Lie
697 2018 Kane Brown Lose It
698 2018 Drake Emotionless
699 2018 Drake Can't Take A Joke

700 rows × 5 columns

2. Generate lyric URLs for these songs:

# 2. Generate lyric URLs for these songs.

# https://genius.com/ricky-nelson-poor-little-fool-lyrics
start = "https://genius.com/"
end = "-lyrics"

for index, row in df.iterrows():
    
    df.at[index, 'lyric_url'] = (start 
                                 + df.at[index, 'artist'].lower().replace(" ", "-")
                                 + "-"
                                 + df.at[index, 'title'].lower().replace(" ", "-")
                                 + end)
    
df
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
year artist title lyric_url lyric
0 1958 Ricky Nelson Poor Little Fool https://genius.com/ricky-nelson-poor-little-fo...
1 1958 Perez Prado And His Orchestra Patricia https://genius.com/perez-prado-and-his-orchest...
2 1958 Bobby Darin Splish Splash https://genius.com/bobby-darin-splish-splash-l...
3 1958 Elvis Presley With The Jordanaires Hard Headed Woman https://genius.com/elvis-presley-with-the-jord...
4 1958 Kalin Twins When https://genius.com/kalin-twins-when-lyrics
5 1958 Duane Eddy His Twangy Guitar And The Rebels Rebel-'rouser https://genius.com/duane-eddy-his-twangy-guita...
6 1958 The Coasters Yakety Yak https://genius.com/the-coasters-yakety-yak-lyrics
7 1958 Jack Scott My True Love https://genius.com/jack-scott-my-true-love-lyrics
8 1958 The Johnny Otis Show Willie And The Hand Jive https://genius.com/the-johnny-otis-show-willie...
9 1958 Peggy Lee Fever https://genius.com/peggy-lee-fever-lyrics
10 1958 Frankie Avalon Ginger Bread https://genius.com/frankie-avalon-ginger-bread...
11 1958 Jimmy Clanton And His Rockets Just A Dream https://genius.com/jimmy-clanton-and-his-rocke...
12 1958 Patti Page Left Right Out Of Your Heart (Hi Lee Hi Lo Hi ... https://genius.com/patti-page-left-right-out-o...
13 1958 Pat Boone If Dreams Came True https://genius.com/pat-boone-if-dreams-came-tr...
14 1958 Jerry Butler and The Impressions For Your Precious Love https://genius.com/jerry-butler-and-the-impres...
15 1958 The Danleers One Summer Night https://genius.com/the-danleers-one-summer-nig...
16 1958 Jody Reynolds Endless Sleep https://genius.com/jody-reynolds-endless-sleep...
17 1958 The Elegants Little Star https://genius.com/the-elegants-little-star-ly...
18 1958 Doris Day Everybody Loves A Lover https://genius.com/doris-day-everybody-loves-a...
19 1958 Bobby Freeman Do You Want To Dance https://genius.com/bobby-freeman-do-you-want-t...
20 1958 Johnny Cash And The Tennessee Two Guess Things Happen That Way https://genius.com/johnny-cash-and-the-tenness...
21 1958 Johnny Mathis A Certain Smile https://genius.com/johnny-mathis-a-certain-smi...
22 1958 The Olympics Western Movies https://genius.com/the-olympics-western-movies...
23 1958 Sheb Wooley The Purple People Eater https://genius.com/sheb-wooley-the-purple-peop...
24 1958 Chuck Willis What Am I Living For https://genius.com/chuck-willis-what-am-i-livi...
25 1958 Poni-Tails Born Too Late https://genius.com/poni-tails-born-too-late-ly...
26 1958 The Crickets Think It Over https://genius.com/the-crickets-think-it-over-...
27 1958 Jimmie Rodgers Secretly https://genius.com/jimmie-rodgers-secretly-lyrics
28 1958 The Four Lads Enchanted Island https://genius.com/the-four-lads-enchanted-isl...
29 1958 Dean Martin Angel Baby https://genius.com/dean-martin-angel-baby-lyrics
... ... ... ... ... ...
670 2018 Morgan Evans Kiss Somebody https://genius.com/morgan-evans-kiss-somebody-...
671 2018 Becky G + Natti Natasha Sin Pijama https://genius.com/becky-g-+-natti-natasha-sin...
672 2018 Lil Dicky Featuring Chris Brown Freaky Friday https://genius.com/lil-dicky-featuring-chris-b...
673 2018 Lauren Daigle You Say https://genius.com/lauren-daigle-you-say-lyrics
674 2018 lovelytheband Broken https://genius.com/lovelytheband-broken-lyrics
675 2018 Dylan Scott Hooked https://genius.com/dylan-scott-hooked-lyrics
676 2018 Ciara Level Up https://genius.com/ciara-level-up-lyrics
677 2018 TK Kravitz Featuring Jacqueez Ocean https://genius.com/tk-kravitz-featuring-jacque...
678 2018 Kygo Featuring Miguel Remind Me To Forget https://genius.com/kygo-featuring-miguel-remin...
679 2018 Khalid, Ty Dolla $ign & 6LACK OTW https://genius.com/khalid,-ty-dolla-$ign-&-6la...
680 2018 Daniel Caesar Featuring H.E.R. Best Part https://genius.com/daniel-caesar-featuring-h.e...
681 2018 G-Eazy Featuring Yo Gotti & YBN Nahmir 1942 https://genius.com/g-eazy-featuring-yo-gotti-&...
682 2018 Post Malone Featuring Nicki Minaj Ball For Me https://genius.com/post-malone-featuring-nicki...
683 2018 Carrie Underwood Cry Pretty https://genius.com/carrie-underwood-cry-pretty...
684 2018 Martin Garrix Featuring Khalid Ocean https://genius.com/martin-garrix-featuring-kha...
685 2018 Weezer Africa https://genius.com/weezer-africa-lyrics
686 2018 Cole Swindell Break Up In The End https://genius.com/cole-swindell-break-up-in-t...
687 2018 twenty one pilots Jumpsuit https://genius.com/twenty-one-pilots-jumpsuit-...
688 2018 The Weeknd Call Out My Name https://genius.com/the-weeknd-call-out-my-name...
689 2018 Dennis Lloyd Nevermind https://genius.com/dennis-lloyd-nevermind-lyrics
690 2018 Kanye West All Mine https://genius.com/kanye-west-all-mine-lyrics
691 2018 Migos Narcos https://genius.com/migos-narcos-lyrics
692 2018 Billie Eilish You Should See Me In A Crown https://genius.com/billie-eilish-you-should-se...
693 2018 Eric Church Desperate Man https://genius.com/eric-church-desperate-man-l...
694 2018 Wiz Khalifa Featuring Swae Lee Hopeless Romantic https://genius.com/wiz-khalifa-featuring-swae-...
695 2018 A$AP Rocky Featuring Skepta Praise The Lord (Da Shine) https://genius.com/a$ap-rocky-featuring-skepta...
696 2018 NF Lie https://genius.com/nf-lie-lyrics
697 2018 Kane Brown Lose It https://genius.com/kane-brown-lose-it-lyrics
698 2018 Drake Emotionless https://genius.com/drake-emotionless-lyrics
699 2018 Drake Can't Take A Joke https://genius.com/drake-can't-take-a-joke-lyrics

700 rows × 5 columns

3. Try to get the lyrics from web:

# 3. Try to get the lyrics from web.

for index, row in df.iterrows():
    
    url = df.at[index, 'lyric_url']
    response = requests.get(url)

    try:
        soup = BeautifulSoup(response.text, 'html.parser')
        lyric_soup = soup.find("div", class_="lyrics")

        df.at[index, 'lyric'] = lyric_soup.text.replace("\n", " ")
    except:
        df.at[index, 'lyric'] = np.NaN
        
df
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
year artist title lyric_url lyric
0 1958 Ricky Nelson Poor Little Fool https://genius.com/ricky-nelson-poor-little-fo... I used to play around with hearts That haste...
1 1958 Perez Prado And His Orchestra Patricia https://genius.com/perez-prado-and-his-orchest... NaN
2 1958 Bobby Darin Splish Splash https://genius.com/bobby-darin-splish-splash-l... Splish splash, I was taking a bath Long abou...
3 1958 Elvis Presley With The Jordanaires Hard Headed Woman https://genius.com/elvis-presley-with-the-jord... NaN
4 1958 Kalin Twins When https://genius.com/kalin-twins-when-lyrics When, when you smile, when you smile at me W...
5 1958 Duane Eddy His Twangy Guitar And The Rebels Rebel-'rouser https://genius.com/duane-eddy-his-twangy-guita... NaN
6 1958 The Coasters Yakety Yak https://genius.com/the-coasters-yakety-yak-lyrics Take out the papers and the trash Or you don...
7 1958 Jack Scott My True Love https://genius.com/jack-scott-my-true-love-lyrics I prayed to the Lord to send me a love He se...
8 1958 The Johnny Otis Show Willie And The Hand Jive https://genius.com/the-johnny-otis-show-willie... NaN
9 1958 Peggy Lee Fever https://genius.com/peggy-lee-fever-lyrics Never know how much I love you Never know ho...
10 1958 Frankie Avalon Ginger Bread https://genius.com/frankie-avalon-ginger-bread... You're full of sugar you're full of spice Yo...
11 1958 Jimmy Clanton And His Rockets Just A Dream https://genius.com/jimmy-clanton-and-his-rocke... NaN
12 1958 Patti Page Left Right Out Of Your Heart (Hi Lee Hi Lo Hi ... https://genius.com/patti-page-left-right-out-o... NaN
13 1958 Pat Boone If Dreams Came True https://genius.com/pat-boone-if-dreams-came-tr... Oh my darling, oh my lover If dreams came t...
14 1958 Jerry Butler and The Impressions For Your Precious Love https://genius.com/jerry-butler-and-the-impres... NaN
15 1958 The Danleers One Summer Night https://genius.com/the-danleers-one-summer-nig... One summer night, we fell in love One summer...
16 1958 Jody Reynolds Endless Sleep https://genius.com/jody-reynolds-endless-sleep... VERSE 1 The night was black, rain falling do...
17 1958 The Elegants Little Star https://genius.com/the-elegants-little-star-ly... Where are you little star? (Where are you?) ...
18 1958 Doris Day Everybody Loves A Lover https://genius.com/doris-day-everybody-loves-a... Everybody loves a lover I'm a lover, everybo...
19 1958 Bobby Freeman Do You Want To Dance https://genius.com/bobby-freeman-do-you-want-t... Do you want to dance and hold my hand? Tell ...
20 1958 Johnny Cash And The Tennessee Two Guess Things Happen That Way https://genius.com/johnny-cash-and-the-tenness... NaN
21 1958 Johnny Mathis A Certain Smile https://genius.com/johnny-mathis-a-certain-smi... A certain smile, a certain face Can lead an ...
22 1958 The Olympics Western Movies https://genius.com/the-olympics-western-movies... To save my soul I can't get a date Baby's go...
23 1958 Sheb Wooley The Purple People Eater https://genius.com/sheb-wooley-the-purple-peop... AAAAAHHHH!!!!!!!!! Well, I saw the thing co...
24 1958 Chuck Willis What Am I Living For https://genius.com/chuck-willis-what-am-i-livi... What am I living for, if not for you? What a...
25 1958 Poni-Tails Born Too Late https://genius.com/poni-tails-born-too-late-ly... NaN
26 1958 The Crickets Think It Over https://genius.com/the-crickets-think-it-over-... Think it over, what you've just said Think i...
27 1958 Jimmie Rodgers Secretly https://genius.com/jimmie-rodgers-secretly-lyrics Do do do do do do do do do do do do Do do do...
28 1958 The Four Lads Enchanted Island https://genius.com/the-four-lads-enchanted-isl... True love Is an enchanted island And we have...
29 1958 Dean Martin Angel Baby https://genius.com/dean-martin-angel-baby-lyrics (Angel baby you're mine forever) (Angel baby...
... ... ... ... ... ...
670 2018 Morgan Evans Kiss Somebody https://genius.com/morgan-evans-kiss-somebody-... [Verse 1] Sometimes you gotta drink about so...
671 2018 Becky G + Natti Natasha Sin Pijama https://genius.com/becky-g-+-natti-natasha-sin... NaN
672 2018 Lil Dicky Featuring Chris Brown Freaky Friday https://genius.com/lil-dicky-featuring-chris-b... NaN
673 2018 Lauren Daigle You Say https://genius.com/lauren-daigle-you-say-lyrics [Verse 1] I keep fighting voices in my mind ...
674 2018 lovelytheband Broken https://genius.com/lovelytheband-broken-lyrics [Chorus] I like that you're broken Broken li...
675 2018 Dylan Scott Hooked https://genius.com/dylan-scott-hooked-lyrics [Verse 1] It was a Friday night No cover but...
676 2018 Ciara Level Up https://genius.com/ciara-level-up-lyrics [Chorus] Five, four, three, two, one Leggo! ...
677 2018 TK Kravitz Featuring Jacqueez Ocean https://genius.com/tk-kravitz-featuring-jacque... NaN
678 2018 Kygo Featuring Miguel Remind Me To Forget https://genius.com/kygo-featuring-miguel-remin... NaN
679 2018 Khalid, Ty Dolla $ign & 6LACK OTW https://genius.com/khalid,-ty-dolla-$ign-&-6la... NaN
680 2018 Daniel Caesar Featuring H.E.R. Best Part https://genius.com/daniel-caesar-featuring-h.e... NaN
681 2018 G-Eazy Featuring Yo Gotti & YBN Nahmir 1942 https://genius.com/g-eazy-featuring-yo-gotti-&... NaN
682 2018 Post Malone Featuring Nicki Minaj Ball For Me https://genius.com/post-malone-featuring-nicki... NaN
683 2018 Carrie Underwood Cry Pretty https://genius.com/carrie-underwood-cry-pretty... [Verse 1] I'm sorry, but I'm just a girl Not...
684 2018 Martin Garrix Featuring Khalid Ocean https://genius.com/martin-garrix-featuring-kha... NaN
685 2018 Weezer Africa https://genius.com/weezer-africa-lyrics [Verse 1] I hear the drums echoing tonight B...
686 2018 Cole Swindell Break Up In The End https://genius.com/cole-swindell-break-up-in-t... [Verse 1] I still walk in that bar a little ...
687 2018 twenty one pilots Jumpsuit https://genius.com/twenty-one-pilots-jumpsuit-... [Intro] Cover me, oh! [Chorus] I can't beli...
688 2018 The Weeknd Call Out My Name https://genius.com/the-weeknd-call-out-my-name... [Verse 1] We found each other I helped you o...
689 2018 Dennis Lloyd Nevermind https://genius.com/dennis-lloyd-nevermind-lyrics [Intro] What if I left and it made no sense ...
690 2018 Kanye West All Mine https://genius.com/kanye-west-all-mine-lyrics [Chorus: Ant Clemons] Yeah, you supermodel t...
691 2018 Migos Narcos https://genius.com/migos-narcos-lyrics [Intro: Offset] Magnífico, arriba Magnífico,...
692 2018 Billie Eilish You Should See Me In A Crown https://genius.com/billie-eilish-you-should-se... [Verse 1] Bite my tongue, bide my time Weari...
693 2018 Eric Church Desperate Man https://genius.com/eric-church-desperate-man-l... [Intro] Boo, boo, boo-boo-boo-boo-boo Boo, b...
694 2018 Wiz Khalifa Featuring Swae Lee Hopeless Romantic https://genius.com/wiz-khalifa-featuring-swae-... NaN
695 2018 A$AP Rocky Featuring Skepta Praise The Lord (Da Shine) https://genius.com/a$ap-rocky-featuring-skepta... NaN
696 2018 NF Lie https://genius.com/nf-lie-lyrics [Chorus] I heard you told your friends that ...
697 2018 Kane Brown Lose It https://genius.com/kane-brown-lose-it-lyrics [Intro] Yeah [Verse 1] That lit up, ringing...
698 2018 Drake Emotionless https://genius.com/drake-emotionless-lyrics [Chorus: Mariah Carey] You've got me feeling...
699 2018 Drake Can't Take A Joke https://genius.com/drake-can't-take-a-joke-lyrics NaN

700 rows × 5 columns

df.to_excel("_SONGS.xlsx")
df.to_csv("_SONGS.csv")

Drop null values (if couldn't get the lyrics via previously generated URL):

df = df.dropna()
df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 371 entries, 0 to 698
Data columns (total 5 columns):
year         371 non-null int64
artist       371 non-null object
title        371 non-null object
lyric_url    371 non-null object
lyric        371 non-null object
dtypes: int64(1), object(4)
memory usage: 17.4+ KB
df.loc[(df["year"] == 1958)]
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
year artist title lyric_url lyric
0 1958 Ricky Nelson Poor Little Fool https://genius.com/ricky-nelson-poor-little-fo... I used to play around with hearts That haste...
2 1958 Bobby Darin Splish Splash https://genius.com/bobby-darin-splish-splash-l... Splish splash, I was taking a bath Long abou...
4 1958 Kalin Twins When https://genius.com/kalin-twins-when-lyrics When, when you smile, when you smile at me W...
6 1958 The Coasters Yakety Yak https://genius.com/the-coasters-yakety-yak-lyrics Take out the papers and the trash Or you don...
7 1958 Jack Scott My True Love https://genius.com/jack-scott-my-true-love-lyrics I prayed to the Lord to send me a love He se...
9 1958 Peggy Lee Fever https://genius.com/peggy-lee-fever-lyrics Never know how much I love you Never know ho...
10 1958 Frankie Avalon Ginger Bread https://genius.com/frankie-avalon-ginger-bread... You're full of sugar you're full of spice Yo...
13 1958 Pat Boone If Dreams Came True https://genius.com/pat-boone-if-dreams-came-tr... Oh my darling, oh my lover If dreams came t...
15 1958 The Danleers One Summer Night https://genius.com/the-danleers-one-summer-nig... One summer night, we fell in love One summer...
16 1958 Jody Reynolds Endless Sleep https://genius.com/jody-reynolds-endless-sleep... VERSE 1 The night was black, rain falling do...
17 1958 The Elegants Little Star https://genius.com/the-elegants-little-star-ly... Where are you little star? (Where are you?) ...
18 1958 Doris Day Everybody Loves A Lover https://genius.com/doris-day-everybody-loves-a... Everybody loves a lover I'm a lover, everybo...
19 1958 Bobby Freeman Do You Want To Dance https://genius.com/bobby-freeman-do-you-want-t... Do you want to dance and hold my hand? Tell ...
21 1958 Johnny Mathis A Certain Smile https://genius.com/johnny-mathis-a-certain-smi... A certain smile, a certain face Can lead an ...
22 1958 The Olympics Western Movies https://genius.com/the-olympics-western-movies... To save my soul I can't get a date Baby's go...
23 1958 Sheb Wooley The Purple People Eater https://genius.com/sheb-wooley-the-purple-peop... AAAAAHHHH!!!!!!!!! Well, I saw the thing co...
24 1958 Chuck Willis What Am I Living For https://genius.com/chuck-willis-what-am-i-livi... What am I living for, if not for you? What a...
26 1958 The Crickets Think It Over https://genius.com/the-crickets-think-it-over-... Think it over, what you've just said Think i...
27 1958 Jimmie Rodgers Secretly https://genius.com/jimmie-rodgers-secretly-lyrics Do do do do do do do do do do do do Do do do...
28 1958 The Four Lads Enchanted Island https://genius.com/the-four-lads-enchanted-isl... True love Is an enchanted island And we have...
29 1958 Dean Martin Angel Baby https://genius.com/dean-martin-angel-baby-lyrics (Angel baby you're mine forever) (Angel baby...
31 1958 Don Gibson Blue Blue Day https://genius.com/don-gibson-blue-blue-day-ly... It's been a blue blue day I feel like runnin...
36 1958 Perry Como Moon Talk https://genius.com/perry-como-moon-talk-lyrics Last night when the moon was bright you call...
39 1958 Bobby Hamilton Crazy Eyes For You https://genius.com/bobby-hamilton-crazy-eyes-f... I've got crazy eyes for you, baby I've got c...
40 1958 Buddy Holly Early In The Morning https://genius.com/buddy-holly-early-in-the-mo... [Verse 1] Well, you gonna miss me Early in t...
42 1958 Clyde McPhatter Come What May https://genius.com/clyde-mcphatter-come-what-m... I am yours and you are mine Come what may Lo...
44 1958 The Diamonds Kathy-O https://genius.com/the-diamonds-kathy-o-lyrics Kathy-O, Kathy-O How do your secrets grow? W...
47 1958 Dean Martin Return To Me https://genius.com/dean-martin-return-to-me-ly... Return to me Oh my dear I am so lonely Hurry...
48 1958 The Everly Brothers All I Have To Do Is Dream https://genius.com/the-everly-brothers-all-i-h... Dream, dream, dream, dream Dream, dream, dre...
55 1958 Jack Scott Leroy https://genius.com/jack-scott-leroy-lyrics I know a boy who was never blue Now he lives...
56 1958 Eydie Gorme You Need Hands https://genius.com/eydie-gorme-you-need-hands-... You need hands To hold someone you care for ...
58 1958 Tony Bennett Young And Warm And Wonderful https://genius.com/tony-bennett-young-and-warm... Young and warm and wonderful You are all I d...
59 1958 Bobby Day Over And Over https://genius.com/bobby-day-over-and-over-lyrics Ah-mmm, ah-mmm, ah-mmm, ah-mmm Well, I went ...
60 1958 Bobby Hendricks Itchy Twitchy Feeling https://genius.com/bobby-hendricks-itchy-twitc... Dih dih dih dih dih dih dih dih dip Dih dih ...
61 1958 Ed Townsend For Your Love https://genius.com/ed-townsend-for-your-love-l... For your love, oh, I would do anything I wou...
62 1958 Toni Arden Padre https://genius.com/toni-arden-padre-lyrics The day that we wed You blessed us and said ...
66 1958 The Four Preps Big Man https://genius.com/the-four-preps-big-man-lyrics I was a big man yesterday but boy you oughta...
71 1958 Connie Francis Stupid Cupid https://genius.com/connie-francis-stupid-cupid... Stupid cupid You're a real mean guy I'd like...
73 1958 Sam Cooke Win Your Love For Me https://genius.com/sam-cooke-win-your-love-for... Wooah ooo little girl How happy I would be I...
75 1958 Paul Anka Midnight https://genius.com/paul-anka-midnight-lyrics Midnight Is for lovers So let's Ispoon Bene...
77 1958 Bobby Freeman Betty Lou Got A New Pair Of Shoes https://genius.com/bobby-freeman-betty-lou-got... Betty Lou got a new pair of shoes Betty Lou ...
78 1958 David Seville The Bird On My Head https://genius.com/david-seville-the-bird-on-m... I'm just sittin' in a vacant lot with a bird...
80 1958 Chuck Berry Beautiful Delilah https://genius.com/chuck-berry-beautiful-delil... [Verse 1] Beautiful Delilah, sweet as apple ...
82 1958 Lee Andrews And The Hearts Try The Impossible https://genius.com/lee-andrews-and-the-hearts-... [Verse 1] Try the impossible Try to understa...
83 1958 Eddie Cochran Summertime Blues https://genius.com/eddie-cochran-summertime-bl... I'm a-gonna raise a fuss I'm a-gonna raise a...
85 1958 Jackie Wilson To Be Loved https://genius.com/jackie-wilson-to-be-loved-l... Someone to care Someone to share Lonely hour...
86 1958 Kitty Wells Jealousy https://genius.com/kitty-wells-jealousy-lyrics Jealousy Is there no cure your distrust Hurt...
87 1958 The Upbeats Just Like In The Movies https://genius.com/the-upbeats-just-like-in-th... Just like in the movies Let's pretend we fal...
88 1958 Jim Reeves Blue Boy https://genius.com/jim-reeves-blue-boy-lyrics Written by: Boudleaux Bryant Blue boy, that...
90 1958 Joe South The Purple People Eater Meets The Witch Doctor https://genius.com/joe-south-the-purple-people... While strolling through the woods Not so far...
91 1958 The Everly Brothers Bird Dog https://genius.com/the-everly-brothers-bird-do... Johnny is the joker (he's a bird) A very fun...
92 1958 Jimmie Rodgers Are You Really Mine https://genius.com/jimmie-rodgers-are-you-real... Are You Really Mine Jimmie Rodgers Roulette ...
94 1958 Fats Domino Little Mary https://genius.com/fats-domino-little-mary-lyrics Little Mary, my baby, don't you hear me call...
99 1958 Frankie Vaughan Judy https://genius.com/frankie-vaughan-judy-lyrics If her voice can bring Every hope of Spring ...

4. Combine all lyrics from one year to a single cell:

# 4. Combine all lyrics from one year to a single cell.

final = df.groupby('year')['lyric'].apply(' '.join).reset_index()

final.to_excel("_FINAL.xlsx")
final.to_csv("_FINAL.csv")

final
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
year lyric
0 1958 I used to play around with hearts That haste...
1 1968 [Instrumental] Can you surry, can you pi...
2 1978 [Produced by The Glimmer Twins] [Verse 1] I...
3 1988 When life is too much, roll with it, baby Do...
4 1998 [Intro: Jermaine Dupri] Yo, yo, yo Yo, yo, y...
5 2008 [Verse 1] This was never the way I planned N...
6 2018 [Intro: Drake] Trap, TrapMoneyBenny This shi...

5. Data cleansing:

# 5. Data cleansing

final['lyric'] = final['lyric'].str.replace('\[.*?\]', '', regex=True)

final.to_excel("_FINAL.xlsx")
final.to_csv("_FINAL.csv")

final
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
year lyric
0 1958 I used to play around with hearts That haste...
1 1968 Can you surry, can you picnic, whoa? Ca...
2 1978 I've been holding out so long I've been s...
3 1988 When life is too much, roll with it, baby Do...
4 1998 Yo, yo, yo Yo, yo, yo, yo Yo See, it's rare...
5 2008 This was never the way I planned Not my int...
6 2018 Trap, TrapMoneyBenny This shit got me in my...

6. Make a word cloud for each year and save it as .png:

# 6. Make a word cloud for each year and save it as .png.

for index, row in final.iterrows():

    year = final.at[index, 'year']
    text = final.at[index, 'lyric']

    wordcloud = WordCloud(collocations=False, background_color="white", max_words=100, width=1200, height=800).generate(text)

    title = str(year) + "\n"

    plt.figure(figsize=(12,8))
    plt.imshow(wordcloud, interpolation='bilinear')
    plt.axis("off")
    plt.title(title, fontsize=16)
    plt.savefig(str(year) + '.png', bbox_inches='tight')

png

png

png

png

png

png

png


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright © 2019 https://git.io/fNK2I ALL RIGHTS RESERVED