This is a repository for ECE143 course in UCSD.
Team members: Kuan-Wei Chen, Xiaoyin Yang, Houjian Yu and Yu Shi.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import plotly
import plotly.plotly as py
import plotly.graph_objs as go
from plotly.offline import download_plotlyjs, init_notebook_mode, plot, iplot
init_notebook_mode(connected=True)
pip install altair
import altair as alt
alt.renderers.enable('notebook')
from altair import Chart, load_dataset
pip install geopy
from geopy.geocoders import Nominatim
We use the dataset NYC_jobs from the government website.
- First, we use 'Job category' to count the entire job category and plot a pie chart to give us an overview.
- Then, we delete some irrelevant columns(parameters that do not have major relationship with software related jobs), e.g. 'Job ID','Posting Date','Posting Updated','Process Date', etc. Also we uppercase all information in the dataframe.
- Then we use software related keyword search in columns 'Job Category' and 'Business Title', e.g. 'INFORMATION|SOFTWARE|DATA|NETWORK|ANALYST|IT|DEVELOPER'.
- Now, we have 373 job positions related to software, attached with detailed information.
We plot a pie chart using all job categories to give us a impression of whole job distribution.
Also, we use Altair to plot a jobs category vs salary range plot to see each job category's salary distribution.
Columns 'Prefered Skills' and 'Minimum Qual Requirements' have a lot of job description, also contains a lot of strings.
Here we apply a function in each column to split each string and count the frequency of each keyword.
We come up with some frequent keywords in both skills and requirement and plot bar chart.
We apply the same function to count each keyword in the column 'Business Title'. Then we have an overview of software related business title distribution.
We focus on software skills and related salary distribution and compare each skill, which gives us a prospective of relation between mastering skills and salary income.
