wondrousWebWorks/house-price-regression

Predicting House Prices Using Regression

A fictional individual, Jane Doe, has received an inheritance from a deceased great-grandfather. Included in the heritage are four houses located in Ames, Iowa. Although Jane has an excellent understanding of property prices in her own state and residential area, she fears that basing her estimates for property worth on her current knowledge might lead to inaccurate appraisals. What makes a house desirable and valuable where she comes from might not be the same in Ames, Iowa.

Jane needs help if she is to maximise the sales price for the inherited properties. She decides to ask me, a Junior Data Practitioner, for help. Her reasons for doing so are two-fold:

1. She would rather give a friend much-needed exposure and experience than approaching a stranger and
2. She also doesn't know the worth of the properties and does not want to take the risk of spending too much money without an idea of the possible return on her expenses.

1. Project Requirements

Jane made her requirements clear and expects a level of service equivalent to that of established Data Practitioners. There is potentially a decent amount of money to be made or lost when selling the four properties. The two requirements are outlined below.

1.1 Visualise data and determine which features contribute most to house prices in Ames, Iowa.

• Inspect the data related to house prices.
• Perform a correlation study to confirm that identified features are accurate and find any that were missed during data analysis.
• Plot the main variables to visualise insights.

1.2 Implement a way to use these features to predict the value of a house.

• Build different regression models to predict house prices based on identified variables.
• Perform validation on models to improve their performance.
• Compare the prediction results of all models to identify the best one.

From a Data Practitioner perspective, the features affecting house prices are labels, while the house price is the target column in the dataset.

2. Dataset Content

The dataset was taken from the House Prices - Advanced Regression Techniques competion page on Kaggle.

This dataset contains 79 explanatory variables which describe a plethory of aspects of residential homes in Ames, Iowa. Although any number of features can be chosen as the target variable, our fictious requirement is being able to predict the house price. Another requirement is identifying which features influence house prices in this particular district most. Exploratory Data Analysis is used to identify these features.

Click to expand the data set!

Variable	Meaning	Units
MSSubClass	Identifies the type of dwelling involved in the sale	20 1-STORY 1946 & NEWER ALL STYLES 30 1-STORY 1945 & OLDER 40 1-STORY W/FINISHED ATTIC ALL AGES 45 1-1/2 STORY - UNFINISHED ALL AGES 50 1-1/2 STORY FINISHED ALL AGES 60 2-STORY 1946 & NEWER 70 2-STORY 1945 & OLDER 75 2-1/2 STORY ALL AGES 80 SPLIT OR MULTI-LEVEL 85 SPLIT FOYER 90 DUPLEX - ALL STYLES AND AGES 120 1-STORY PUD (Planned Unit Development) - 1946 & NEWER 150 1-1/2 STORY PUD - ALL AGES 160 2-STORY PUD - 1946 & NEWER 180 PUD - MULTILEVEL - INCL SPLIT LEV/FOYER 190 2 FAMILY CONVERSION - ALL STYLES AND AGES
MSZoning	Identifies the general zoning classification of the sale	A Agriculture C Commercial FV Floating Village Residential I Industrial RH Residential High Density RL Residential Low Density RP Residential Low Density Park RM Residential Medium Density
LotFrontage	Linear feet of street connected to property	Feet
LotArea	Lot size in square feet	Square Feet
Street	Type of road access to property	Grvl Gravel Pave Paved
Alley	Type of alley access to property	Grvl Gravel Pave Paved NA No alley access
LotShape	General shape of property	Reg Regular IR1 Slightly irregular IR2 Moderately Irregular IR3 Irregular
LandContour	Flatness of the property	Lvl Near Flat/Level Bnk Banked - Quick and significant rise from street grade to building HLS Hillside - Significant slope from side to side Low Depression
Utilities	Type of utilities available	AllPub All public Utilities (E,G,W,& S) NoSewr Electricity, Gas, and Water (Septic Tank) NoSeWa Electricity and Gas Only ELO Electricity only
LotConfig	Lot configuration	Inside Inside lot Corner Corner lot CulDSac Cul-de-sac FR2 Frontage on 2 sides of property FR3 Frontage on 3 sides of property
LandSlope	Slope of property	Gtl Gentle slope Mod Moderate Slope Sev Severe Slope
Neighborhood	Physical locations within Ames city limits	Blmngtn Bloomington Heights Blueste Bluestem BrDale Briardale BrkSide Brookside ClearCr Clear Creek CollgCr College Creek Crawfor Crawford Edwards Edwards Gilbert Gilbert IDOTRR Iowa DOT and Rail Road MeadowV Meadow Village Mitchel Mitchell Names North Ames NoRidge Northridge NPkVill Northpark Villa NridgHt Northridge Heights NWAmes Northwest Ames OldTown Old Town SWISU South & West of Iowa State University Sawyer Sawyer SawyerW Sawyer West Somerst Somerset StoneBr Stone Brook Timber Timberland Veenker Veenker
Condition1	Proximity to various conditions	Artery Adjacent to arterial street Feedr Adjacent to feeder street Norm Normal RRNn Within 200' of North-South Railroad RRAn Adjacent to North-South Railroad PosN Near positive off-site feature--park, greenbelt, etc. PosA Adjacent to postive off-site feature RRNe Within 200' of East-West Railroad RRAe Adjacent to East-West Railroad
Condition2	Proximity to various conditions (if more than one is present)	Artery Adjacent to arterial street Feedr Adjacent to feeder street Norm Normal RRNn Within 200' of North-South Railroad RRAn Adjacent to North-South Railroad PosN Near positive off-site feature--park, greenbelt, etc. PosA Adjacent to postive off-site feature RRNe Within 200' of East-West Railroad RRAe Adjacent to East-West Railroad
BldgType	Type of dwelling	1Fam Single-family Detached 2FmCon Two-family Conversion; originally built as one-family dwelling Duplx Duplex TwnhsE Townhouse End Unit TwnhsI Townhouse Inside Unit
HouseStyle	Style of dwelling	1Story One story 1.5Fin One and one-half story: 2nd level finished 1.5Unf One and one-half story: 2nd level unfinished 2Story Two story 2.5Fin Two and one-half story: 2nd level finished 2.5Unf Two and one-half story: 2nd level unfinished SFoyer Split Foyer SLvl Split Level
OverallQual	Rates the overall material and finish of the house	10 Very Excellent 9 Excellent 8 Very Good 7 Good 6 Above Average 5 Average 4 Below Average 3 Fair 2 Poor 1 Very Poor
OverallCond	Rates the overall condition of the house	10 Very Excellent 9 Excellent 8 Very Good 7 Good 6 Above Average 5 Average 4 Below Average 3 Fair 2 Poor 1 Very Poor
YearBuilt	Original construction date	Date
YearRemodAdd	Remodel date (same as construction date if no remodeling or additions)	Date
RoofStyle	Type of roof	Flat Flat Gable Gable Gambrel Gabrel (Barn) Hip Hip Mansard Mansard Shed Shed
RoofMatl	Roof material	ClyTile Clay or Tile CompShg Standard (Composite) Shingle Membran Membrane Metal Metal Roll Roll Tar&Grv Gravel & Tar WdShake Wood Shakes WdShngl Wood Shingles
Exterior1st	Exterior covering on house	AsbShng Asbestos Shingles AsphShn Asphalt Shingles BrkComm Brick Common BrkFace Brick Face CBlock Cinder Block CemntBd Cement Board HdBoard Hard Board ImStucc Imitation Stucco MetalSd Metal Siding Other Other Plywood Plywood PreCast PreCast Stone Stone Stucco Stucco VinylSd Vinyl Siding Wd Sdng Wood Siding WdShing Wood Shingles
Exterior2nd	Exterior covering on house (if more than one material)	AsbShng Asbestos Shingles AsphShn Asphalt Shingles BrkComm Brick Common BrkFace Brick Face CBlock Cinder Block CemntBd Cement Board HdBoard Hard Board ImStucc Imitation Stucco MetalSd Metal Siding Other Other Plywood Plywood PreCast PreCast Stone Stone Stucco Stucco VinylSd Vinyl Siding Wd Sdng Wood Siding WdShing Wood Shingles
MasVnrType	Masonry veneer type	BrkCmn Brick Common BrkFace Brick Face CBlock Cinder Block None None Stone Stone
MasVnrArea	Masonry veneer area in square feet	Square feet
ExterQual	Evaluates the quality of the material on the exterior	Ex Excellent Gd Good TA Average/Typical Fa Fair Po Poor
ExterCond	Evaluates the present condition of the material on the exterior	Ex Excellent Gd Good TA Average/Typical Fa Fair Po Poor
Foundation	Type of foundation	BrkTil Brick & Tile CBlock Cinder Block PConc Poured Contrete Slab Slab Stone Stone Wood Wood
BsmtQual	Evaluates the height of the basement	Ex Excellent (100+ inches) Gd Good (90-99 inches) TA Typical (80-89 inches) Fa Fair (70-79 inches) Po Poor (<70 inches) NA No Basement
BsmtCond	Evaluates the general condition of the basement	Ex Excellent Gd Good TA Typical - slight dampness allowed Fa Fair - dampness or some cracking or settling Po Poor - Severe cracking, settling, or wetness NA No Basement
BsmtExposure	Refers to walkout or garden level walls	Gd Good Exposure Av Average Exposure (split levels or foyers typically score average or above) Mn Mimimum Exposure No No Exposure NA No Basement
BsmtFinType1	Rating of basement finished area	GLQ Good Living Quarters ALQ Average Living Quarters BLQ Below Average Living Quarters Rec Average Rec Room LwQ Low Quality Unf Unfinshed NA No Basement
BsmtFinSF2	Type 2 finished square feet	Square feet
BsmtUnfSF	Unfinished square feet of basement area	Square feet
TotalBsmtSF	Total square feet of basement area	Square feet
Heating	Type of heating	Floor Floor Furnace GasA Gas forced warm air furnace GasW Gas hot water or steam heat Grav Gravity furnace OthW Hot water or steam heat other than gas Wall Wall furnace
HeatingQC	Heating quality and condition	Ex Excellent Gd Good TA Average/Typical Fa Fair Po Poor
CentralAir	Central air conditioning	N No Y Yes
Electrical	Electrical system	SBrkr Standard Circuit Breakers & Romex FuseA Fuse Box over 60 AMP and all Romex wiring (Average) FuseF 60 AMP Fuse Box and mostly Romex wiring (Fair) FuseP 60 AMP Fuse Box and mostly knob & tube wiring (poor) Mix Mixed
1stFlrSF	First Floor square feet	Square feet
2ndFlrSF	Second floor square feet	Square feet
LowQualFinSF	Low quality finished square feet (all floors)	Square feet
GrLivArea	Above grade (ground) living area square feet	Square feet
BsmtFullBath	Basement full bathrooms	Quantity
BsmtHalfBath	Basement half bathrooms	Quantity
FullBath	Full bathrooms above grade	Quantity
HalfBath	Half baths above grade	Quantity
Bedroom	Bedrooms above grade (does NOT include basement bedrooms)	Quantity
Kitchen	Kitchens above grade	Quantity
KitchenQual	Kitchen quality	Ex Excellent Gd Good TA Typical/Average Fa Fair Po Poor
TotRmsAbvGrd	Total rooms above grade (does not include bathrooms)	Quantity
Functional	Home functionality (Assume typical unless deductions are warranted)	Typ Typical Functionality Min1 Minor Deductions 1 Min2 Minor Deductions 2 Mod Moderate Deductions Maj1 Major Deductions 1 Maj2 Major Deductions 2 Sev Severely Damaged Sal Salvage only
Fireplaces	Number of fireplaces	Quantity
FireplaceQu	Fireplace quality	Ex Excellent - Exceptional Masonry Fireplace Gd Good - Masonry Fireplace in main level TA Average - Prefabricated Fireplace in main living area or Masonry Fireplace in basement Fa Fair - Prefabricated Fireplace in basement Po Poor - Ben Franklin Stove NA No Fireplace
GarageType	Garage location	2Types More than one type of garage Attchd Attached to home Basment Basement Garage BuiltIn Built-In (Garage part of house - typically has room above garage) CarPort Car Port Detchd Detached from home NA No Garage
GarageYrBlt	Year garage was built	Date
GarageFinish	Interior finish of the garage	Fin Finished RFn Rough Finished Unf Unfinished NA No Garage
GarageCars	Size of garage in car capacity	Car capacity
GarageArea	Size of garage in square feet	Square feet
GarageQual	Garage quality	Ex Excellent Gd Good TA Typical/Average Fa Fair Po Poor NA No Garage
GarageCond	Garage condition	Ex Excellent Gd Good TA Typical/Average Fa Fair Po Poor NA No Garage
PavedDrive	Paved driveway	Y Paved P Partial Pavement N Dirt/Gravel
WoodDeckSF	Wood deck area in square feet	Square feet
OpenPorchSF	Open porch area in square feet	Square feet
EnclosedPorch	Enclosed porch area in square feet	Square feet
3SsnPorch	Three season porch area in square feet	Square feet
ScreenPorch	Screen porch area in square feet	Square feet
PoolArea	Pool area in square feet	Square feet
PoolQC	Pool quality	Ex Excellent Gd Good TA Average/Typical Fa Fair NA No Pool
Fence	Fence quality	GdPrv Good Privacy MnPrv Minimum Privacy GdWo Good Wood MnWw Minimum Wood/Wire NA No Fence
MiscFeature	Miscellaneous feature not covered in other categories	Elev Elevator Gar2 2nd Garage (if not described in garage section) Othr Other Shed Shed (over 100 SF) TenC Tennis Court NA None
MiscVal	$Value of miscellaneous feature	Dollars
MoSold	Month Sold (MM)	Month
YrSold	Year Sold (YYYY)	Year
SaleType	Type of sale	WD Warranty Deed - Conventional CWD Warranty Deed - Cash VWD Warranty Deed - VA Loan New Home just constructed and sold COD Court Officer Deed/Estate Con Contract 15% Down payment regular terms ConLw Contract Low Down payment and low interest ConLI Contract Low Interest ConLD Contract Low Down Oth Other
SaleCondition	Condition of sale	Normal Normal Sale Abnorml Abnormal Sale - trade, foreclosure, short sale AdjLand Adjoining Land Purchase Alloca Allocation - two linked properties with separate deeds, typically condo with a garage unit Family Sale between family members Partial Home was not completed when last assessed (associated with New Homes)

3. Machine Learning (ML) Business Case

We have been tasked with identifying the features which most affect the price of houses in Ames, Iowa. Once these feature variable have been identified, they need to be used to train a ML model that can predict house prices with a certain degree of certainty.

3.1 Models

Prediction calls for the use of regression models. Though there are a variety of regression models to choose from, this study will focus on the following:

3.1.1 Linear Regression Model

Linear regression is a Supervised Learning algorithm. It assumes a linear relationship between the input variables (x) and the single output variable (y). In essence, y can be calculated from a linear combination of the input variables (x). More information on the formulas used for Linear Regression can be found here.

The existing LinearRegression model from the Scikit Learn library will suit our purposes.

3.1.2 Random Forest Regression Model

Random forest is also a Supervised Learning algorithm. An ensemble learning method for regression and classification is used. In our study, we will focus in its regression capabilities. A multitude of decision trees are built at training time and the class that is the mean prediction (regression) of the individual trees are generated as output.

A random forest is a meta-estimator which combines the result of multiple predictions and aggregates many decision trees. For information, please read this Medium article.

We will use the RandomForestRegressor model from the Scikit Learn library in our study.

4. Bugs

• When loading the deployed app, the following error message is displayed occasionally: StreamlitAPIException: set_page_config() can only be called once per app, and must be called as the first Streamlit command in your script.
• Prices are shown with four decimal places. I've tried to set the way Pandas renders float values, but nothing worked.

5. Features left to implement

• Format float prices to two decimal places
• Normalise the data for more accurate predictions
• Add a Linear Regression model and compare to the Random Forest Regressor
• Add an XGBoost model and compare to the Random Forest Regressor and Linear Regressor models

6. Credits

• The Churnometer walkthrough project from Code Institute was referenced:
  • for the general project structure
  • to see how Streamlit apps are written
• Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, Aurelien Geron, 2019
  • correlation
  • RandomizedSearchCV for model optimisation