Kaggle_House_Prices_Transformer_Pytorch
A light-weight Transformer model for Kaggle House Prices Regression Competition
Kaggle House Prices -Advance Regression Techniques
A simple Pytorch deep learning model for predicting the house price. Lightweight Transformer model is tested for accuracy. The Transformer architecture is utilized to capture pair-wise affinity of all the features.
Project Objective
This is my first project in PyTorch. The aim of the project is to perform a simple multivariate regression using Transformer model.
Python Packages
Get packages by using conda or pip.
- PyTorch=1.8.0
- numpy=1.19.2
- matplotlib=3.3.4
- pandas=1.1.5
- seaborn=0.11.2
Kaggle
Once finished, you can upload your prediction.csv to the kaggle website where you can compare your score with other users.
Model |
5-Fold Validation |
Test loss (rmse) (on official test dataset) |
|
Train loss (rmse) |
Test loss (rmse) |
||
MLP (1 Block) |
0.127530 |
0.140763 |
0.15460 |
MLP (2 Blocks) |
0.108675 |
0.163794 |
0.15125 |
Ours |
0.017307 |
0.129986 |
0.12760 |
Supplementary
You need to have more than 4GB GPU memory to train the model with default settings, or you need to change batchsize or the network sturctures.