Project 1 of the Machine Learning course given at the EPFL Fall 2020.
- Marijn VAN DER MEER
- Bradley MATHEZ
- Timothée DURAN
In this project, we implemented simple machine learning models: Least Squares Regression (Normal, with GD/SGD), Ridge Regression (Normal) and (Regularized) Logistic Regression with GD (c.f. implementations.py
) . Furthermore, we optimized one of them, Logistic Regression to do binary classification on the Higgs Boson experiment challenge data-set (c.f. run.py
). For this, before optimizing the model, we did some data pre-processing and feature engineering. Those steps can be seen in the helper function pre_processing
in helper_functions/helper_functions.py
and in our report.
implementations.py
: contains implementations of Least Squares Regression (Normal, with GD/SGD), Ridge Regression (Normal) and (Regularized) Logistic Regression with GDrun.py
: main executable to recreate our best score on AICrowdmain.py
: user friendly executable to run other models than the one that gave the best performanceproject_1.ipynb
: notebook with plots for the reportREADME.md
- helper_functions/:
helper_functions.py
: helper functions needed to run the projectlosses.py
: different losses (MSE, Logisitic) and their gradientsml_methods_labs.py
: methods from the Machine Learning labs
- data/:
train.csv
: training datatest.csv
: test dataparameters.json
: JSON file with hyper-parameters for running different models inmain.py
parameters_best_model.json
: JSON fiel with parameters for best score- results/: Attention; the directories listed below could be inexistant if you have not run the
run.py
script.run.py
creates them, otherwise you can create them on your own.- predictions/: labels predicted after training models
- plots/: plots with evolution of training/validation losses
- weights/: weights with best loss found during training
Python modules requirements: numpy
, matplotlib.pyplot
, datetime
, json
, csv
and os
.
Predictions will be saved in data/results/predictions/
and weights will be saved in data/results/weights/
. To see plots of evolution of losses during training, see plots saved in data/results/plots/
There are two methods to run our program.
- To reproduce our best score with logistic regression that we submitted on AIcrowd:
python run.py
- We did a small user-friendly interface to run our code with the other models. Please execute:
python main.py