Practice_Gradient_Descent_And_Comparison
In this repository, I will be writing a gradient descent algorithm to solve a logistic regression problem and compare the result with the statistics packages. Start simple and end with regularization.
The intuition here is that I haven't done any hands-on coding in terms of numerical analysis since Junior (4 years till now). I thought this might be helpful to keep myself familiar with the back-end as well as get myself familiar with the statistical packages in python.
The steps would be:
- Choose a dataset
- Simple data manipulation
- Model 1: use a stat package in python & regularization
- Model 2: write gradient descent from scratch
- Model 3: add regularization in model 2
- Model 4: stochastic gradient descent (later)
- Model 5: mini-batch gradient descent (later)