There are many cases when as data scientists we need to find the best curve’s equation that can fit a given dataset or sample. In other words, you have some data points and you want to know what is the curve that can modelise them the best. This is known as the “Curve-fitting problem”.
In this project we will work you through an example of this problem. This project is divided into sections where in each section you will be exercising to use one of the fundamental concepts we saw in this first unit: From OOP, to SQL to Linear Algebra to the final Machine learning pipeline lesson.
You will have enough guidance throughout the project sections and this project will be graded and reviewed by a teacher assistant as well as the chance for you to reach out to the TA via slack whenever you feel you are stuck.
The key is to know that the best way to learn is the hardest! :)
We will be using a special dataset called “Longley's Economic Regression Data” and we will study the problem step by step:
-
You will need to create a new environment for your project including all necessary libraries installed in it. Also Keep in mind that it’s always a good practice to use a Jupyter notebook as a tool to experiment and not to write the whole project’s code in one python file. This will add visibility and organization to what you do.
-
The first section is about fetching your data which is provided to you in the file longley.db in q SQLlite database format, your first mission is to read this database and retrieve your data.
-
The second section is about managing the data and variable types. Then there is the linear Algebra part where you get to learn how to make a working curve fitting example.
- Fork this repository into your Github account. To do so, click on the fork button in the upper right hand corner of a repo page.
- Head to the forked version on your github then clone it on your local space.
- Open the notebook "experiment.ipynb" and you will find the necessary steps and guidance you need for this project.
- Remember: only the sections which are marked as "To-Do" are the ones you should be changing.
- Feel free to manage your versions as you want.
- Once you finished your project and you are confident about the results, You have to push those details to your forked version of the project
The following Git commands should be helpful for you as a reminder :
$git add .$git commit -m "commit msg"$git push origin main - Once you're done working on the project, submit the link to you repository in the platform.