piranha

an EDA tool in django

EDA Tool (Django App)

Modules: 1. Load CSV 2. Preprocess 3. Visualizing CSV Data 4. Train Model 5. Evaluate Model

Load CSV

* Upload CSV
* custom delimiter
* determine type of features
* determine all posiible value that feature has
* determine missing data
* uploaded file name resolution

Visualize CSV

* type of plots
	- line plot
	- histogram
	- scatter plot
	- piechart/bargraph
	- box and whiskers (candle stick plot)
	- pair plots
	- violin plots
	- joint plots
	- 
	- 
* features to visualize

Preprocessing

* fixing missing data
*

Train Model

requirements.txt

* Numpy
* Pandas
* Scikit-Learn

Django commands

1. to create a django project

$ django-admin startproject EDA

2. to create a django app

$ python3 manage.py startapp loadCSV

3. adding app to project
4. setting up routing
5. creating a super user
6. migrate and make migrate
7. setting up models
8. setting up forms
9. setting up admin
10. messages in django
11. sessions in django
12. cookies in django
13. reset password in django
14. setting up media and storage
15. reverse_lazy (reverse name lookup)
16. generic views, class views, function views

to run project

$ python3 manage.py runserver

to make migrations to database

$ python3 manage.py makemigrations
$ python3 manage.py migrate

to check installed django version

$ python3 -m django --version

structure of a django app init.py admin.py apps.py forms.py functions.py models.py urls.py views.py

#settings.py (project)

STATIC_URL = '/static/' STATICFILES_DIRS = (os.path.join(BASE_DIR, 'static'), ) STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles') MEDIA_URL = '/media/' MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

#urls.py (project) from django.conf import settings from django.conf.urls.static import static

urlpatterns = [ # Project url patterns… ]

if settings.DEBUG: urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

REFERENCES: https://ultimatedjango.com/

D3.js

CDN

<!--D3js-->
<script type = "text/javascript" src="https://d3js.org/d3.v6.min.js"></script>
<!--Bootsrap-->
<!--jquery-->
<!--poper-->
<!--others-->

Category	currency	sellerRating	Duration	endDay	ClosePrice	OpenPrice
Music/Movie/Game	US	32495	Mon	0.01	0.01	0
Music/Movie/Game	US	32495	Mon	0.01	0.01	0
Automotive	US	31157	Tue	0.01	0.01	0
Automotive	US	31157	Tue	0.01	0.01	0
Automotive	US	31157	Tue	0.01	0.01	0
Automotive	US	31157	Tue	0.01	0.01	0
Automotive	US	31157	Tue	0.01	0.01	1
Automotive	US	31157	Tue	0.01	0.01	1

How to setup django project in py3 virtual environment

$ pythom3 -m pip install django
$ python3 -m pip install virtualenv
$ mkdir ~/sample_venv
$ cd ~/sample_venv
$ virtualenv env_name
$ source env/bin/activate
$ python3 -m django --version
$ django-admin startproject EDA
$ python3 manage.py runserver

Machine Learning Life Cycle

Data Collection
Feature Engineering
- Stats
- Fix missing data
- outlier detection and fixing
- Standardization
- Normalization
- Data Augmentation
- One Hot Encoding
- Label Encoding
- Feature Engineering
- Feature Selection
EDA
- line plots
- histogram
- scatter plot
- piechart/bargraph
- box and whiskers (candle stick plot)
- pair plots
- violin plots
- joint plots
Model Selection
Model Training
Model Evaluation

REFERENCES: CSV Header read only looping pandas dataframe in django template

Django Rest Framework (setup and samples)

Installing Django and Django Rest Framework

sudo apt-get install python3 pip3
python3 -m pip installed django djangorestframework

setting up django project

django-admin startproject <project-name>

cd <project-name>
python3 manage.py makemigrations
python3 manage.py migrate
python3 manaeg.py runserver

# Django Projec Structure

Root
├── Project
|  ├── __pycache__
|  ├── __init__.py
|  ├── asgi.py
|  ├── settings.py
|  ├── urls.py
|  └── wsgi.py
├── app1
|  ├── __pycache__
|  ├── migrations
|  ├── templates
|  ├── __init__.py
|  ├── admin.py
|  ├── apps.py
|  ├── functions.py
|  ├── models.py
|  ├── tests.py
|  ├── urls.py
|  └── views.py
├── app2
|  ├── __pycache__
|  └── ...
├── db.sqlite3
└── manage.py

python3 manage.py startapp <app-name>

# root/project/settings.py
INSTALLED_APPS=[
    ....,
    'rest_framework',
    '<app-name>',
]

# root/project/urls.py
from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include("<app-name>.urls")),
]

# root/<app-name>/urls.py
from django.contrib import admin
from django.urls import path, include

from . import views

urlpatterns = [
    path('', views.index, name="basic_index")
]

# root/<app-name>/views.py
from django.shortcuts import render
from django.http import HttpResponse

def index(request):
    return HttpResponse("Fuck, world.")

def renderhtml(request):
    return render(request,"sample.html",{'data':'this is context data.'})

<!-- root/<app-name>/templates/sample.html -->
{% include "base.html" %}
<h1>Hello, World!</h1>
{{ data }}

<!-- root/<app-name>/templates/base.html -->
<!DOCTYPE html>
<html lang="en">
{% load static %}
<head>
    <meta charset="UTF-8">
    <title>base html page</title>
</head>
<body>
{%block content%}

{% endblock %}
</body>
</html>

inkfil/piranha