/DataScienceTools

Useful Data Science and Machine Learning Tools,Libraries and Packages

Primary LanguageJupyter Notebook

DataScienceTools

Aim: A Collection of Useful Data Science and Machine Learning Tools,Libraries and Packages

Data Science Tools List

Based On General Work Flow

Work Flow

Work Flow

Fetching Data/Web Scrabing

Data Cleaning

  • Pandas
  • Dedupe: Remove duplicates
  • Fuzzywuzzy: String Matching
  • Scrubadab: Anonymize privacy
  • Address Parsing/usaddress
  • Dora : Working with missing/nulls
  • PdfTables : Extracting Tables in PDF
  • Tabulate
  • Arrow : Dates,Timezone
  • Pendulum : Dates
  • Inflect/Num2Words : Convert numbers to text
  • Imbalance Learn
  • Flashtext

Data Analysis

  • Numpy
  • Scipy
  • Pandas
  • Tabel

Data Visualization

Machine Learning/Deep Learning/Model Building

Natural Language Processing

  • NLTK
  • SpaCy
  • TextBlob
  • Stanford NLP
  • AllenNLP
  • Polyglot

Computer Vision

  • OpenCV
  • Scikit Images/Scikit Video
  • Pillow

Workspace and Environment

  • Virtual Env

    • Anaconda
    • Pipenv
    • Venv
  • IDE

    • Jupyter lab/Jupyter Notebook
    • Nteract
    • VS code/Sublime Text/Juno Atom/ etc
    • Pycharm,etc
  • Cloud

    • Colab

Serializers

  • Joblib
  • Pickle
  • Ray
  • Csvkit
  • Json

Speed and Large Dataset

Forecasting

PyData stack

  • numpy
  • scipy
  • pandas
  • jupyter
  • statsmodels

Profiling

Forecasting

Niche stats libraries

Jupyter Notebook Related

Database related

  • pyodbc
  • turbodbc
  • ipython-sql
  • db.py (dead project?)
  • sqlalchemy
  • sqlalchemy-turbodbc

ETL or data engineering related sorted from lightest to heaviest framework

Data validation and cleaning frameworks

R related

  • rpy2
  • plydata (dplyr clone)
  • plotnine (ggplot2 clone)

Machine Learning Related - mostly for tabular data or non-NN

Webscraping

Utilities

Amazon Web Services

CLI

Progress Bars

Misc.

debugging

logging

Languages ,Tools, ETC

Languages

  • R
  • Python
  • Julia
  • Scala,Matlab,Octave
  • C,C++,C#
  • Perl,Ruby
  • Javascript(Tensorflow.js/Web apps)
  • Java
  • Clojure
  • Fortran

Tools

  • SQL
  • Hadoop
  • SAS
  • Java
  • Hive
  • Pig
  • SPSS
  • Perl
  • Tableau
  • E
  • NoSQL
  • AWS
  • C
  • HBase
  • Bash
  • Spark
  • ElasticSearch
  • PHP
  • Scala
  • Shark
  • awk
  • Cascading
  • Cassandra
  • JavaScript
  • JMP
  • Mahout
  • objective-C
  • QlickView
  • Redis
  • Redshift
  • sed
By
  • Jesse E.Agbe (JCharis)
  • Jesus Saves @JCharisTech
  • J-Secur1ty
Inspired By
  • Others