/Data-Manipulation-and-Analysis

Data processing implementation with tools in Python

Primary LanguageJupyter Notebook

Data Manipulation and Analysis

This part works on data harvesting, processing, aggregation, and analysis in Python jupyter notebook.

Introduction

  • Data analysis is crucial to evaluating and designing solutions and applications, as well as understanding user's information needs and use. In many cases the data we need to access is distributed online among many webpages, stored in a database, or available in a large text file. Often these data (e.g. web server logs) are too large to obtain and/or process manually.
  • We need an automated way of gathering data, parsing it, and summarizing it before more advanced analysis.
  • Topics would contain techniques of exploratory data analysis, using scripting, text parsing, structured query language, regular expressions, graphing, and clustering methods to explore data.

Guideline

  1. Get Started
  2. Basic Data Manipulation
  3. Univariate Statistics
  4. pandas operations
  5. Visualization, Correlation, and Linear Models1
  6. Visualization, Correlation, and Linear Models2-case based
  7. Pivoting, contingency tables, crosstabs, mosaic plots and chi-squared
  8. Natural Language Processing Introduction
  9. Natural Language Processing for Project Gutenberg
  10. Clustering for handwriting and document
  11. Clustering for music preference and Vector Quantization
  12. Classification
  13. Dimensionality Reduction Notes
  14. Dimension_Reduction Implementation
  15. Dimension Reduction for gene expression dataset

Appendix: Some data ready to use