lyu-yy's Stars
jiankang1991/IGARSS2020_BWMS
Codes for IGARSS2020 paper: Band-Wise Multi-Scale CNN Architecture for Remote Sensing Image Scene Classification.
PuhongDuan/MSTV-Noise-Robust-Hyperspectral-Image-Classification-via-Multi-Scale-Total-Variation
Matlab code for Noise-Robust Hyperspectral Image Classification via Multi-Scale Total Variation, JSTARS, 2019
ran337287/MRCNN-for-Scene-Classification
Pytorch implementation of MRCNN for SceneClassification( Knowledge Guided Disambiguation for Large-Scale Scene Classification with Multi-Resolution CNNs)
mdsatria/MultiAttentionMIL
ECCV-AIMIA 2022 paper: "Multi-Scale Attention-based Multiple Instance Learning for Classification of Multi-Gigapixel Histology Images"
trinhvg/MSBP_Net
IEEE J-BHI paper: MSBP_Net: Multi-scale binary pattern encoding network for cancer classification in pathology images
djdprogramming/adfa2
# David's Personal Roadmap to Learning Data Science #### Based on the article [Learn Data Science for free in 2021](https://www.kdnuggets.com/2021/01/learn-data-science-free-2021.html) from KDnuggets. Some additions have been made. ###### I'm new to data science and programming. Some areas of study in this roadmap may be researched to a point of redundancy while materials for other topics could be seriously lacking. As I progress through this learning path, I'll be able to gauge which areas need more (or less) focus and will add and remove resources as needed. ## Schoolwork ##### Required readings for my Data Science classes. - [ ] [Doing Data Science: Straight Talk from the Frontline](https://www.amazon.com/Doing-Data-Science-Straight-Frontline/dp/1449358659) by Cathy O'Neil & Rachel Schutt - [ ] 1. Introduction: What is Data Science? - [ ] 2. Statistical Inference, Exploratory Data Analysis, and the Data Science Process - [ ] 3. Algorithms - [ ] 4. Spam Filters, Naive Bayes, and Wrangling - [ ] 5. Logistic Regression - [ ] 6. Time Stamps and Financial Modeling - [ ] 7. Extracting Meaning from Data - [ ] 8. Recommendation Engines: Building a User-Facing Data Product at Scale - [ ] 9. Data Visualization and Fraud Detection - [ ] 10. Social Networks and Data Journalism - [ ] 11. Causality - [ ] 12. Epidemiology - [ ] 13. Lessons Learned from Data Competitions: Data Leakage and Model Evaluation - [ ] 14. Data Engineering: MapReduce, Pregel, and Hadoop - [ ] 15. The Students Speak - [ ] 16. Next-Generation Data Scientists, Hubris, and Ethics - [ ] [Practical Statistics for Data Scientists: 50 Essential Concepts](https://www.amazon.com/Practical-Statistics-Data-Scientists-Essential/dp/149207294X/ref=sr_1_1?dchild=1&keywords=Practical+Statistics+for+Data+Scientists&qid=1609991269&s=books&sr=1-1) by Peter Bruce, Andrew Bruce & Peter Gedeck - [ ] 1. Exploratory Data Analysis - [ ] 2. Data and Sampling Distributions - [ ] 3. Statistical Experiments and Significance Testing - [ ] 4. Regression and Prediction - [ ] 5. Classification - [ ] 6. Statistical Machine Learning - [ ] 7. Unsupervised Learning ## Programming Skills ##### Learn programming basics. - [ ] [Python 3 Basics Tutorial Series](https://www.youtube.com/playlist?list=PLQVvvaa0QuDe8XSftW-RAxdo6OmaeL85M) by sentdex - [ ] 1. Python 3 Programming Tutorial: Why Python 3? Python 2 vs Python 3 `7:36` - [ ] 2. Python 3 Programming Tutorial: Installing Python 3 - How to Install Both Python 2 and Python 3 `15:47` - [ ] 3. Python 3 Programming Tutorial: Print Function and Strings `9:31` - [ ] 4. Python 3 Programming Tutorial: Math `4:49` - [ ] 5. Python 3 Programming Tutorial: Variables `4:26` - [ ] 6. Python 3 Programming Tutorial: While Loop `5:55` - [ ] 7. Python 3 Programming Tutorial: For Loop `9:05` - [ ] 8. Python 3 Programming Tutorial: If Statement `4:54` - [ ] 9. Python 3 Programming Tutorial: If Else `3:20` - [ ] 10. Python 3 Programming Tutorial: If Elif Else `4:19` - [ ] 11. Python 3 Programming Tutorial: Functions `3:05` - [ ] 12. Python 3 Programming Tutorial: Function Parameters `4:00` - [ ] 13. Python 3 Programming Tutorial: Function Parameter Defaults `6:06` - [ ] 14. Python 3 Programming Tutorial: Global and Local Variables `6:31` - [ ] 15. Python 3 Programming Tutorial: Installing Modules `7:44` - [ ] 16. Python 3 Programming Tutorial: How to Download and Install Python Packages and Modules with Pip `8:32` - [ ] 17. Python 3 Programming Tutorial: Common Errors `4:49` - [ ] 18. Python 3 Programming Tutorial: Writing to File `3:35` - [ ] 19. Python 3 Programming Tutorial: Appending Files `2:42` - [ ] 20. Python 3 Programming Tutorial: Read from a File `1:49` - [ ] 21. Python 3 Programming Tutorial: Classes `4:56` - [ ] 22. Python 3 Programming Tutorial: Frequently Asked Questions `5:33` - [ ] 23. Python 3 Programming Tutorial: Getting User Input `1:43` - [ ] 24. Python 3 Programming Tutorial: Statistics (Mean, Standard Deviation) `2:36` - [ ] 25. Python 3 Programming Tutorial: Module Import Syntax `5:31` - [ ] 26. Python 3 Programming Tutorial: Making Modules `4:58` - [ ] 27. Python 3 Programming Tutorial: Lists and Tuples `5:51` - [ ] 28. Python 3 Programming Tutorial: List Manipulation `9:35` - [ ] 29. Python 3 Programming Tutorial: Multi-Dimensional List `5:45` - [ ] 30. Python 3 Programming Tutorial: Reading from a CSV Spreadsheet `9:24` - [ ] 31. Python 3 Programming Tutorial: Try and Except Error Handlings `7:04` - [ ] 32. Python 3 Programming Tutorial: Multi-Line Print `3:19` - [ ] 33. Python 3 Programming Tutorial: Dictionaries `7:11` - [ ] 34. Python 3 Programming Tutorial: Built-in Functions `10:58` - [ ] 35. Python 3 Programming Tutorial: OS Module `5:01` - [ ] 36. Python 3 Programming Tutorial: Sys Module `11:00` - [ ] 37. Python 3 Programming Tutorial: urllib Module `24:04` - [ ] 38. Python 3 Programming Tutorial: Regular Expressions/Regex with re `19:58` - [ ] 39. Python 3 Programming Tutorial: Parsing Websites with re and urllib `7:29` - [ ] 40. Python 3 Programming Tutorial: Tkinter Module Making Windows `8:03` - [ ] 41. Python 3 Programming Tutorial: Tkinter Adding Buttons `6:29` - [ ] 42. Python 3 Programming Tutorial: Tkinter Event Handling `5:40` - [ ] 43. Python 3 Programming Tutorial: Tkinter Menu Bar `10:25` - [ ] 44. Python 3 Programming Tutorial: Tkinter Adding Images and Text `11:59` - [ ] 45. Python 3 Programming Tutorial: Threading Module `18:43` - [ ] 46. Python 3 Programming Tutorial: cx_freeze Python to .exe `12:08` - [ ] 47. Python 3 Programming Tutorial: Subprocess Module `13:17` - [ ] 48. Python 3 Programming Tutorial: Matplotlib Graphing Intro `10:25` - [ ] 49. Python 3 Programming Tutorial: Matplotlib Labels and Titles `5:03` - [ ] 50. Python 3 Programming Tutorial: Matplotlib Styles `10:38` - [ ] 51. Python 3 Programming Tutorial: Matplotlib Legends `4:07` - [ ] 52. Python 3 Programming Tutorial: Scatter Plots and Bar Charts `6:38` - [ ] 53. Python 3 Programming Tutorial: Matplotlib Plotting from a CSV `7:21` - [ ] 54. Python 3 Programming Tutorial: ftplib FTP Transfers Python `8:47` - [ ] 55. Python 3 Programming Tutorial: Sockets Intro `10:48` - [ ] 56. Python 3 Programming Tutorial: Sockets Simple Port Scanner `5:08` - [ ] 57. Python 3 Programming Tutorial: Threaded Port Scanner `9:36` - [ ] 58. Python 3 Programming Tutorial: Sockets Binding and Listening `5:53` - [ ] 59. Python 3 Programming Tutorial: Sockets Client Server System `10:27` - [ ] [Intermediate Python Programming](https://www.youtube.com/playlist?list=PLQVvvaa0QuDfju7ADVp5W1GF9jVhjbX-_) by sentdex - [ ] 1. Intermediate Python Programming: Introduction `7:48` - [ ] 2. Intermediate Python Programming: String Concatenation and Formatting `13:40` - [ ] 3. Intermediate Python Programming: Argparse for CLI `10:49` - [ ] 4. Intermediate Python Programming: List Comprehension and Generator Expressions `6:52` - [ ] 5. Intermediate Python Programming: More on List Comp and Generators `15:28` - [ ] 6. Intermediate Python Programming: Timeit Module `11:28` - [ ] 7. Intermediate Python Programming: Enumerate `4:48` - [ ] 8. Intermediate Python Programming: Zip `7:23` - [ ] 9. Intermediate Python Programming: Writing Our Own Generator `11:08` - [ ] 10. Intermediate Python Programming: Multiprocessing `11:30` - [ ] 11. Intermediate Python Programming: Getting Returned Values from Processes `4:22` - [ ] 12. Intermediate Python Programming: Multiprocessing Spider Example `24:18` - [ ] 13. Intermediate Python Programming: Object Oriented Programming Introductions `11:35` - [ ] 14. Intermediate Python Programming: Creating an Environment for Our Project `11:49` - [ ] 15. Intermediate Python Programming: Many Blob Objects `8:30` - [ ] 16. Intermediate Python Programming: Object Modularity Thoughts `16:41` - [ ] 17. Intermediate Python Programming: OOP Inheritance `10:17` - [ ] 18. Intermediate Python Programming: Decorators `8:50` - [ ] 19. Intermediate Python Programming: Operator Overloading `10:19` - [ ] 20. Intermediate Python Programming: Detecting Collisions `15:20` - [ ] 21. Intermediate Python Programming: Special Methods, OOP, Iteration `13:30` - [ ] 22. Intermediate Python Programming: Logging `15:00` - [ ] 23. Intermediate Python Programming: Error Handling `6:11` - [ ] 24. Intermediate Python Programming: --str-- and --repr-- `11:32` - [ ] 25. Intermediate Python Programming: Args and Kwargs `11:58` - [ ] 26. Intermediate Python Programming: Asyncio - Asynchronous Programming with Coroutines `28:37` - [ ] [2021 Complete Python Bootcamp From Zero to Hero in Python](https://www.udemy.com/course/complete-python-bootcamp/) by Jose Portilla - [ ] 1. Course Overview - [ ] 2. Python Setup - [ ] 3. Python Object and Data Structure Basics - [ ] 4. Python Comparison Operators - [ ] 5. Python Statements - [ ] 6. Methods and Functions - [ ] 7. Milestone Project 1 - [ ] 8. Object Oriented Programming - [ ] 9. Modules and Packages - [ ] 10. Errors and Exceptions Handlings - [ ] 11. Milestone Project 2 - [ ] 12. Python Decorators - [ ] 13. Python Generators - [ ] 14. Advanced Python Modules - [ ] 15. Web Scraping with Python - [ ] 16. Working with Images with Python - [ ] 17. Working with PDFs and Spreadsheet CSV Files - [ ] 18. Emails with Python - [ ] 19. Final Capstone Python Project - [ ] 20. Advanced Python Objects and Data Structures - [ ] 21. Bonus Material - Introduction to GUIs - [ ] Build the 5 projects listed in the [5 Intermediate Python Projects](https://www.youtube.com/watch?v=o5sb8ehRSYA&ab_channel=TechWithTim) video by Tech With Tim - [ ] 1. Build a Website with Django/Flask - [ ] 2. Use a WebScraper - [ ] 3. Create a Game with PyGame - [ ] 4. Build a GUI with Tkinter/PyQt5 - [ ] 5. Robotics/Raspberry Pi Project ## Data Analysis and Visualization ##### Learn NumPy, Pandas and Matplotlib. - [ ] [Python NumPy Tutorial for Beginners](https://www.youtube.com/watch?v=QUT1VHiLmmI&ab_channel=freeCodeCamp.org) by freeCodeCamp.org `58:09` - [ ] Read the [Introduction to NumPy](https://jakevdp.github.io/PythonDataScienceHandbook/02.00-introduction-to-numpy.html) chapter from the Python Data Science Handbook by Jake VanderPlas - [ ] 1. Introduction to NumPy - [ ] 2. Understanding Data Types in Python - [ ] 3. The Basics of NumPy Arrays - [ ] 4. Computation on NumPy Arrays: Universal Functions - [ ] 5. Aggregations: Min, Max, and Everything in Between - [ ] 6. Computation on Arrays: Broadcasting - [ ] 7. Comparisons, Masks, and Boolean Logic - [ ] 8. Fancy Indexing - [ ] 9. Sorting Arrays - [ ] 10. Structured Data: NumPy's Structured Arrays - [ ] [Pandas Tutorials](https://www.youtube.com/playlist?list=PL-osiE80TeTsWmV9i9c58mdDCSskIFdDS) by Corey Schafer - [ ] 1. Python Pandas Tutorial: Getting Started with Data Analysis - Installation and Loading Data `23:01` - [ ] 2. Python Pandas Tutorial: DataFrame and Series Basics - Selecting Rows and Columns `33:35` - [ ] 3. Python Pandas Tutorial: Indexes - How to Set, Reset, and Use Indexes `17:27` - [ ] 4. Python Pandas Tutorial: Filtering - Using Conditionals to Filter Rows and Columns `23:04` - [ ] 5. Python Pandas Tutorial: Updating Rows and Columns - Modifying Data within DataFrames `40:03` - [ ] 6. Python Pandas Tutorial: Add/Remove Rows and Columns from DataFrames `16:55` - [ ] 7. Python Pandas Tutorial: Sorting Data `15:40` - [ ] 8. Python Pandas Tutorial: Grouping and Aggregating - Analyzing and Exploring Your Data `49:06` - [ ] 9. Python Pandas Tutorial: Cleaning Data - Casting Data Types and Handling Missing Values `31:54` - [ ] 10. Python Pandas Tutorial: Working with Dates and Time Series Data `35:41` - [ ] 11. Python Pandas Tutorial: Reading/Writing Data to Different Sources - Excel, JSON, SQL, Etc. `32:45` - [ ] Read the [Data Manipulation with Pandas](https://jakevdp.github.io/PythonDataScienceHandbook/03.00-introduction-to-pandas.html) chapter from the Python Data Science Handbook by Jake VanderPlas - [ ] 1. Data Manipulation with Pandas - [ ] 2. Introducing Pandas Objects - [ ] 3. Data Indexing and Selection - [ ] 4. Operating on Data in Pandas - [ ] 5. Handling Missing Data - [ ] 6. Hierarchical Indexing - [ ] 7. Combining Datasets: Concat and Append - [ ] 8. Combining Datasets: Merge and Join - [ ] 9. Aggregation and Grouping - [ ] 10. Pivot Tables - [ ] 11. Vectorized String Operations - [ ] 12. Working with Time Series - [ ] 13. High-Performance Pandas: eval() and query() - [ ] [Matplotlib Tutorials](https://www.youtube.com/playlist?list=PL-osiE80TeTvipOqomVEeZ1HRrcEvtZB_) by Corey Schafer - [ ] 1. Matplotlib Tutorial: Creating and Customizing Our First Plots `35:01` - [ ] 2. Matplotlib Tutorial: Bar Charts and Analyzing Data from CSVs `34:26` - [ ] 3. Matplotlib Tutorial: Pie Charts `17:02` - [ ] 4. Matplotlib Tutorial: Stack Plots `14:49` - [ ] 5. Matplotlib Tutorial: Filling Area on Line Plots `15:18` - [ ] 6. Matplotlib Tutorial: Histograms `16:36` - [ ] 7. Matplotlib Tutorial: Scatter Plots `21:24` - [ ] 8. Matplotlib Tutorial: Plotting Time Series Data `17:09` - [ ] 9. Matplotlib Tutorial: Plotting Live Data in Real-Time `20:34` - [ ] 10. Matplotlib Tutorial: Subplots `21:22` - [ ] Read the [Visualization with Matplotlib](https://jakevdp.github.io/PythonDataScienceHandbook/04.00-introduction-to-matplotlib.html) chapter from the Python Data Science Handbook by Jake VanderPlas - [ ] 1. Visualization with Matplotlib - [ ] 2. Simple Line Plots - [ ] 3. Simple Scatter Plots - [ ] 4. Visualizing Errors - [ ] 5. Density and Contour Plots - [ ] 6. Histograms, Binnings, and Density - [ ] 7. Customizing Plot Legends - [ ] 8. Customizing Colorbars - [ ] 9. Multiple Subplots - [ ] 10. Text and Annotation - [ ] 11. Customizing Ticks - [ ] 12. Customizing Matplotlib: Configurations and Stylesheets - [ ] 13. Three-Dimensional Plotting in Matplotlib - [ ] 14. Geographic Data with Basemap - [ ] 15. Visualization with Seaborn - [ ] [Python for Data Science - Course for Beginners (Learn Python, Pandas, NumPy, Matplotlib)](https://www.youtube.com/watch?v=LHBE6Q9XlzI&t=2s&ab_channel=freeCodeCamp.org) by freeCodeCamp.org `12:19:51` ## Data Preprocessing ##### Learn the basics of data preprocessing. - [ ] [Data Cleaning](https://www.kaggle.com/learn/data-cleaning) by Kaggle - [ ] 1. Handling Missing Values - [ ] 2. Scaling and Normalization - [ ] 3. Parsing Dates - [ ] 4. Character Encodings - [ ] 5. Inconsistent Data Entry - [ ] Do the [Titanic - Machine Learning from Disaster](https://www.kaggle.com/c/titanic) competition by Kaggle - [ ] Do the [Housing Prices](https://www.kaggle.com/c/home-data-for-ml-course) competition by Kaggle - [ ] [Feature Engineering](https://www.kaggle.com/learn/feature-engineering) by Kaggle - [ ] 1. Baseline Model - [ ] 2. Categorical Encodings - [ ] 3. Feature Generation - [ ] 4. Feature Selection ## Databases ##### Learn about databases. - [ ] [Intro to SQL](https://www.kaggle.com/learn/intro-to-sql) by Kaggle - [ ] 1. Getting Started with SQL and BigQuery - [ ] 2. Select, From & Where - [ ] 3. Group By, Having & Count - [ ] 4. Order By - [ ] 5. As & With - [ ] 6. Joining Data - [ ] [Advanced SQL](https://www.kaggle.com/learn/advanced-sql) by Kaggle - [ ] 1. JOINs and UNIONs - [ ] 2. Analytic Functions - [ ] 3. Nested and Repeated Data - [ ] 4. Writing Efficient Queries - [ ] [MongoDB with Python Crash Course - Tutorial for Beginners](https://www.youtube.com/watch?v=E-1xI85Zog8&ab_channel=freeCodeCamp.org) by freeCodeCamp.org `1:57:33` ## Machine Learning ##### Taking our first steps into the world of ML. - [ ] [Machine Learning](https://www.coursera.org/learn/machine-learning#syllabus) by Andrew Ng (skipping the MATLAB section) - [ ] 1. Introduction - [ ] 2. Linear Regression with One Variable - [ ] 3. Linear Algebra Review - [ ] 4. Linear Regression with Multiple Variables - [ ] 5. Logistic Regression - [ ] 6. Regularization - [ ] 7. Neural Networks: Representation - [ ] 8. Neural Networks: Learning - [ ] 9: Advice for Applying Machine Learning - [ ] 10. Machine Learning System Design - [ ] 11. Support Vector Machines - [ ] 12. Unsupervised Learning - [ ] 13. Dimensionality Reduction - [ ] 14. Anomaly Detection - [ ] 15. Recommender Systems - [ ] 16. Large Scale Machine Learning - [ ] 17. Application Example: Photo OCR - [ ] [Coursera Machine Learning MOOC by Andrew Ng Python Programming Assignments](https://github.com/dibgerge/ml-coursera-python-assignments) - [ ] Exercise 1 - [ ] Exercise 2 - [ ] Exercise 3 - [ ] Exercise 4 - [ ] Exercise 5 - [ ] Exercise 6 - [ ] Exercise 7 - [ ] Exercise 8 - [ ] Do any [Kaggle](https://www.kaggle.com/) competition - [ ] [Intermediate Machine Learning](https://www.kaggle.com/learn/intermediate-machine-learning) by Kaggle - [ ] 1. Introduction - [ ] 2. Missing Values - [ ] 3. Categorical Variables - [ ] 4. Pipelines - [ ] 5. Cross-Validation - [ ] 6. XGBoost - [ ] 7. Data Leakage ## Linear Algebra and Statistics ##### Learn linear algebra and statistics. - [ ] [Linear Algebra](https://www.khanacademy.org/math/linear-algebra) on Khan Academy - [ ] 1. Vectors and Spaces - [ ] 2. Matrix Transformations - [ ] 3. Alternate Coordinate Systems (Bases) - [ ] [Linear Algebra](https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/index.htm) on MIT OpenCourseWare - [ ] Problem Set 1 - [ ] Problem Set 2 - [ ] Problem Set 3 - [ ] Problem Set 4 - [ ] Problem Set 5 - [ ] Problem Set 6 - [ ] Problem Set 7 - [ ] Problem Set 8 - [ ] Problem Set 9 - [ ] Problem Set 10 - [ ] Exam 1 - [ ] Exam 2 - [ ] Exam 3 - [ ] Final Exam - [ ] [Statistics and Probability](https://www.khanacademy.org/math/statistics-probability) on Khan Academy - [ ] 1. Analyzing Categorical Data - [ ] 2. Displaying and Comparing Quantitative Data - [ ] 3. Summarizing Quantitative Data - [ ] 4. Modeling Data Distributions - [ ] 5. Exploring Bivariate Numerical Data - [ ] 6. Study Design - [ ] 7. Probability - [ ] 8. Counting, Permutations, and Combinations - [ ] 9. Random Variables - [ ] 10. Sampling Distributions - [ ] 11. Confidence Intervals - [ ] 12. Significance Tests (Hypothesis Testing) - [ ] 13. Two-Sample Inference for the Difference Between Groups - [ ] 14. Inference for Categorical Data (Chi-Square Tests) - [ ] 15. Advanced Regression (Inference and Transforming) - [ ] 16. Analysis of Variance (ANOVA) - [ ] [Introduction to Probability and Statistics](https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/index.htm) on MIT OpenCourseWare - [ ] Problem Set 1 - [ ] Problem Set 2 - [ ] Problem Set 3 - [ ] Problem Set 4 - [ ] Problem Set 5 - [ ] Problem Set 6 - [ ] Problem Set 7 - [ ] Problem Set 8 - [ ] Problem Set 9 - [ ] Exam 1 Practice Questions I - [ ] Exam 1 Practice Questions II - [ ] Exam 1 Practice Questions: Long List - [ ] Exam 1 - [ ] Exam 2 Practice Questions - [ ] Exam 2 - [ ] Final Exam Practice Questions - [ ] Final Exam - [ ] [Deep Learning Book](https://www.deeplearningbook.org/) by Ian Goodfellow, Yoshua Bengio & Aaron Courville - [ ] 1. Introduction - [ ] 2. Linear Algebra - [ ] 3. Probability and Information Theory - [ ] 4. Numerical Computation - [ ] 5. Machine Learning Basics - [ ] 6. Deep Feedforward Networks - [ ] 7. Regularization for Deep Learning - [ ] 8. Optimization for Training Deep Models - [ ] 9. Convolutional Networks - [ ] 10. Sequence Modeling: Recurrent and Recursive Nets - [ ] 11. Practical Methodology - [ ] 12. Applications - [ ] 13. Linear Factor Models - [ ] 14. Autoencoders - [ ] 15. Representation Learning - [ ] 16. Structured Probabilistic Models for Deep Learning - [ ] 17. Monte Carlo Methods - [ ] 18. Confronting the Partition Function - [ ] 19. Approximate Inference - [ ] 20. Deep Generative Models ## Deep Learning ##### Learning about deep learning. - [ ] [Practical Deep Learning for Coders](https://course.fast.ai/) by fast.ai - [ ] Lesson 1 - [ ] Lesson 2 - [ ] Lesson 3 - [ ] Lesson 4 - [ ] Lesson 5 - [ ] Lesson 6 - [ ] Lesson 7 - [ ] Lesson 8 - [ ] [Part 2: Deep Learning from the Foundations](https://course19.fast.ai/part2) by fast.ai - [ ] Lesson 1 - [ ] Lesson 2 - [ ] Lesson 3 - [ ] Lesson 4 - [ ] Lesson 5 - [ ] Lesson 6 - [ ] Lesson 7 - [ ] Lesson 8 - [ ] Lesson 9 - [ ] Lesson 10 - [ ] Lesson 11 - [ ] Lesson 12 - [ ] Lesson 13 - [ ] Lesson 14 - [ ] [Deep Learning Specialization](https://www.coursera.org/specializations/deep-learning) by Andrew Ng - [ ] Course 1: Neural Networks and Deep Learning - [ ] 1. Introduction to Deep Learning - [ ] 2. Neural Network Basics - [ ] 3. Shallow Neural Networks - [ ] 4. Deep Neural Networks - [ ] Course 2: Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - [ ] 1. Practical Aspects of Deep Learning - [ ] 2. Optimization Algorithms - [ ] 3. Hyperparameter tuning, Batch Normalization and Programming Frameworks - [ ] Course 3: Structuring Machine Learning Projects - [ ] 1. ML Strategy (1) - [ ] 2. ML Strategy (2) - [ ] Course 4: Convolutional Neural Networks - [ ] 1. Foundations of Convolutional Neural Networks - [ ] 2. Deep Convolutional Models: Case Studies - [ ] 3. Object Detection - [ ] 4. Special applications: Face recognition & Neural style transfer - [ ] Course 5: Sequence Models - [ ] 1. Recurrent Neural Networks - [ ] 2. Natural Language Processing & Word Embeddings - [ ] 3. Sequence Models & Attention Mechanism - [ ] [DeepLearning.AI TensorFlow Developer Professional Certificate](https://www.coursera.org/professional-certificates/tensorflow-in-practice?) by Laurence Moroney - [ ] Course 1: Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning - [ ] 1. A New Programming Paradigm - [ ] 2. Introduction to Computer Vision - [ ] 3. Enhancing Vision with Convolutional Neural Networks - [ ] 4. Using Real-World Images - [ ] Course 2: Convolutional Neural Networks in TensorFlow - [ ] 1. Exploring a Larger Dataset - [ ] 2. Augmentation: A Technique to Avoid Overfitting - [ ] 3. Transfer Learning - [ ] 4. Multiclass Classifications - [ ] Course 3: Natural Language Processing in TensorFlow - [ ] 1. Sentiment in Text - [ ] 2. Word Embeddings - [ ] 3. Sequence Models - [ ] 4. Sequence Models and Literature - [ ] Course 4: Sequences, Time Series and Prediction - [ ] 1. Sequences and Prediction - [ ] 2. Deep Neural Networks for Time Series - [ ] 3. Recurrent Neural Networks for Time Series - [ ] 4. Real-World Time Series Data ## Cloud for Model Deployment ##### Learn how to build, train, test, and deploy a machine learning model on AWS. - [ ] [AWS Machine Learning Specialty](https://www.youtube.com/playlist?list=PLEF5xKHm-3ZNDvdJpMCLu9xa1oDNvAmMr) by Amazon - [ ] 1. AWS Training and Certification: Machine Learning `1:31` - [ ] 2. Build, Train and Deploy Machine Learning Models on AWS with Amazon SageMaker - AWS Online `35:51` - [ ] 3. AWS re:Invent 2018: Leadership Session: Machine Learning (AIM202-L) `58:01` - [ ] 4. Machine Learning Models with TensorFlow Using Amazon SageMaker - AWS Online Tech Talks `40:16` - [ ] 5. AWS re:Invent 2018: Build & Deploy ML Models Quickly & Easily with Amazon SageMaker `57:53` - [ ] 6. AWS re:Invent 2018: CI/CD for Your Machine Learning Pipeline with Amazon SageMaker `57:13` - [ ] 7. AWS Berlin Summit 2018 - Building and Running Your First ML Application on Amazon SageMaker `52:54` - [ ] 8. Predictive Analytics with Amazon SageMaker `1:03:29` - [ ] 9. AWS re:Invent 2018: AI/ML with Data Lakes: Counterintuitive Consumer Insights in Retail `1:00:10` - [ ] 10. AWS re:Invent 2018: Industrialize Machine Learning Using CI/CD Techniques (FSV304-i) `45:34` - [ ] 11. AWS re:Invent 2018: Driving Machine Learning and Analytics Use Cases with AWS Storage (STG302) `40:16` - [ ] 12. AWS re:Invent 2018: Deep Learning Applications Using TensorFlow (AIM401-R) `1:02:29` - [ ] 13. AWS re:Invent 2017: Machine Learning State of the Union (MCL210) `1:00:55` - [ ] 14. AWS re:Invent 2017: Containerized Machine Learning on AWS (CON309) `1:03:21` - [ ] 15. AWS re:Invent 2017: Introduction to Deep Learning (MCL205) `46:17` - [ ] 16. Continuous Delivery with AWS CodePipeline and Amazon SageMaker `25:24` - [ ] 17. AWS re:Invent 2017: Best Practices for Distributed Machine Learning and Predictive A (ABD403) `1:16:16` - [ ] 18. AWS re:Invent 2017: GPS: Enhancing Customer Security Using AI/ML on AWS (GPSTEC311) `50:21` - [ ] 19. How to Wrangle Data for Machine Learning on AWS `59:24` - [ ] 20. Extract Data from Images and Videos with Amazon Rekognition (Level 300) `26:52` - [ ] 21. Exploring the Business Use Cases for Amazon Machine Learning - 2017 AWS Online Tech Talks `30:35` - [ ] 22. AWS re:Invent 2017: Orchestrating Machine Learning Training for Netflix Recommendation (MCL317) `54:21` - [ ] 23. AWS re:Invent 2017: Reinforcement Learning - The Ultimate AI (ARC320) `1:00:00` - [ ] 24. Amazon Machine Learning: Empowering Developers to Build Smart Applications `55:09` - [ ] 25. Amazon SageMaker's Built-in Algorithm Webinar Series: DeepAR Forecasting `53:41` - [ ] 26. Amazon SageMaker's Built-in Algorithm Webinar Series: Linear Learner `58:55` - [ ] 27. Amazon SageMaker's Built-in Algorithm Webinar Series: Clustering with K Means `58:52` - [ ] 28. Amazon SageMaker's Built-in Algorithm Webinar Series: Latent Dirichlet Allocation (LDA) `57:25` - [ ] 29. Amazon SageMaker's Built-in Algorithm Webinar Series: XGBoost `1:01:02` - [ ] 30. Amazon SageMaker's Built-in Algorithm Webinar Series: ResNet `55:56` - [ ] 31. Amazon SageMaker-s Built-in Algorithm Webinar Series: Blazing Text `1:14:37` - [ ] 32. AWS re:Invent 2017: NEW LAUNCH! Introducing Amazon SageMaker (MCL365) `1:02:08` - [ ] 33. Fully Managed Notebook Instances with Amazon SageMaker - a Deep Dive `16:45` - [ ] 34. Built-in Machine Learning Algorithms with Amazon SageMaker - a Deep Dive `15:38` - [ ] [Machine Learning with TensorFlow on Google Cloud Platform Specialization](https://www.coursera.org/specializations/machine-learning-tensorflow-gcp) by Google Cloud Training - [ ] Course 1: How Google does Machine Learning - [ ] 1. Introduction to Course - [ ] 2. What It Means to Be AI First - [ ] 3. How Google Does ML - [ ] 4. Inclusive ML - [ ] 5. Python Notebooks in the Cloud - [ ] 6. Summary - [ ] Course 2: Launching into Machine Learning - [ ] 1. Introduction to Course - [ ] 2. Improve Data Quality and Exploratory Data Analysis - [ ] 3. Practical ML - [ ] 4. Optimization - [ ] 5. Generalization and Sampling - [ ] 6. Summary - [ ] Course 3: Introduction to TensorFlow - [ ] 1. Introduction to Course - [ ] 2. Introduction to TensorFlow - [ ] 3. Design and Build a TensorFlow Input Data Pipeline - [ ] 4. Training Neural Networks with TensorFlow 2 and the Keras Sequential API - [ ] 5. Training Neural Networks with TensorFlow 2 and the Keras Functional API - [ ] 6. Summary - [ ] Course 4: Feature Engineering - [ ] 1. Introduction to Course - [ ] 2. Raw Data to Features - [ ] 3. Preprocessing and Feature Creation - [ ] 4. Feature Crosses - [ ] 5. TensorFlow Transform - [ ] 6. Summary - [ ] Course 5: Art and Science of Machine Learning - [ ] 1. Introduction - [ ] 2. The Art of ML - [ ] 3. Hyperparameter Tuning - [ ] 4. A Pinch of Science - [ ] 5. The Science of Neural Networks - [ ] 6. Embeddings - [ ] 7. Summary
zachysun/MACNN-Pytorch
Unofficial implementation of paper “Multi-scale Attention Convolutional Neural Network for time series classification”[Neural Networks2021]
DSIP-UPatras/sEMG-hilbert-curve
Accompaniment code for 'Hilbert sEMG data scanning for hand gesture recognition based on Deep Learning' published in NCAA.
minhokim93/LCZ_MSMLA
Local climate zone classification using a multi-scale, multi-level attention network (ISPRS J)
jainsee24/Parallel-Face-detection
Image segmentation is the process of dividing an image into multiple parts. It is typically used to identify objects or other relevant information in digital images. There are many ways to perform image segmentation including Thresholding methods, Color-based segmentation, Transform methods among many others. Alternately edge detection can be used for image segmentation and data extraction in areas such as image processing, computer vision, and machine vision. Image thresholding is a simple, yet effective, way of partitioning an image into a foreground and background. This image analysis technique is a type of image segmentation that isolates objects by converting grayscale images into binary images. Image thresholding is most effective in images with high levels of contrast. Otsu's method, named after Nobuyuki Otsu, is one such implementation of Image Thresholding which involves iterating through all the possible threshold values and calculating a measure of spread for the pixel levels each side of the threshold, i.e. the pixels that either fall in foreground or background. The aim is to find the threshold value where the sum of foreground and background spreads is at its minimum. Edge detection is an image processing technique for finding the boundaries of objects within images. It works by detecting discontinuities in brightness. An image can have horizontal, vertical or diagonal edges. The Sobel operator is used to detect two kinds of edges in an image by making use of a derivative mask, one for the horizontal edges and one for the vertical edges. 1. Introduction Face detection is a computer technology being used in a variety of applications that identifies human faces in digital images. Face detection also refers to the psychological process by which humans locate and attend to faces in a visual scene. Face detection can be regarded as a specific case of object-class detection. In object-class detection, the task is to find the locations and sizes of all objects in an image that belong to a given class. Examples include upper torsos, pedestrians, and cars. Face-detection algorithms focus on the detection of frontal human faces. It is analogous to image detection in which the image of a person is matched bit by bit. Image matches with the image stores in database. Any facial feature changes in the database will invalidate the matching process. 2. Needs/Problems There have been widely applied many researches related to face recognition system. The system is commonly used for video surveillance, human and computer interaction, robot navigation, and etc. Along with the utilization of the system, it leads to the need for a faster system response, such as robot navigation or application for public safety. A number of classification algorithms have been applied to face recognition system, but it still has a problem in terms of computing time. In this system, computing time of the classification or feature extraction is an important thing for further concern. To improve the algorithmic efficiency of face detection, we combine the eigenface method using Haar-like features to detect both of eyes and face, and Robert cross edge detector to locate the human face position. Robert Cross uses the integral image representation and simple rectangular features to eliminate the need of expensive calculation of multi-scale image pyramid. 3. Objectives Some techniques used in this application are 1. Eigen-face technique 2. KLT Algorithm 3. Parallel for loop in openmp 4. OpenCV for face detection. 5. Further uses of the techniques
EdwardHaoz/IEEE_TGRS_AMGCFN
H. Zhou, F. Luo, H. Zhuang, Z. Weng, X. Gong and Z. Lin, "Attention Multi-hop Graph and Multi-scale Convolutional Fusion Network for Hyperspectral Image Classification," in IEEE Transactions on Geoscience and Remote Sensing, doi: 10.1109/TGRS.2023.3265879.
ysxGitHub/MVMS-net
The code for paper "A Multi-View Multi-Scale Neural Network for Multi-Label ECG Classification"
wesley1001/MACNN
Multi-scale Attention Convolutional Neural Network for Time Series Classification
Ashleshk/Machine-Learning-Stanford-Andrew-Ng
# Machine Learning (Coursera) This is my solution to all the programming assignments and quizzes of Machine-Learning (Coursera) taught by Andrew Ng. After completing this course you will get a broad idea of Machine learning algorithms. Try to solve all the assignments by yourself first, but if you get stuck somewhere then feel free to browse the code. ## Contents * Lectures Slides * Solution to programming assignment * Solution to Quizzes by Andrew Ng, Stanford University, [Coursera](https://www.coursera.org/learn/machine-learning/home/welcome) ### Week 1 - [X] Videos: Introduction - [X] Quiz: Introduction - [X] Videos: Linear Regression with One Variable - [X] Quiz: Linear Regression with One Variable ### Week 2 - [X] Videos: Linear Regression with Multiple Variables - [X] Quiz: Linear Regression with Multiple Variables - [X] Videos: Octave/Matlab Tutorial - [X] Quiz: Octave/Matlab Tutorial - [X] Programming Assignment: Linear Regression ### Week 3 - [X] Videos: Logistic Regression - [X] Quiz: Logistic Regression - [X] Videos: Regularization - [X] Quiz: Regularization - [X] Programming Assignment: Logistic Regression ### Week 4 - [X] Videos: Neural Networks: Representation - [X] Quiz: Neural Networks: Representation - [X] Programming Assignment: Multi-class Classification and Neural Networks ### Week 5 - [X] Videos: Neural Networks: Learning - [X] Quiz: Neural Networks: Learning - [X] Programming Assignment: Neural Network Learning ### Week 6 - [X] Videos: Advice for Applying Machine Learning - [X] Quiz: Advice for Applying Machine Learning - [X] Videos: Programming Assignment: Regularized Linear Regression and Bias/Variance - [X] Machine Learning System Design - [X] Quiz: Machine Learning System Design ### Week 7 - [X] Videos: Support Vector Machines - [X] Quiz: Support Vector Machines - [X] Programming Assignment: Support Vector Machines ### Week 8 - [X] Videos: Unsupervised Learning - [X] Quiz: Unsupervised Learning - [X] Videos: Dimensionality Reduction - [X] Quiz: Principal Component Analysis - [X] Programming Assignment: K-Means Clustering and PCA ### Week 9 - [X] Videos: Anomaly Detection - [X] Quiz: Anomaly Detection - [X] Videos: Recommender Systems - [X] Quiz: Recommender Systems - [X] Programming Assignment: Anomaly Detection and Recommender Systems ### Week 10 - [X] Videos: Large Scale Machine Learning - [X] Quiz: Large Scale Machine Learning ### Week 11 - [X] Videos: Application Example: Photo OCR - [X] Quiz: Application: Photo OCR ## Certificate * [Verified Certificate]() ## References [[1] Machine Learning - Stanford University](https://www.coursera.org/learn/machine-learning)
iliaschalkidis/lmtc-emnlp2020
An Empirical Study on Large-Scale Multi-Label Text Classification including Few and Zero-Shot Labels
monaen/Large-Scale-Multi-Class-Image-Based-Cell-Classification-with-Deep-Learning
Large-Scale Multi-Class Image-Based Cell Classification with Deep Learning
ziyujia/ECML-PKDD_MMCNN
MMCNN: A Multi-branch Multi-scale Convolutional Neural Network for Motor Imagery Classification
Exp-Time-Series-Tools/Ms-CNN
Multi-Scale Convolutional Neural Network for Time Series Classification (MCNN)
quqixun/BTClassification
Brain tumor classification on structural MR images of BraTS dataset based on 3D Multi-Scale Convolutional Neural Network, which is a part of my master thesis project.
AnthonyMRios/leml
Method for large-scale multi-label classification
leeguandong/Multi-Scale-Dense-Networks-for-Hyperspectral-Remote-Sensing-Image-Classification
paper：Multi-Scale Dense Networks for Hyperspectral Remote Sensing Image Classification
anindox8/Ensemble-of-Multi-Scale-CNN-for-Dermatoscopy-Classification
Fully supervised binary classification of skin lesions from dermatoscopic images using an ensemble of diverse CNN architectures (EfficientNet-B6, Inception-V3, SEResNeXt-101, SENet-154, DenseNet-169) with multi-scale input.
haarburger/multi-scale-curriculum
Code for Paper: Multi Scale Curriculum CNN for Context-Aware Breast MRI Malignancy Classification
iliaschalkidis/lmtc-eurlex57k
Large-Scale Multi-Label Text Classification on EU Legislation
samihaija/mixhop
Official Implementation of ICML 2019 Paper. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing; and UAI 2019 Paper: N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification
huoxiangzuo/HiFuse
HiFuse: Hierarchical Multi-Scale Feature Fusion Network for Medical Image Classification
wangshy31/Densely-Connected-CNN-with-Multiscale-Feature-Attention
Densely Connected CNN with Multi-scale Feature Attention for Text Classification
sayantann11/all-classification-templetes-for-ML
Classification - Machine Learning This is ‘Classification’ tutorial which is a part of the Machine Learning course offered by Simplilearn. We will learn Classification algorithms, types of classification algorithms, support vector machines(SVM), Naive Bayes, Decision Tree and Random Forest Classifier in this tutorial. Objectives Let us look at some of the objectives covered under this section of Machine Learning tutorial. Define Classification and list its algorithms Describe Logistic Regression and Sigmoid Probability Explain K-Nearest Neighbors and KNN classification Understand Support Vector Machines, Polynomial Kernel, and Kernel Trick Analyze Kernel Support Vector Machines with an example Implement the Naïve Bayes Classifier Demonstrate Decision Tree Classifier Describe Random Forest Classifier Classification: Meaning Classification is a type of supervised learning. It specifies the class to which data elements belong to and is best used when the output has finite and discrete values. It predicts a class for an input variable as well. There are 2 types of Classification: Binomial Multi-Class Classification: Use Cases Some of the key areas where classification cases are being used: To find whether an email received is a spam or ham To identify customer segments To find if a bank loan is granted To identify if a kid will pass or fail in an examination Classification: Example Social media sentiment analysis has two potential outcomes, positive or negative, as displayed by the chart given below. https://www.simplilearn.com/ice9/free_resources_article_thumb/classification-example-machine-learning.JPG This chart shows the classification of the Iris flower dataset into its three sub-species indicated by codes 0, 1, and 2. https://www.simplilearn.com/ice9/free_resources_article_thumb/iris-flower-dataset-graph.JPG The test set dots represent the assignment of new test data points to one class or the other based on the trained classifier model. Types of Classification Algorithms Let’s have a quick look into the types of Classification Algorithm below. Linear Models Logistic Regression Support Vector Machines Nonlinear models K-nearest Neighbors (KNN) Kernel Support Vector Machines (SVM) Naïve Bayes Decision Tree Classification Random Forest Classification Logistic Regression: Meaning Let us understand the Logistic Regression model below. This refers to a regression model that is used for classification. This method is widely used for binary classification problems. It can also be extended to multi-class classification problems. Here, the dependent variable is categorical: y ϵ {0, 1} A binary dependent variable can have only two values, like 0 or 1, win or lose, pass or fail, healthy or sick, etc In this case, you model the probability distribution of output y as 1 or 0. This is called the sigmoid probability (σ). If σ(θ Tx) > 0.5, set y = 1, else set y = 0 Unlike Linear Regression (and its Normal Equation solution), there is no closed form solution for finding optimal weights of Logistic Regression. Instead, you must solve this with maximum likelihood estimation (a probability model to detect the maximum likelihood of something happening). It can be used to calculate the probability of a given outcome in a binary model, like the probability of being classified as sick or passing an exam. https://www.simplilearn.com/ice9/free_resources_article_thumb/logistic-regression-example-graph.JPG Sigmoid Probability The probability in the logistic regression is often represented by the Sigmoid function (also called the logistic function or the S-curve): https://www.simplilearn.com/ice9/free_resources_article_thumb/sigmoid-function-machine-learning.JPG In this equation, t represents data values * the number of hours studied and S(t) represents the probability of passing the exam. Assume sigmoid function: https://www.simplilearn.com/ice9/free_resources_article_thumb/sigmoid-probability-machine-learning.JPG g(z) tends toward 1 as z -> infinity , and g(z) tends toward 0 as z -> infinity K-nearest Neighbors (KNN) K-nearest Neighbors algorithm is used to assign a data point to clusters based on similarity measurement. It uses a supervised method for classification. The steps to writing a k-means algorithm are as given below: https://www.simplilearn.com/ice9/free_resources_article_thumb/knn-distribution-graph-machine-learning.JPG Choose the number of k and a distance metric. (k = 5 is common) Find k-nearest neighbors of the sample that you want to classify Assign the class label by majority vote. KNN Classification A new input point is classified in the category such that it has the most number of neighbors from that category. For example: https://www.simplilearn.com/ice9/free_resources_article_thumb/knn-classification-machine-learning.JPG Classify a patient as high risk or low risk. Mark email as spam or ham. Keen on learning about Classification Algorithms in Machine Learning? Click here! Support Vector Machine (SVM) Let us understand Support Vector Machine (SVM) in detail below. SVMs are classification algorithms used to assign data to various classes. They involve detecting hyperplanes which segregate data into classes. SVMs are very versatile and are also capable of performing linear or nonlinear classification, regression, and outlier detection. Once ideal hyperplanes are discovered, new data points can be easily classified. https://www.simplilearn.com/ice9/free_resources_article_thumb/support-vector-machines-graph-machine-learning.JPG The optimization objective is to find “maximum margin hyperplane” that is farthest from the closest points in the two classes (these points are called support vectors). In the given figure, the middle line represents the hyperplane. SVM Example Let’s look at this image below and have an idea about SVM in general. Hyperplanes with larger margins have lower generalization error. The positive and negative hyperplanes are represented by: https://www.simplilearn.com/ice9/free_resources_article_thumb/positive-negative-hyperplanes-machine-learning.JPG Classification of any new input sample xtest : If w0 + wTxtest > 1, the sample xtest is said to be in the class toward the right of the positive hyperplane. If w0 + wTxtest < -1, the sample xtest is said to be in the class toward the left of the negative hyperplane. When you subtract the two equations, you get: https://www.simplilearn.com/ice9/free_resources_article_thumb/equation-subtraction-machine-learning.JPG Length of vector w is (L2 norm length): https://www.simplilearn.com/ice9/free_resources_article_thumb/length-of-vector-machine-learning.JPG You normalize with the length of w to arrive at: https://www.simplilearn.com/ice9/free_resources_article_thumb/normalize-equation-machine-learning.JPG SVM: Hard Margin Classification Given below are some points to understand Hard Margin Classification. The left side of equation SVM-1 given above can be interpreted as the distance between the positive (+ve) and negative (-ve) hyperplanes; in other words, it is the margin that can be maximized. Hence the objective of the function is to maximize with the constraint that the samples are classified correctly, which is represented as : https://www.simplilearn.com/ice9/free_resources_article_thumb/hard-margin-classification-machine-learning.JPG This means that you are minimizing ‖w‖. This also means that all positive samples are on one side of the positive hyperplane and all negative samples are on the other side of the negative hyperplane. This can be written concisely as : https://www.simplilearn.com/ice9/free_resources_article_thumb/hard-margin-classification-formula.JPG Minimizing ‖w‖ is the same as minimizing. This figure is better as it is differentiable even at w = 0. The approach listed above is called “hard margin linear SVM classifier.” SVM: Soft Margin Classification Given below are some points to understand Soft Margin Classification. To allow for linear constraints to be relaxed for nonlinearly separable data, a slack variable is introduced. (i) measures how much ith instance is allowed to violate the margin. The slack variable is simply added to the linear constraints. https://www.simplilearn.com/ice9/free_resources_article_thumb/soft-margin-calculation-machine-learning.JPG Subject to the above constraints, the new objective to be minimized becomes: https://www.simplilearn.com/ice9/free_resources_article_thumb/soft-margin-calculation-formula.JPG You have two conflicting objectives now—minimizing slack variable to reduce margin violations and minimizing to increase the margin. The hyperparameter C allows us to define this trade-off. Large values of C correspond to larger error penalties (so smaller margins), whereas smaller values of C allow for higher misclassification errors and larger margins. https://www.simplilearn.com/ice9/free_resources_article_thumb/machine-learning-certification-video-preview.jpg SVM: Regularization The concept of C is the reverse of regularization. Higher C means lower regularization, which increases bias and lowers the variance (causing overfitting). https://www.simplilearn.com/ice9/free_resources_article_thumb/concept-of-c-graph-machine-learning.JPG IRIS Data Set The Iris dataset contains measurements of 150 IRIS flowers from three different species: Setosa Versicolor Viriginica Each row represents one sample. Flower measurements in centimeters are stored as columns. These are called features. IRIS Data Set: SVM Let’s train an SVM model using sci-kit-learn for the Iris dataset: https://www.simplilearn.com/ice9/free_resources_article_thumb/svm-model-graph-machine-learning.JPG Nonlinear SVM Classification There are two ways to solve nonlinear SVMs: by adding polynomial features by adding similarity features Polynomial features can be added to datasets; in some cases, this can create a linearly separable dataset. https://www.simplilearn.com/ice9/free_resources_article_thumb/nonlinear-classification-svm-machine-learning.JPG In the figure on the left, there is only 1 feature x1. This dataset is not linearly separable. If you add x2 = (x1)2 (figure on the right), the data becomes linearly separable. Polynomial Kernel In sci-kit-learn, one can use a Pipeline class for creating polynomial features. Classification results for the Moons dataset are shown in the figure. https://www.simplilearn.com/ice9/free_resources_article_thumb/polynomial-kernel-machine-learning.JPG Polynomial Kernel with Kernel Trick Let us look at the image below and understand Kernel Trick in detail. https://www.simplilearn.com/ice9/free_resources_article_thumb/polynomial-kernel-with-kernel-trick.JPG For large dimensional datasets, adding too many polynomial features can slow down the model. You can apply a kernel trick with the effect of polynomial features without actually adding them. The code is shown (SVC class) below trains an SVM classifier using a 3rd-degree polynomial kernel but with a kernel trick. https://www.simplilearn.com/ice9/free_resources_article_thumb/polynomial-kernel-equation-machine-learning.JPG The hyperparameter coefθ controls the influence of high-degree polynomials. Kernel SVM Let us understand in detail about Kernel SVM. Kernel SVMs are used for classification of nonlinear data. In the chart, nonlinear data is projected into a higher dimensional space via a mapping function where it becomes linearly separable. https://www.simplilearn.com/ice9/free_resources_article_thumb/kernel-svm-machine-learning.JPG In the higher dimension, a linear separating hyperplane can be derived and used for classification. A reverse projection of the higher dimension back to original feature space takes it back to nonlinear shape. As mentioned previously, SVMs can be kernelized to solve nonlinear classification problems. You can create a sample dataset for XOR gate (nonlinear problem) from NumPy. 100 samples will be assigned the class sample 1, and 100 samples will be assigned the class label -1. https://www.simplilearn.com/ice9/free_resources_article_thumb/kernel-svm-graph-machine-learning.JPG As you can see, this data is not linearly separable. https://www.simplilearn.com/ice9/free_resources_article_thumb/kernel-svm-non-separable.JPG You now use the kernel trick to classify XOR dataset created earlier. https://www.simplilearn.com/ice9/free_resources_article_thumb/kernel-svm-xor-machine-learning.JPG Naïve Bayes Classifier What is Naive Bayes Classifier? Have you ever wondered how your mail provider implements spam filtering or how online news channels perform news text classification or even how companies perform sentiment analysis of their audience on social media? All of this and more are done through a machine learning algorithm called Naive Bayes Classifier. Naive Bayes Named after Thomas Bayes from the 1700s who first coined this in the Western literature. Naive Bayes classifier works on the principle of conditional probability as given by the Bayes theorem. Advantages of Naive Bayes Classifier Listed below are six benefits of Naive Bayes Classifier. Very simple and easy to implement Needs less training data Handles both continuous and discrete data Highly scalable with the number of predictors and data points As it is fast, it can be used in real-time predictions Not sensitive to irrelevant features Bayes Theorem We will understand Bayes Theorem in detail from the points mentioned below. According to the Bayes model, the conditional probability P(Y|X) can be calculated as: P(Y|X) = P(X|Y)P(Y) / P(X) This means you have to estimate a very large number of P(X|Y) probabilities for a relatively small vector space X. For example, for a Boolean Y and 30 possible Boolean attributes in the X vector, you will have to estimate 3 billion probabilities P(X|Y). To make it practical, a Naïve Bayes classifier is used, which assumes conditional independence of P(X) to each other, with a given value of Y. This reduces the number of probability estimates to 2*30=60 in the above example. Naïve Bayes Classifier for SMS Spam Detection Consider a labeled SMS database having 5574 messages. It has messages as given below: https://www.simplilearn.com/ice9/free_resources_article_thumb/naive-bayes-spam-machine-learning.JPG Each message is marked as spam or ham in the data set. Let’s train a model with Naïve Bayes algorithm to detect spam from ham. The message lengths and their frequency (in the training dataset) are as shown below: https://www.simplilearn.com/ice9/free_resources_article_thumb/naive-bayes-spam-spam-detection.JPG Analyze the logic you use to train an algorithm to detect spam: Split each message into individual words/tokens (bag of words). Lemmatize the data (each word takes its base form, like “walking” or “walked” is replaced with “walk”). Convert data to vectors using scikit-learn module CountVectorizer. Run TFIDF to remove common words like “is,” “are,” “and.” Now apply scikit-learn module for Naïve Bayes MultinomialNB to get the Spam Detector. This spam detector can then be used to classify a random new message as spam or ham. Next, the accuracy of the spam detector is checked using the Confusion Matrix. For the SMS spam example above, the confusion matrix is shown on the right. Accuracy Rate = Correct / Total = (4827 + 592)/5574 = 97.21% Error Rate = Wrong / Total = (155 + 0)/5574 = 2.78% https://www.simplilearn.com/ice9/free_resources_article_thumb/confusion-matrix-machine-learning.JPG Although confusion Matrix is useful, some more precise metrics are provided by Precision and Recall. https://www.simplilearn.com/ice9/free_resources_article_thumb/precision-recall-matrix-machine-learning.JPG Precision refers to the accuracy of positive predictions. https://www.simplilearn.com/ice9/free_resources_article_thumb/precision-formula-machine-learning.JPG Recall refers to the ratio of positive instances that are correctly detected by the classifier (also known as True positive rate or TPR). https://www.simplilearn.com/ice9/free_resources_article_thumb/recall-formula-machine-learning.JPG Precision/Recall Trade-off To detect age-appropriate videos for kids, you need high precision (low recall) to ensure that only safe videos make the cut (even though a few safe videos may be left out). The high recall is needed (low precision is acceptable) in-store surveillance to catch shoplifters; a few false alarms are acceptable, but all shoplifters must be caught. Learn about Naive Bayes in detail. Click here! Decision Tree Classifier Some aspects of the Decision Tree Classifier mentioned below are. Decision Trees (DT) can be used both for classification and regression. The advantage of decision trees is that they require very little data preparation. They do not require feature scaling or centering at all. They are also the fundamental components of Random Forests, one of the most powerful ML algorithms. Unlike Random Forests and Neural Networks (which do black-box modeling), Decision Trees are white box models, which means that inner workings of these models are clearly understood. In the case of classification, the data is segregated based on a series of questions. Any new data point is assigned to the selected leaf node. https://www.simplilearn.com/ice9/free_resources_article_thumb/decision-tree-classifier-machine-learning.JPG Start at the tree root and split the data on the feature using the decision algorithm, resulting in the largest information gain (IG). This splitting procedure is then repeated in an iterative process at each child node until the leaves are pure. This means that the samples at each node belonging to the same class. In practice, you can set a limit on the depth of the tree to prevent overfitting. The purity is compromised here as the final leaves may still have some impurity. The figure shows the classification of the Iris dataset. https://www.simplilearn.com/ice9/free_resources_article_thumb/decision-tree-classifier-graph.JPG IRIS Decision Tree Let’s build a Decision Tree using scikit-learn for the Iris flower dataset and also visualize it using export_graphviz API. https://www.simplilearn.com/ice9/free_resources_article_thumb/iris-decision-tree-machine-learning.JPG The output of export_graphviz can be converted into png format: https://www.simplilearn.com/ice9/free_resources_article_thumb/iris-decision-tree-output.JPG Sample attribute stands for the number of training instances the node applies to. Value attribute stands for the number of training instances of each class the node applies to. Gini impurity measures the node’s impurity. A node is “pure” (gini=0) if all training instances it applies to belong to the same class. https://www.simplilearn.com/ice9/free_resources_article_thumb/impurity-formula-machine-learning.JPG For example, for Versicolor (green color node), the Gini is 1-(0/54)2 -(49/54)2 -(5/54) 2 ≈ 0.168 https://www.simplilearn.com/ice9/free_resources_article_thumb/iris-decision-tree-sample.JPG Decision Boundaries Let us learn to create decision boundaries below. For the first node (depth 0), the solid line splits the data (Iris-Setosa on left). Gini is 0 for Setosa node, so no further split is possible. The second node (depth 1) splits the data into Versicolor and Virginica. If max_depth were set as 3, a third split would happen (vertical dotted line). https://www.simplilearn.com/ice9/free_resources_article_thumb/decision-tree-boundaries.JPG For a sample with petal length 5 cm and petal width 1.5 cm, the tree traverses to depth 2 left node, so the probability predictions for this sample are 0% for Iris-Setosa (0/54), 90.7% for Iris-Versicolor (49/54), and 9.3% for Iris-Virginica (5/54) CART Training Algorithm Scikit-learn uses Classification and Regression Trees (CART) algorithm to train Decision Trees. CART algorithm: Split the data into two subsets using a single feature k and threshold tk (example, petal length < “2.45 cm”). This is done recursively for each node. k and tk are chosen such that they produce the purest subsets (weighted by their size). The objective is to minimize the cost function as given below: https://www.simplilearn.com/ice9/free_resources_article_thumb/cart-training-algorithm-machine-learning.JPG The algorithm stops executing if one of the following situations occurs: max_depth is reached No further splits are found for each node Other hyperparameters may be used to stop the tree: min_samples_split min_samples_leaf min_weight_fraction_leaf max_leaf_nodes Gini Impurity or Entropy Entropy is one more measure of impurity and can be used in place of Gini. https://www.simplilearn.com/ice9/free_resources_article_thumb/gini-impurity-entrophy.JPG It is a degree of uncertainty, and Information Gain is the reduction that occurs in entropy as one traverses down the tree. Entropy is zero for a DT node when the node contains instances of only one class. Entropy for depth 2 left node in the example given above is: https://www.simplilearn.com/ice9/free_resources_article_thumb/entrophy-for-depth-2.JPG Gini and Entropy both lead to similar trees. DT: Regularization The following figure shows two decision trees on the moons dataset. https://www.simplilearn.com/ice9/free_resources_article_thumb/dt-regularization-machine-learning.JPG The decision tree on the right is restricted by min_samples_leaf = 4. The model on the left is overfitting, while the model on the right generalizes better. Random Forest Classifier Let us have an understanding of Random Forest Classifier below. A random forest can be considered an ensemble of decision trees (Ensemble learning). Random Forest algorithm: Draw a random bootstrap sample of size n (randomly choose n samples from the training set). Grow a decision tree from the bootstrap sample. At each node, randomly select d features. Split the node using the feature that provides the best split according to the objective function, for instance by maximizing the information gain. Repeat the steps 1 to 2 k times. (k is the number of trees you want to create, using a subset of samples) Aggregate the prediction by each tree for a new data point to assign the class label by majority vote (pick the group selected by the most number of trees and assign new data point to that group). Random Forests are opaque, which means it is difficult to visualize their inner workings. https://www.simplilearn.com/ice9/free_resources_article_thumb/random-forest-classifier-graph.JPG However, the advantages outweigh their limitations since you do not have to worry about hyperparameters except k, which stands for the number of decision trees to be created from a subset of samples. RF is quite robust to noise from the individual decision trees. Hence, you need not prune individual decision trees. The larger the number of decision trees, the more accurate the Random Forest prediction is. (This, however, comes with higher computation cost). Key Takeaways Let us quickly run through what we have learned so far in this Classification tutorial. Classification algorithms are supervised learning methods to split data into classes. They can work on Linear Data as well as Nonlinear Data. Logistic Regression can classify data based on weighted parameters and sigmoid conversion to calculate the probability of classes. K-nearest Neighbors (KNN) algorithm uses similar features to classify data. Support Vector Machines (SVMs) classify data by detecting the maximum margin hyperplane between data classes. Naïve Bayes, a simplified Bayes Model, can help classify data using conditional probability models. Decision Trees are powerful classifiers and use tree splitting logic until pure or somewhat pure leaf node classes are attained. Random Forests apply Ensemble Learning to Decision Trees for more accurate classification predictions. Conclusion This completes ‘Classification’ tutorial. In the next tutorial, we will learn 'Unsupervised Learning with Clustering.'
rishikksh20/CrossViT-pytorch
Implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification
gaohuang/MSDNet
Multi-Scale Dense Networks for Resource Efficient Image Classification （ICLR 2018 Oral）