/Machine-Learning-Algorithms

This repository contains implementation of Neural Network,k-Means and Gaussian Mixture Models with Python

Primary LanguagePythonMIT LicenseMIT

Machine-Learning-Models

This repository contains implementation of basic ML algorithms Models in Python without use of any frameworks.

Algorithms

Supervised learning using a Neural Network to predict a function that classisfies MNIST digits.

Neural network structure: The neural network takes in (28x28) 784 inputs, one hidden layer with n hidden units (where n is a parameter that can be changed), and 10 output units(0-9). The hidden and output units uses sigmoid activation function. The network is fully connected —that is, every input unit connects to every hidden unit, and every hidden unit connects to every output unit. Every hidden and output unit also has a weighted connection from a bias unit, whose value is set to 1.

Parameters of the network:

Number of neurons in the hidden layer - n = 100

Number of epochs - epochs = 50

Learining rate - eta = 0.1

Momentum - alpha = 0.9

Accuracy vs Parameters
η α n Training Accuracy Test Accuracy
0.1 0.9 20 96% 93%
0.1 0.9 50 98% 96.8%
0.1 0.9 100 99.6% 96.85%
0.1 0 100 99.6% 97.7%
0.1 0.25 100 99.6% 97.7%
0.1 0.5 100 99.6% 97.6%

Unsupervised learning with k-Means clustering using EM algorithm

Parameters

k - number of clusters

Input is 1500 data points(2d) taken from random Gaussian distribution. Although it can accept any 2d data as input. Plots and log-likelihood for different values of k can be found here

Clusters

k = 2 k=2 k = 3 k=3 k = 4 k=4 k = 5 k=5 3.

Unsupervised learning with GMM using EM algorithm

Parameters

k - number of clusters

Input is 1500 data points(2d) taken from random Gaussian distribution. Although it can accept any 2d data as input. Plots and log-likelihood for different values of k can be found here. It makes use of k-Means Clustering to get k clusters and then run Gaussian model and predicts a Gaussian around the data.

Gaussians

k = 2 k=2 k = 3 k=3 k = 4 k=2 k = 5 k=2

Dependencies

  1. Python
  2. Matplotlib