luke-davidson/MachineLearning

Assignments completed for my Machine Learning course: Topics include probability and statistics proofs, MLE/MAP parameter estimation, EM Algorithm, Bayes Theorem implementations, gradient descent methods, Neural Networks and Deep Learning.

Machine Learning

This repo holds all programming assignments completed for my Machine Learning course (Fall 2022).

Assignment Descriptions

A1 --- Probability, MLE+MAP Estimations and the EM Algorithm

Includes probability proofs, PMF derivations, MLE, MAP and Bayes parameter estimation calculations, EM algorithm derivation and implementation and a high dimensional hypercube proof.

• Code
  • a1_Prob_MLE+MAP_EM/a1_EM-Algorithm.py
    • Implementation of the EM algorithm.
  • a1_Prob_MLE+MAP_EM/a1_Hypercube.py
    • Generating points in a high dimensional hypercube and hypersphere and proving the performed derivation in the report.
• Report: a1_Prob_MLE+MAP_EM/a1_report.pdf

A2 --- Bayes Theorem Implementation + Gradient Descent

Implementation of the perceptron algorithm, Naive Bayes classifier, basis functions, optimal decision surface derivation, linear regression gradient descent derivations.

• Code
  • a2_Bayes_GradDescent/a2_NaiveBayesClassifier.py
    • Implementation of a Naive Bayes Classifier with binary classifications. Calculates class priors, posteriors and final class assignments for both low and high dimensional data points.
  • a2_Bayes_GradDescent/a2_GradDescent.py
    • Implementation of gradient descent derivations for minimizing sum of squared errors and sum of squared distances.
  • a2_Bayes_GradDescent/a2_HyperplaneAccuracy.py
    • Implementation of the perceptron algorithm as it relates to hyperplane accuracy.
• Report: a2_Bayes_GradDescent/a2_report.pdf

A3 --- Project Proposal

Copy of my semester project proposal. See TimeSeriesMotionClassification for whole project.

• Report: a3_ProjectProposal/a3_ProjectProposal.pdf

A4 --- Neural Networks + Performance Evaluation

Implementation of differently sized Neural Networks, matrix factorization, the Alternating Least Squares algorithm and representational bias in neural network applications.

• Code: a4_NeuralNetworks_ROC/a4_NeuralNetworks.py
  • Test data is generated based on decision regions (defined in self.bounds) and is assigned a class based on probabilities (ex. 98% will be correctly labeled, 2% will be incorrectly labeled). Neural networks of various sizes are then created, trained and tested on the generated data. Performance of differently sized neural nets is then evaluated.
• Report: a4_NeuralNetworks_ROC/a4_report.pdf