Numpy implementation of some ML Algorithms

This repo contains a numpy from-scratch implementation of some ML algorithms, initially designed for the MNIST Digit classification task (~3% error, averaged over 20 runs of 5 fold cross-validation)

• kNN
• 3 Layer MLP (ReLU & Softmax activations), Cross-Entropy Loss
• Least Squares
• Winnow
• One-vs-One and One-vs-All Muliticlass Kernel Perceptron
• logistic regression with AdaGrad optimiser
• SVM (primal + dual)

Additional Functions

files: CV.py and helper_functions.py

• random train/test split
• Cross Validation
• Gram Matrix (for polynomial and Gaussian kernels)
• numerical gradient check

TODO: change output type from error rate to prediction