Machine Learning Algorithms

Table of contents

• kNN
• Linear Classifier
• SVM
• Tree
• Ensemble
• GMM
• K-means
• PCA
• SGD
• CNN
• RNN

kNN

k Nearest Neighbor classifier

1. How To
   For the query data point, find the k closest training data points,
   and predict using their labels.

2. Hyper Parameter
   k, metric

3. Issues

   • time complexity : $O(1)$ for training, $O(n)$ for predicting
   • curse of dimensionality (exponential)
   • In computer vision, pixel-level nearness doesn’t represent high-level similarity.

Linear Classifier

by affine transformation

SVM

Support Vector Machine

Tree

Ensemble

GMM

K-means

clustering

PCA

Principal Component Analysis

SGD

Stochastic Gradient Descent

CNN

Convolutional Neural Network

RNN

Recurrent Neural Network