Basics of Machine Learning: From Naive Bayes to Backpropagation
Introduction: categorical vs. numeric, supervised vs. unsupervised, generative vs. discriminative
Naive Bayes classifier: conditional independence, Gaussian example, spam detection, missing data
Decision Trees: ID3 algorithm, entropy, information gain, overfitting, Random Forest algorithm
Evaluation: over-fitting, generalization, stratification, cross-validation, ROC curve, MSE vs MAE
k-Nearest Neighbors: classification, regression, Parzen windows, K-D trees, Locality-Sensitive Hashing
Clustering: K-means algorithm, intrinsic vs intrinsic evaluation, image representation
Gaussiam Mixture Models (GMM) and the Expectation Maximization (EM) Algorithm
PCA: Eigenvectors and Eigenvalues, Principal Components, Linear Discriminant Analysis
Hierarchical Clustering: single- vs complete-link, Dendrogram, Lance-Williams algorithm
Max-margin classifiers: PA algorithm, Support Vector Machine (SVM), SMO algorithm
Neural Networks: feed-forward vs. recurrent networks, XOR problem, deep networks, Backpropagation