Minerva CS156 Machine Learning

Logistics (Fall 2023)

Pre-requisites: Minerva CS113 Linear Algebra (Spring 2023), Minerva CS111 Single and Multivariable Calculus (Fall 2022),

Syllabus: https://course-resources.minerva.edu/uploaded_files/mu/00335826-4661/cs156-syllabus.pdf

Textbook: Probabilistic Machine Learning https://probml.github.io/pml-book/book1.html

Concept Notes

Unit 1: Simple ML models

• Session 1 Introduction
  • machine learning
    • supervised learning
    • unsupervised learning
    • self-supervised learning
    • reinforcement learning
• Session 2: Linear Algebra 1: Tensors, Classification, and Regression
  • tensor
  • simple linear regression
  • binomial logistic regression
• Session 3: Linear Algebra 2: Collinearity
  • collinearity
  • multiple regression
  • Review Minerva CS113 Linear Algebra (Spring 2023)
• Session 4: Trees 1: Classification by Partition
  • decision tree (ML)
• Session 5: Max Likelihood 1: Naive Bayes
  • Naive Bayes
  • gaussian naive bayes
• Session 6: Networks 1: Feed-Forward Neural Networks
  • neural network
    • neuron
    • activation function
  • gradient descent
• Session 7: Gradients: Multivariate Derivatives
  • Review partial derivatives, gradient and chain rule (calculus)
  • backpropagation
  • jacobian matrix
• Session 8: Max likelihood 2: Parameter Estimation from Data
  • Review likelihood function
  • maximum likelihood estimation
• Session 9: Metrics and Cross-Validation
  • classification metric
    • confusion matrix
    • precision-recall trade off
    • ROC curve
    • AUC
  • machine learning development process
  • cross validation, including k-fold cross validation

Unit 2: Intuitions for learning in high-dimensional spaces

• Session 11: Data Ethics and the Bias-Variance Trade of
  • Bias Variance Tradeoff
    • overfitting + underfitting
  • data augmentation (to overcome data bias)
  • data synthesis
• Session 12: Basis Functions 1: Functions as Parameters
  • regression as linear model
  • basis functions
• Session 13: Basis Functions 2: Ridge and Lasso
  • regularization
    • Ridge regularization
    • Lasso regularization
  • Review Bias Variance Tradeoff
• Session 14: Dimensionality reduction 1: PCA
  • Review eigenstuff, vector projection, change of basis
  • principal component analysis
• Session 15: Projections 1: Filters, Convolutions, & Transforms
  • convolutional neural network
• Session 16: Dimensionality Reduction 2: Autoencoders
  • autoencoder
• Session 17: Projections 2: Transfer Learning
  • transfer learning
    • fine tuning
  • language model
• Session 18: Projections 3: The kernel trick
  • Review inverse matrix, column space, null space
  • Support Vector Machines
• Session 19: Time Series 1: Recurrent neural networks
  • recurrent neural network
    • long short-term memory (LSTM)
    • gated recurrent unit (GRU)
  • Enrichment: sequence-to-sequence model
• Session 20: Trees 2: XGBoost
  • Review decision tree (ML)
  • extreme gradient boosting
  • Enrichment: ensemble method
    • bootstrap aggregation
      • Adaboost
    • boosting
      • XGBoost

Unit 3: Complex Models

• Session 22: Dimensionality Reduction 3: Gaussian Mixture Models
  • Review unsupervised learning, Gaussian distribution
  • latent variable
  • gaussian mixture model
• Session 23: Time Series 2: Hidden Markov models
  • Review Bayes’ Theorem
  • Markov chain
  • Markov transition matrix
  • Hidden Markov model
• Session 24: Transformer-Based Large Language Models
  • Review neural network, recurrent neural network
  • transformer
    • attention mechanisms
      • self-attention
      • cross attention
  • neural language model
• Session 25: Review 3
  • diffusion model

References

• ML Specialization
• NLP Specialization