Ferdowsi University of Mashhad, Spring 2024 (1402 SH)
About
Table of contents
About
Course Description: Learning Theory
Welcome to the Learning Theory course! In this course, we will explore fundamental concepts and techniques related to machine learning, statistical modeling, and neural networks. Whether you’re a beginner or an experienced learner, this course will provide you with a solid foundation in various aspects of learning theory.
Topics Covered
Bayesian and Machine Learning
- Understand the principles of Bayesian inference.
- Explore machine learning algorithms and their probabilistic foundations.
- Dive into the Expectation-Maximization (EM) algorithm for parameter estimation.
- Investigate Gaussian Mixture Models (GMMs) and their applications.
- Study kernel methods for non-linear data representations.
Discriminant Linear Functions
- Learn about regression techniques and the normal equations.
- Understand how linear discriminant functions can be used for classification tasks.
Neural Networks
- Explore the basics of neural networks.
- Study the Perceptron model and its training process.
- Dive into Multilayer Perceptrons (MLPs) and their architectures.
- Investigate Hopfield Neural Networks for associative memory.
- Discover Self-Organizing Maps (Kohonen Networks) for unsupervised learning.
Other Topics
- Introduction to Image Processing and Computer Vision
- Delve into Genetic Algorithms and their optimization capabilities.
- Explore Ridge Regression as a regularization technique.
- Understand the concept of VC Dimension in machine learning.
- Study Hidden Markov Models (HMMs) and their applications in sequential data analysis.
Prerequisites
- Basic knowledge of linear algebra, probability, and programming (Python).
- Curiosity and enthusiasm for understanding the theoretical foundations of machine learning.
Course Goals
By the end of this course, you will have a solid understanding of learning theory concepts, practical skills in implementing various algorithms, and the ability to critically analyze and apply these techniques to real-world problems.
Let’s embark on this exciting journey of learning theory together! 🌟
Class time and Location
Suturday and Monday 8:00-9:40 (Spring 2024), Room 10.
References
Some of the used resources are:
- Data Mining and Analysis: Fundamental Concepts and Algorithms by Mohammed J. Zaki and Wagner Meira Jr., 2021 (Data Mining & Analysis) PDF
- Pattern Recognition, by Sergios Theodoridis and Konstantinos Koutroumpas, 2009
- The Elements of Statistical Learning (ESL)
- Foundations of Data Science (FDS), by Avrim Blum, John Hopcroft, and Ravindran Kannan, 2018
- Hands On Machine Learning with Scikit-Learn, Keras, and TensorFlow Concepts, Tools, and Techniques to Build Intelligent Systems-O’Reilly Media, Book by Aurélien Géron, 2022, Part II.
- Learn PyTorch: A website that provides tutorials and examples on how to use PyTorch for various deep learning tasks.
Grading:
- Homework – 20%
- Midterm – 30%
— Will consist of mathematical problems and/or programming assignments. - Seminars - 10%
- Final – 40%
Academic Honor Code:
Honesty and integrity are vital elements of the academic works. All your submitted assignments must be entirely your own (or your own group’s).
We will follow the standard of Faculty of Mathematical Sciences approach:
- You should not use code of others or be looking at code of others when you write your own: You can talk to people but have to write your own solution/code
- You can talk to others about the algorithm(s) to be used to solve a homework problem; as long as you then mention their name(s) on the work you submit
Questions?
I will be having office hours for this course on Monday (10:00 AM–11:30 AM). If this is not convenient, email me at m.amintoosi@um.ac.ir, talk to me after class or schedule an appointment via Calendly.
Our Slack workspace
Come and join our Slack group to engage in course discussions.