Welcome to Statistical Machine Learning Course#
Mahmood Amintoosi, Spring 2026
Computer Science Dept, Ferdowsi University of Mashhad
Table of Contents#
- What Is Statistical Machine Learning?
- Understanding the Covariance Matrix
- Maximum Likelihood Estimation (MLE)
- Mahalanobis Distance
- Naive Bayes Classification
- Bayes Classifiers
- Naive Bayes Classification with multiple features: Fruit Example
- Bayesian Classification
- Bayesian Decision Theory
- Linear models
- Orthogonal Matching Pursuit (OMP) for Sparse Recovery
- Connection Between Sparse Learning and Functional Analysis (Further Reading)
- MRI reconstruction (Further Reading)
- Linear models for Classification
- Feature Maps: Bridging to Kernel Methods
- The Kernel Method (Kernel Trick)
- Kernel K-means Clustering
- Kernel Regression: From Linear to Nonlinear Modeling
- Gaussian Mixture Models, Part 1
- Gaussian Mixture Model, Part 2
- Hidden Markov Models
- Viterbi Algorithm
- Dynamic Programming
- Principal Component Analysis, Part 1
- Principal Component Analysis, Part 2
- PCA Workflow: From Single Sample to Full Matrix
- Equivalence of PCA’s Two Optimization Perspectives and Error Analysis (Further Reading)
- Examples and Applications
- Summary
- Appendix: Mathematics and Machine Learning
- Sparse Optimization: Theory and Applications
- Appendix: Bayes Decision Theory — Discrete Features
- Appendix: The Moore-Penrose Pseudoinverse
Course Resources for Statistical Machine Learning#
Pattern Classification, by Richard O. Duda, Peter E. Hart, David G. Stork [DHS00]
The Elements of Statistical Learning: Data Mining, Inference, and Prediction , by Trevor Hastie, Robert Tibshirani, and Jerome Friedman [HTF09]
Pattern Recognition and Machine Learning, by Christopher M. Bishop [Bis16]
Machine Learning: A Bayesian and Optimization Perspective, by Sergios Theodoridis [The20]
Python Data Science Handbook by Jake VanderPlas: This website contains the full text of the Python Data Science Handbook by Jake VanderPlas; the content is available on GitHub in the form of Jupyter notebooks.
Course Staff#
Instructors#
|
Mahmood Amintoosi Email: m.amintoosi AT um.ac.ir |
Teaching Assistants#
Hoda MehrBagherpour Email: mehrbagherpour AT mail.um.ac.ir |
(TA Head) |
Questions?#
I will be having office hours for this course on Saturday (10:00 AM–11:30 AM). If this is not convenient, email me at m dot amintoosi AT um.ac.ir, talk to me after class or schedule an appointment via Calendly. Also you can find me at Bale, Rubika, Eitaa and FUM-VU.
Note
These lectures were built using the new Sphinx-based Jupyter Book 2.0 tool set, as part of the ExecutableBookProject. They are intended mainly as a demonstration of these tools. Instructions for how to build them from source can be found in the Jupyter Book documentation.
References#
M. Amintoosi, F. Farbiz, and M. Fathy. A QR Decomposition based mixture model algorithm for background modeling. In ICICS2007, Sixth International Conference on Information, Communication and Signal Processing, 1–5. Singapore, December 2007. Indexed by IEEE and SCOPUS.
M. Amintoosi, M. Fathy, N. Mozayani, and A.T. Rahmani. A fish school clustering algorithm: applied to student sectioning problem. Dynamics of Continuous Discrete & Impulse Systems, series B: Applications and Algorithms, 2:696–699, December 2007. Post Proceeding of LSMS2007, Life System Modeling and Simulation 2007, China.
M. Amintoosi and J. Haddadnnia. Feature selection in a fuzzy student sectioning algorithm. Lecture Notes in Computer Science, 3616:147–160, 2005. Indexed by DBLP.
M. Amintoosi, H. Sadoghi Yazdi, and J. Haddadnnia. Fuzzy student sectioning. In PATAT04: Practice and Theory of Automated Timetabling, 421–424. USA, Aug 2004.
Mahmood Amintoosi. Retina vessel segmentation using knn matting. In 3rd International Conference on Pattern Recognition and Image Analysis of Iran. Shahrekord University, 2017. دقیقتر کردن استخراج رگهای خونی شبکیه چشم با روش درهمتنیدگی تصویر مبتنی بر نزدیکترین همسایگی. URL: https://www.dropbox.com/s/klkoagz3m3dc98t/1396-IPRIA2017-Matting.pdf?dl=0.
Mahmood Amintoosi. The application of taylor expansion in reducing the size of convolutional neural networks for classifying impressionism and miniature style paintings. Mathematics and Society, 5(1):1–16, 2020. کاربرد بسط تیلور در کاهش حجم شبکه های عصبی پیچشی برای طبقه بندی نقاشی های سبک امپرسیونیسم و مینیاتور.
Mahmood Amintoosi. Combining regularization and optimal brain damage methods for reducing a deep learning model size. Machine Vision and Image Processing, 9(1):31–45, 2021. ترکیب روش منظمسازی تُنُک و آسیب مغزی بهینه در کوچکسازی یک مدل یادگیری عمیق.
Mahmood Amintoosi. Overlapping clusters in cluster graph convolutional networks. Journal of Algorithms and Computation, 53(2):33–45, 2021.
Mahmood Amintoosi. Fully connected to fully convolutional: road to yesterday. Soft Computing and Information Technology, 11(1):60–72, 2022. تمام متصل به تمام پیچشی: پلی به گذشته.
Mahmood Amintoosi. Style transfer for data augmentation in convolutional neural networks applied to fire detection. Computational Intelligence in Electrical Engineering, 13(4):97–114, 2022. انتقال سبک برای افزایش دادههای آموزشی شبکههای کانولوشنی در شناسایی شعلۀ آتش. doi:10.22108/isee.2021.124044.1490.
Mahmood Amintoosi. Improving housing price prediction with spatial information representation based on random walk. In 3rd Specialized Seminar on Data Science and Its Applications, 20. Ferdowsi University of Mashhad, 2024. بهبود پیشبینی قیمت مسکن با بازنمایی اطلاعات مکانی مبتنی بر قدمزنی تصادفی.
Mahmood Amintoosi. Traffic prediction using graph convolutional networks based on learning. In 55th Annual Iranian Mathematics Conference, 145–148. Ferdowsi University of Mashhad, 2024. پیشبینی ترافیک با شبکههای پیچشی گراف مبتنی بر یادگیری.
Mahmood Amintoosi. Deepwalk for student sectioning. Data & Knowledge Engineering, xx(yy):–, 202x.
Mahmood Amintoosi and Farzam Farbiz. Eigenbackground revisited: can we model the background with eigenvectors? Journal of Mathematical Imaging and Vision, 64(5):463–477, 2022.
Mahmood Amintoosi and Tayyebe Fayyaz. Genetic algorithms for spectral clustering parameter estimation in mri images. In 8th International Conference of the Iranian Operations Research Society. Ferdowsi University of Mashhad, 2015. محاسبه پارامترهای خوشهبندی طیفی در تصاویر MRI با الگوریتم ژنتیک.
Mahmood Amintoosi, Sakineh Khorsandi, and Mehdi Zaferanieh. Elm evaluation for image segmentation. In 3rd International Conference on Pattern Recognition and Image Analysis of Iran. Shahrekord University, 2017. ارزیابی عملکرد ماشین یادگیر نهایی در قطعهبندی تصاویر.
Mahmood Amintoosi and Eisa Kohan-Baghkheirati. Graph feature selection for anti-cancer plant recommendation. Control and Optimization in Applied Mathematics, 8(2):1–15, 2023.
Mahmood Amintoosi and Somayeh Poursediq. افزایش سرعت شناسایی زمینه ویديو با محاسبه مرحلهای تجزیه QR. In هشتمین سمینار جبرخطی و کابردهای آن. دانشگاه کردستان, 2015. Increasing the speed of video background Subtraction by incremental QR decomposition, (2015).
Mahmood Amintoosi and Farzaneh Rashidabadi. Enhancement of heart coronary vessel segmentation using semi-supervised learning. In 8th International Conference of the Iranian Operations Research Society. Ferdowsi University of Mashhad, 2015. آشکارسازی بهتر شریانهای کرونری قلب با یادگیری نیمهنظارتیخودکار.
Alale Asaran. Super resolution via sparse representation. Master's thesis, Hakim Sabzevari University, Faculty of Mathematics and Computer Science, Winter 2016. فراتفکیک پذیری با نمایش تنک. URL: http://hcloud.hsu.ac.ir/index.php/s/W9ImIzeV6C1mqZo.
Amir Jologir Baghestan, Mahmood Amintoosi, and Mohammad Arashi. Graph neural networks for predicting molecular properties. In 3rd Specialized Seminar on Data Science and Its Applications, 37. Ferdowsi University of Mashhad, 2024. شبکههای عصبی گراف در پیشگویی خواص مولکولی.
Mahboube Bakhshali. The subspace pursuit method in sparseoptimization. Master's thesis, Hakim Sabzevari University, Faculty of Mathematics and Computer Science, September 2018. روش جستجوی زیرفضا در بهینه سازی تنک. URL: http://hcloud.hsu.ac.ir/index.php/s/jacmnZiPfNFpYfk.
C.M. Bishop. Pattern Recognition and Machine Learning. Information Science and Statistics. Springer New York, 2016. ISBN 9781493938438. URL: https://books.google.it/books?id=kOXDtAEACAAJ.
David L. Donoho. Compressed sensing. IEEE Transactions on Information Theory, 52(4):1289–1306, 2006. doi:10.1109/TIT.2006.871582.
Richard O. Duda, Peter E. Hart, and David G. Stork. Pattern Classification (2nd Edition). Wiley-Interscience, 2 edition, November 2000. ISBN 0471056693. URL: https://file.fouladi.ir/courses/pr/books/%5BDuda%5D_PatternClassification.pdf.
R. Farhadi, M. Hadavifar, M. Moeinaddini, and M. Amintoosi. Prediction of co and pm10 in cold and warm seasons and survey of the effect of instability indices on contaminants using artificial neural network: a case study in tehran city. Iranian (Iranica) Journal of Energy & Environment, 13(1):71–78, 2022. doi:10.5829/ijee.2022.13.01.08.
Razieh Farhadi, Mojtaba Hadavifar, Mazaher Moeinaddini, and Mahmood Amintoosi. Prediction of the air quality by artificial neural network using instability indices in the city of tehran-iran. AUT Journal of Civil Engineering, 4(4):–, 2020. doi:10.22060/ajce.2019.17018.5609.
Mohammadreza Fatehinia, Mahmood Amintoosi, and Seyed Masih Sajadi. Click rate prediction in online advertising industry with real data and its challenges. In 3rd Specialized Seminar on Data Science and Its Applications, 26. Ferdowsi University of Mashhad, 2024. پیشبینی نرخ کلیک در صنعت تبلیغات آنلاین با دادههای واقعی و چالشهای آن.
M. Fathy, N. Mozayani, and M. Amintoosi. Outlier removal for super-resolution problem using QR-Decomposition. In Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition, 271–277. USA, July 14-17 2008. Indexed by DBLP. URL: http://webpages.iust.ac.ir/mamintoosi/papers/IPCV08.pdf.
Loubna El Gueddari, Chaithya Giliyar Radhakrishna, Zaccharie Ramzi, Samuel Farrens, Sophie Starck, Antoine Grigis, Jean-Luc Starck, and Philippe Ciuciu. PySAP-MRI: a Python Package for MR Image Reconstruction. In ISMRM workshop on Data Sampling and Image Reconstruction. Sedona, AZ, United States, January 2020. URL: https://inria.hal.science/hal-02399267.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: data mining, inference and prediction. Springer, 2 edition, 2009. Edition 2, print 12. URL: https://hastie.su.domains/ElemStatLearn/.
Fatemeh Sadat Hosseini and Mahmood Amintoosi. Graph minimum cut using simulated annealing. In 7th International Conference of the Iranian Operations Research Society. Semnan, 2014. برش کمینهی گراف با شبیهسازی تبریدی.
Fatemeh Sadat Hosseini and Mahmood Amintoosi. Graph minimum cut using tabu search. In 7th International Conference of the Iranian Operations Research Society. Semnan, 2014. برش کمینهی گراف باجستجوی ممنوعه.
Fatemeh Sadat Hosseini and Mahmood Amintoosi. Inefficiency of the karger's algorithm in min-cut of weighted graphs. In 3rd Seminar on Control and Optimization, 21–24. Hakim Sabzevari University, 2019. بررسی نا کارآمدی الگوریتم کارگر در برش کمینه گرافهای وزن دار.
Hoda Mehrabagherpour, Mahmood Amintoosi, and Mohammad Arashi. Urban traffic prediction using graph convolutional networks. In 3rd Specialized Seminar on Data Science and Its Applications, 25. Ferdowsi University of Mashhad, 2024. پیشبینی ترافیک شهری با بهرهگیری از شبکههای پیچشی گراف.
Mehdi Moghimi and Mahmood Amintoosi. Mri image segmentation using pulse-coupled neural networks. In 5th National Conference on Electrical and Electronics Engineering. Gonabad, 2013. تشخیص ناحیه چربی در تصاویر MRI با استفاده از شبكه عصبی با كوپلاژ پالسی.
Mina Mortazavi, Morteza Gachpazan, Mahmood Amintoosi, and Soheil Salashour. Fractional derivative approach to sparse super resolution. The Visual Computer, 39(7):3011–3028, Jul 2023.
Mehdi Nemati. A gradient based method for thespectral graph partitioning. Master's thesis, Hakim Sabzevari University, Faculty of Mathematics and Computer Science, September 2018. روشی مبتنی بر گرادیان برای افرازبندی طیفی گراف. URL: https://hcloud.hsu.ac.ir/index.php/s/tR53d9306yZSAc7.
Mehdi Nemati and Mahmood Amintoosi. Enhancing text extraction from scanned medical documents using large language models. In Third Seminar on Data Science and its Applications, 58. Ferdowsi University of Mashhad, 2024.
Mehdi Nemati, Mahmood Amintoosi, and Mehdi Zaferanieh. Conjugate gradient initialization using genetic algorithm in spectral clustering. In 6th Seminar on Harmonic Analysis and Applications. Hakim Sabzevari University, 2018. مقدار دهی اولیه گرادیان مزدوج در خوشه بندی طیفی با الگوریتم ژنتیک.
Farzane Rashidabadi. Image matting. Master's thesis, Hakim Sabzevari University, Faculty of Mathematics and Computer Science, January 2016. برش هوشمند تصویر. URL: http://hcloud.hsu.ac.ir/index.php/s/OaHkkTSsO8mNrk0.
A.K.M.E. Saleh, M. Arashi, and B.M.G. Kibria. Theory of Ridge Regression Estimation with Applications. Wiley Series in Probability and Statistics. Wiley, 2019. ISBN 9781118644614. URL: https://books.google.com/books?id=v0KCDwAAQBAJ.
A.K.M.E. Saleh, M. Arashi, R.A. Saleh, and M. Norouzirad. Rank-Based Methods for Shrinkage and Selection: With Application to Machine Learning. Wiley, 2022. ISBN 9781119625391. URL: https://books.google.com/books?id=f8p6EAAAQBAJ.
Mehdi Sheida, Hessam Ekhtiyar, and Mahmood Amintoosi. A unified algorithm for segmentation of various medical images. In 2nd National Conference on Soft Computing and Information Technology. Mahshahr, 2012. الگوریتمی واحد برای ناحیه بندی انواع تصاویر پزشکی.
S. Theodoridis. Machine Learning: A Bayesian and Optimization Perspective. Academic Press, 2020. ISBN 9780128188040. URL: https://books.google.com/books?id=l-nEDwAAQBAJ.
Jake VanderPlas. Python Data Science Handbook. O'Reilly Media, Inc., 2016. ISBN 9781491912058. URL: https://www.oreilly.com/library/view/python-data-science/9781491912126/.
H. Sadoghi Yazdi, M. Amintoosi, and M. Fathy. Facial expression recognition in video using QIM and ITMI. In 4th Conference on Machine Vision and Image Processing. Mashhad, Iran, February 2006. Ferdowsi University of Mashhad. شناسایی حالت چهره با استفاده از پایگاه دادهٔ مكانی- زمانی QIM و ITMI.
M.J. Zaki, W. Meira, and W. Meira. Data Mining and Machine Learning: Fundamental Concepts and Algorithms. Cambridge University Press, 2020. ISBN 9781108473989.
Ghasem Zolfaghari, Sara Nezamparvar, and Mahmood Amintoosi. Modeling dust particles from stack with artificial neural network and studying electrofilter performance: a case study of zaveh cement factory. Journal of Natural Environment, ():–, 2025. مدلسازی ذرات غبار خروجی از دود کش با شبکه عصبی مصنوعی و مطالعه عملکرد الکتروفیلتر: مطالعه موردی کارخانه سیمان زاوه. URL: https://jne.ut.ac.ir/article_100076.html, doi:10.22059/jne.2025.380898.2713.