Optimization Theory (DATA 620020)

Lecture Time and Venue:

Reading Materials:

• 刘浩洋, 户将, 李勇锋, 文再文. "最优化：建模、算法与理论". 高教出版社, 2020. [link]
• 林宙辰, 李欢, 方聪. "机器学习中的加速一阶优化算法". 机械工业出版社, 2021. [link]
• Jorge Nocedal, Stephen J. Wright. "Numerical optimization". Springer, 2006. [link]
• Yurii Nesterov. "Lectures on convex optimization". Springer, 2018. [link]
• Ralph Tyrell Rockafellar. "Convex Analysis". Princeton University Press, 1997. [link]

Courseware (subject to changes):

• Feb. 28: Course Overview, Optimization for Machine Learning, Review of Linear Algebra. [pdf]
• Mar. 06: Matrix Calculus, Topology, Convergence Rates. [pdf]
• Mar. 13: Convex Set, Convex Function, Optimal Condition. [pdf]
• Mar. 20: Subgradient, Subdifferential. [pdf]
• Mar. 27: More Optimal Conditions, Regularity Conditions. [pdf]
• Apr. 03: Second-Order Characterization, Examples and Applications. [pdf]
• Apr. 10: Black-Box Model, Gradient Descent Methods, Polyak–Łojasiewicz Condition. [pdf]
• Apr. 17: Line Search, Barzilai–Borwein Step Size, Parameter-Free Methods. [pdf]
• Apr. 24: Polyak's Heavy Ball Method, Nesterov's Acceleration. [pdf]