This is a graduate-level course designed to equip students with both fundamental and advanced optimization techniques and methods relevant to large-scale statistics and machine learning. We will begin with a concise review of the fundamentals of convex optimization, then progress to explore significant developments in first-order optimization methods across convex, nonconvex, stochastic, and distributed settings. Upon completing the course, students are expected to be capable of handling optimization-related challenges they encounter in statistics and machine learning research. This includes appropriately formulating an optimization problem, selecting or developing an efficient optimization algorithm for it, and analyzing the algorithm, based on structural properties such as convexity, smoothness, and sparsity, as well as specific settings such as online, distributed, and memory-limited contexts. Prerequisites: Fluency with reasoning and analysis using linear algebra and probability is required (MATH 309 and Math/SDS 493 or Math/SDS 3211). Students are expected to be familiar with the basics of at least one computing platform/ programming language, such as Matlab, Julia, Python, and R. Students should learn by themselves the basics of the (very user-friendly) convex optimization interpreter cvx (http://cvxr.com/cvx/) in the Matlab environment. cvx is also available in the Julia and Python environments.
Course Attributes:
