Stochastic finite-sum optimization problems are ubiquitous in many areas such as machine learning, and stochastic optimization algorithms to solve these finite-sum problems are actively studied in the literature. However, there is a major gap between practice and theory: practical algorithms shuffle and iterate through component indices, while most theoretical analyses of these algorithms assume uniformly sampling the indices. In this talk, we talk about recent research efforts to close this theory-practice gap. We will discuss recent developments in the theoretical convergence analysis of shuffling-based optimization methods. We will first consider minimization algorithms, mainly focusing on stochastic gradient descent (SGD) with shuffling; we will then briefly talk about some new progress on minimax optimization methods.
Deep generative models (DGM) have been an intersection between the probabilistic modeling and the machine learning communities. Particularly, DGM has impacted the field by introducing VAE, GAN, Flow, and recently Diffusion models with its capability to learn the data density of datasets. While there are many model variations in DGM, there are also common fundamental theories, assumptions and limitations to study from the theoretic perspectives. This seminar provides such general and fundamental challenges in DGMs, and later we particularly focus on the key developments in diffusion models and their mathematical properties in detail.