학과 세미나 및 콜로퀴엄

구분 학과 세미나/콜로퀴엄
분류 응용 및 계산수학 세미나
제목 A Nonlocal Gradient for High-Dimensional Black-Box Optimization in Scientific Applications
Abstract In this talk, we consider the problem of minimizing multi-modal loss functions with a large number of local optima. Since the local gradient points to the direction of the steepest slope in an infinitesimal neighborhood, an optimizer guided by the local gradient is often trapped in a local minimum. To address this issue, we develop a novel nonlocal gradient to skip small local minima by capturing major structures of the loss's landscape in black-box optimization. The nonlocal gradient is defined by a directional Gaussian smoothing (DGS) approach. The key idea is to conducts 1D long-range exploration with a large smoothing radius along orthogonal directions, each of which defines a nonlocal directional derivative as a 1D integral. Such long-range exploration enables the nonlocal gradient to skip small local minima. We use the Gauss-Hermite quadrature rule to approximate the d 1D integrals to obtain an accurate estimator. We also provide theoretical analysis on the convergence of the method on nonconvex landscape. In this work, we investigate the scenario where the objective function is composed of a convex function, perturbed by a highly oscillating, deterministic noise. We provide a convergence theory under which the iterates converge to a tightened neighborhood of the solution, whose size is characterized by the noise frequency. Furthermore, if the noise level decays to zero when approaching global minimum, we prove that the DGS optimization converges to the exact global minimum with linear rates, similarly to standard gradient-based method in optimizing convex functions. We complement our theoretical analysis with numerical experiments to illustrate the performance of this approach.
일시 2023-05-19 (Fri) / 11:00 ~ 12:00 ** 날짜에 유의하세요. **
장소 Zoom (https://kaist.zoom.us/j/89977002928)
강연언어 영어
강연자성명 Guannan Zhang
강연자소속 Oak Ridge National Lab
강연자홈페이지
기타정보
초청인 신연종
URL https://sites.google.com/view/kaist-acm
담당자 박준서
연락처