This final talk illustrates the scope and utility of the auxiliary-space viewpoint through several important classes of numerical methods. The framework applies to a broad range of advanced iterative methods, including subspace correction methods, Hiptmair--Xu preconditioners, saddle point solvers, and iterative substructuring methods. Through these applications, we show how apparently different methods can be understood within a common structure: each may be interpreted as an elementary iteration on a suitably enlarged space. This perspective clarifies relationships among existing algorithms and suggests new ways to design efficient solvers. We conclude by discussing how this viewpoint may inform the development of numerical methods for problems arising in machine learning, where complexity is often shifted from the optimization procedure to the underlying representation.

This second talk develops the theoretical framework behind the interpretation of advanced iterative methods as elementary iterations on larger spaces. The main tool is an auxiliary-space framework, which recasts an iterative method for the original system as an equivalent, but more elementary, method for a suitably enlarged auxiliary system. Within this framework, many methods that appear sophisticated on the original problem can be understood as simple iterations applied to auxiliary variables. In particular, multigrid and domain decomposition methods can be viewed as Jacobi- or Gauss--Seidel-type iterations for appropriate expanded systems. This provides a rigorous way to connect advanced solvers with elementary iterative principles and clarifies the algebraic structure underlying these methods.

A central goal of scientific computing is to develop accurate and efficient solvers for scientific problems, and this goal is often pursued through sophisticated numerical methods. In modern machine learning, by contrast, the basic optimization procedure is often comparatively simple, typically gradient descent and its variants, while much of the complexity is shifted to larger models. This first talk introduces the main motivation of the talk series: to examine advanced iterative methods in scientific computing from a viewpoint inspired by this contrast. We begin with basic examples and preliminary concepts, including classical iterative methods, convergence of stationary iterations, and elementary schemes such as Jacobi, Gauss--Seidel, and Richardson iterations. These examples provide the foundation for the auxiliary-space perspective developed in the subsequent talks.