April 27, 2018, Webb 1100

A central problem in machine learning is the following. Given data $(x_i,y_i)$ sampled randomly from an unknown probability distribution, find a model $P$ that fits the data well. The generalization error is defined as the expected value of $(y-P(x))^2$ with respect to the unknown probability distribution. We point out that this paradigm is not appropriate for deep learning. We then propose an alternative look at the problem, posing it as a pure function approximation problem. We will review briefly some basic approximation theory concepts in the case of shallow networks in our new paradigm, in particular, an explanation of how the training error can be driven to zero while the test error can be kept in control. We will then point out the extensions for deep networks, and if time permits, a connection with the problem of super-resolution.

Hrushikesh Mhaskar (b. 1956, Pune, India) did his undergraduate studies in Institute of Science, Nagpur, and received his first M. Sc. in mathematics from the Indian Institute of Technology in Mumbai in 1976. He received his Ph. D. in mathematics and M. S. in computer science from the Ohio State University, Columbus, in 1980. He then joined Cal. State L. A., and was promoted to full professor in 1990. After retirement in 2012, he was a visiting associate with California Institute of Technology until 2017, and occasionally served as a consultant for Qualcomm. Since 2012, he is also Research Professor at Claremont Graduate University. He has published more than 140 refereed articles in the area of approximation theory, potential theory, neural networks, wavelet analysis, and data processing. His book,“Weighted polynomial approximation”, was published in 1997 by World Scientific, and the book with Dr. D. V. Pai, “Fundamentals of Approximation Theory” was published by Narosa Publishers, CRC, and Alpha Science in 2000. He serves on the editorial boards of Applied and Computational Harmonic Analysis, Frontier Journal on mathematics of computation and data science, Journal of Approximation Theory, and Jaen Journal of Approximation. In addition, he was a co-editor of a special issue of Advances in Computational Mathematics on mathematical aspects of neural networks, two volumes of Journal of Approximation Theory, dedicated to the memory of G. G. Lorentz, as well as three edited collections of research articles: Wavelet Analysis and Applications, Narosa Publishers, 2001, Frontiers in interpolation and approximation, Chapman and Hall/CRC, 2006, and New Methods in Harmonic Analysis, Springer, 2017. He has held visiting positions, as well as given several invited lectures throughout North America, Europe, and Asia. He was awarded the Humboldt Fellowship for research in Germany four times. He was John von Neumann distinguished professor at Technical University of Munich in 2011, visiting associate at California Institute of Technology, 2012-2017. He is listed in Outstanding Young Men of America (1985) and Who’s Who in America’s Teachers (1994). His research was supported by the National Science Foundation and the U. S. Army Research Office, the Air Force Office of Scientific Research, the National Security Agency, and the Research and Development Laboratories.