Alfred Hero, University of Michigan. Learning to Benchmark: predicting performance limits from data

Department of Electrical and Computer Engineering ECE Guest Lecturer Series

Abstract: The objective of benchmark learning  is to use a training sample to learn about fundamental limits on performance of a classifier or other statistical inference procedure.  This meta-learning problem is a crucial component of data science and interpretable AI. Examples include sequential design of experiments, reinforcement learning and sensor management in the fields of statistics, machine learning and systems engineering, respectively. The challenge is learn about best achievable accuracy directly from the data sample without having to approximate and implement an optimal classifier algorithm. In this talk we will introduce a general information theoretic framework that yields benchmark learners having both linear computational complexity and linear sample complexity.  We will illustrate how this framework in the context of benchmarking image classification, autonomous navigation, and deep neural network performance. 

Alfred Hero III is the John H. Holland Distinguished University Professor of Electrical Engineering and Computer Science and the R. Jamison and Betty Williams Professor of Engineering at the University of Michigan, Ann Arbor. Read full bio online.

Refreshments will be provided.