Understanding how Neural Networks learn patterns from data (Dec. 4, Adit Radhakrishnan)

Title: Understanding how Neural Networks learn patterns from data

Speaker: Adit Radhakrishnan, Postdoctoral Fellow in the School of Engineering and Applied Sciences at Harvard

Abstract: Understanding how neural networks learn features, or relevant patterns in data, for prediction is necessary for their reliable use in technological and scientific applications. We present a unifying mechanism that characterizes feature learning in neural network architectures. Namely, we show that features learned by neural networks are captured by a mathematical operator known as the average gradient outer product (AGOP). We demonstrate that the AGOP captures neural features such as edge detectors in convolutional networks and groups of related tokens in language models. Moreover, we demonstrate that AGOP, which is backpropagation-free, enables feature learning in general machine learning models that apriori could not identify task-specific features. We apply our findings to the biomedical domain by developing new, computationally-efficient, and effective models to screen synthetically lethal gene pairs for cancer treatment. Overall, this line of work provides new tools for pinpointing the features used by neural networks for prediction and how such tools can be leveraged to develop novel, interpretable, and effective models for use in scientific applications.

Bio: Adit is currently the George F. Carrier Postdoctoral Fellow in the School of Engineering and Applied Sciences at Harvard and an affiliate with the Broad Institute of MIT and Harvard. He completed his Ph.D. in electrical engineering and computer science (EECS) at MIT advised by Caroline Uhler and was a Ph.D. fellow at the Eric and Wendy Schmidt Center at the Broad Institute. He received his M.Eng. in EECS and his Bachelor’s of Science in Math and EECS from MIT. His research focuses on advancing theoretical foundations of machine learning and developing new methods for tackling biomedical problems.