Tai-Sing Lee Professor Website ORCiD Office 115 Mellon Institute Email taislee@cmu.edu Phone (412) 268-1060 Department Computer Science Department Administrative Support Person Marcella Baker Research Interests Artificial Intelligence Computational Neuroscience Advisees Tianqin Li CSD Courses Taught 15686 - Spring, 2024 15386 - Spring, 2024 15387 - Fall, 2024 Research in my laboratory seeks to elucidate the computational principles and neural mechanisms underlying visual perception. We use computational, mathematical and neurophysiological experimental techniques to study the biological visual system with a view to discovering the secrets of neural computation, advancing computer vision technology and restoring vision to the visually impaired. Statistical approaches to understand neural codes and perceptual computation. Our perceptual systems are shaped by the natural environment. Our investigation of the neural codes and computational circuits in the brain thus often begins with a rigorous study of the statistical regularities in our visual environment and a parallel computational study to understand how these statistical regularities or probability distributions can support effective perceptual computation in a Bayesian inference framework. We then test theoretical predictions from these computational works on neural coding and computational circuits by examining the activities of hundreds of individual neurons recorded using multi-electrode array techniques. We have pursued this agenda to elucidate how statistical structures in 3D surfaces are encoded in neuronal tunings and connectivity, and how they can be used to facilitate 3D perceptual inference. We have developed new machine learning techniques to perform large-scale neural data analysis to study these and other issues, particularly on how elementary codes can be used to flexibly compose higher order concepts in the visual hierarchy. By elucidating the principles of neural coding and neural computation, we seek to discover new computational algorithms for perceptual computation to improve computer vision systems and to develop approaches to generate mental visual images in the visually impaired subjects by stimulating the appropriate neurons in their brains. Learning, adaptation and development in neural systems. Learning and adaptation are what make biological systems so much more robust and powerful than current man-made vision systems. Our research explores the basic mechanisms and principles underlying adaptation and learning in the visual system at different time scales and at the level of neurons and of neural systems. We have studied theoretically how neurons adapt dynamically to the statistical context of the visual stimuli. We have determined biophysical features in spiking neurons and identified statistical features in natural stimuli underlying neuronal adaptation. We have demonstrated experimentally that the neural machinery of perceptual processing is very flexible and subject to modification by behavioral experience. These works have lead to a new perspective on the functional role of the early visual cortex in vision. We are exploring how these design principles can be used to develop new learning algorithms for learning hierarchical codes and for building the computational structures in our visual systems. Publications Journal Article Large-scale calcium imaging reveals a systematic V4 map for encoding natural scenes 2024 • Nature Communications • 15(1): Wang T, Lee TS, Yao H, Hong J, Li Y, Jiang H, Andolina IM, Tang S Preprint A large calcium-imaging dataset reveals a systematic V4 organization for natural scenes 2023 Wang T, Yao H, Lee TS, Hong J, Li Y, Jiang H, Andolina IM, Tang S Conference Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity 2023 • Advances in Neural Information Processing Systems Li T, Wen Z, Li Y, Lee TS Conference PROTOTYPE MEMORY AND ATTENTION MECHANISMS FOR FEW SHOT IMAGE GENERATION 2022 • ICLR 2022 - 10th International Conference on Learning Representations Li T, Li Z, Luo A, Rockwell H, Farimani AB, Lee TS Journal Article Complexity and diversity in sparse code priors improve receptive field characterization of Macaque V1 neurons 2021 • PLoS Computational Biology • 17(10): Wu Z, Rockwell H, Zhang Y, Tang S, Lee TS
Journal Article Large-scale calcium imaging reveals a systematic V4 map for encoding natural scenes 2024 • Nature Communications • 15(1): Wang T, Lee TS, Yao H, Hong J, Li Y, Jiang H, Andolina IM, Tang S
Preprint A large calcium-imaging dataset reveals a systematic V4 organization for natural scenes 2023 Wang T, Yao H, Lee TS, Hong J, Li Y, Jiang H, Andolina IM, Tang S
Conference Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity 2023 • Advances in Neural Information Processing Systems Li T, Wen Z, Li Y, Lee TS
Conference PROTOTYPE MEMORY AND ATTENTION MECHANISMS FOR FEW SHOT IMAGE GENERATION 2022 • ICLR 2022 - 10th International Conference on Learning Representations Li T, Li Z, Luo A, Rockwell H, Farimani AB, Lee TS
Journal Article Complexity and diversity in sparse code priors improve receptive field characterization of Macaque V1 neurons 2021 • PLoS Computational Biology • 17(10): Wu Z, Rockwell H, Zhang Y, Tang S, Lee TS