Imagination Machines: From Cognitive Neuroscience to Human-Level Synthetic Intelligence

28. April 2011


Mental imagery occurs when perceptual information is accessed from memory, giving rise to the experience of ‘seeing with mind’s eye’ (visual imagery), ‘hearing with mind’s ear’ (auditory imagery), and ‘acting with mind’s hands’ (motor imagery). This capability underlies almost all brain functions, ranging from action planning to language and creative thinking. Understanding the imagery mechanisms will offer a fundamentally new computational foundation for constructing human-level synthetic intelligence beyond the AI approaches so far. In this talk we first review the experimental and theoretical studies of mental imagery in cognitive neuroscience, and discuss some of representational and computational ingredients that seem essential for producing mental imagery. We then propose to use these findings to develop “imagination machines” that simulate multimodal grounded cognition, such as perceptual symbol systems a la Barsalou. As a concrete example of this direction of research, we present a random graph-based self-organizing network architecture for cognitive memory, and demonstrate how this network learns probabilistically to perform pattern-completion-based imagery in the domains of language, music, and movies. We also share the challenges to develop up-scale imagination machines and their potential as a computational tool for research on cognitive dynamics, cognitive development, and neurocognitive linguistics as well as for cognitive robotics and human-level AI.


Byoung-Tak Zhang is Professor of Computer Science and Cognitive Science at Seoul National University (SNU), Seoul, Korea and directs the Biointelligence Laboratory ( He is Director of the SNU Interdisciplinary Program in Cognitive Science and also affiliated with Brain Science and Bioinformatics Programs. He is Founding President of the Korean Society for Computational Intelligence (KSCI) and serves on the editorial boards of Journal of Cognitive Science, Applied Intelligence, Biosystems, and IEEE Transactions on Evolutionary Computation. Byoung-Tak Zhang received his PhD in Computer Science from University of Bonn in 1992 and has worked for the German National Research Center for Information Technology (GMD) in Sankt Augustin from 1992-1995. For 2003-2004 he has been Visiting Scientist at MIT CSAIL (Computer Science and Artificial Intelligence Lab) and BCS (Brain and Cognitive Sciences Department). Prof. Zhang is currently visiting CITEC and SFB 673 at Bielefeld University on his sabbatical leave from Seoul National University.