The language of vision

The descriptions of surfaces, objects, and events computed by visual processes are not solely for use in the visual system but are also passed on to other brain centers to inform them about the surrounding visual world. Clearly, the description of the visual scene cannot be sent in its entirety, like a picture or a movie, or each brain module receiving the description would require some mini-visual system of its own to interpret it. Some very compressed, language-like version must be constructed that can be passed on in a format that other centers – memory, language, planning – can understand. If this is a “visual” language, what is its grammar? In a simple first pass, we see, among other things, differences in processing of visual “nouns” and visual “verbs”. The difference rests on the level of visual attention required to construct the descriptions for these two categories. The possibility of a visual language also raises the question of the acquisition of its grammar from the visual environment and the possibility that this acquisition process was borrowed and adapted for spoken language.