

The products of this spatial and temporal coherence are synergistic intersensory interactions within the central nervous system, interactions that are presumed to enhance the salience of the initiating event. Sensory systems have evolved to work in concert, and normally, different sensory cues that originate from the same event are concordant in both space and time.

In many circumstances, events are more readily perceived, have less ambiguity, and elicit a response far more rapidly when signaled by the coordinated action of multiple sensory modalities.

This “parallel” processing of multiple cues not only increases the probability of detecting a given stimulus but, because the information carried along each sensory channel reflects a different feature of that stimulus, it also increases the likelihood of its accurate identification. Humans possess an impressive array of specialized sensory systems that allow them to monitor simultaneously a host of environmental cues. Audible and visible speech are complementary in that one source of information is more informative when the other source is less so. Speech perception is a prototypical situation in which information from the face and voice is seamlessly processed to impose meaning in face-to-face communication. Perceptual as well as behavioral processes are influenced by simultaneous inputs from several senses. A more thorough understanding of the underlying causes of expert breakdown in novel interpreting environments could indeed lead to improved interface designs. Adaptive expertise, on the other hand, looks at the task from a variety of perspectives, is not wedded to a single definition of the problem, and is ready to explore entirely new approaches to getting the job done. Routine expertise, impressive as it can be, is derailed the moment the task environment or the task sequence changes. Research in the field of expertise has highlighted the importance of creativity and innovative approaches to the development of true high-level performance, such as simultaneous interpreting. One of the reasons is that we do not yet understand how novices and experts process multiple sources of information in a media rich environment.

Grafted onto traditional tasks or work processes, however, they have often not met with unconditional approval on the part of users. The development of technological support systems to facilitate communication in a multilingual environment has certainly been beneficial in terms of making more information available to the multilingual user. Comprehension in a foreign language relies heavily on redundancy and active discourse construction to offset the difficulties inherent in listening in another language. This usually leads to a feeling of alienation, as well as to the need to communicate with the help of multiple media, in itself not always an easy task and usually one requiring considerable cognitive resources. Distant communication across languages and cultures in a virtual space adds yet another layer of complexity and a new challenge, that of not being in the same place at the same time. Even expert communicators, such as conference interpreters, are not immune to constraints. Listeners as comprehenders in this situation operate under a variety of constraints such as less than adequate proficiency in the language/culture of the speaker, which may increase the need to integrate information from several sources as information from any one individual source may be inadequate for successful comprehension. The value of multiple sources of information in communication is even more apparent in a bilingual or multilingual setting. For example, information from the face improves speech intelligibility of the message and visible body language complements auditory information perceived from the same source, provided the information is time-aligned and not contradictory. Research has convincingly demonstrated that comprehension and communication in a monolingual setting usually succeed when listeners integrate information from several sources in an optimal way. In addition to standard desktop computers, these advanced interfaces are being implemented in telephones, mobile phones, and small handheld devices. The burgeoning developments in technology make it feasible to present information at a distance not only by text but also by speech, computer-animated agents, gesture, and even by touch. Feasibility and success of multilingual communication depend largely on the competence of the speakers and listeners on the one hand and, increasingly, the availability of human-machine interfaces that can facilitate rapid access to information from a variety of sources – auditory, visual, tactile – on the other.
