N.S Sreekanth, Supriya N Pal, Meera Thomas, Ahmad Haassan, N.K Narayanan

In this paper we present an approach for design and implementation of multimodal interface for human computer interaction. The paper discuss how multiple input modalities may be combined to construct more natural user interfaces. An agent based multimodal architecture is discussed with help of a simulated application. An application is simulated for performing operation on graph data structure to illustrate the concept. The user can interact with the system through speech, gestures and text based input. The gestures can be a pen gesture input through graphic tablets or it can be human hand-finger gesture captured through camera. Here we use separate software agent to process the different input modalities .A MONITOR agents is used to synchronize various inputs from the user and act according to it. Users response will be mapped to system vocabulary space with help of a semantic analyzer and results will be passed to MONITOR for further processing.