Multi-modal Language
Education Technology
Language teaching Virtual Agents and Robots
Project Principal Investigator(s): Dinesh Babu J
Learning a language involves acquiring both speaking as well as writing skills. The objective is to build an automatic speaking and writing skill assessing engine, and augment the capability of social interaction as well. This will take the form of a Virtual Agent and later a robot as well. It is also envisioned that the interaction can happen in the wild, where learners can be recognized and learning can happen over multiple sessions without any intervention.
Technically, this would entail many sub-problems to come together: Behavior Analysis, Automatic Speech Recognition, Dialog Modeling, Text to Speech Synthesis and Behavior Synthesis. The project will focus on Behavior Analysis and Synthesis to create a simple task accomplishment. We will investigate behavior synthesis using a Machine Learning framework.
