Early language abilities (e.g., vocabulary skills and oral language knowledge during preschool) are an important predictor of children’s academic success in subsequent school years. Personalized education technologies capable of delivering adaptive interventions could play an important role in supplementing teachers and parents to address the needs of diverse young learners at a critical time of school readiness. Social robots are a particularly interesting intervention because they can engage children in educational activities as peer-like companions to support how young children socially interact and learn in the real world with others. Social robots can also offer unique opportunities of guided, personalized and controlled social interaction during the delivery of a desired curriculum.

Intellectual Merit

The technical research goal was to advance the state-of-art of social robotics with respect to their ability to successfully personalize to individual users during social interaction over multiple encounters during a challenging real-world context and task.  This research project developed and evaluated a fully autonomous social robot to provide personalized education support for preschool vocabulary and oral language skills. The robot interacted with each child in a dialogic storytelling task; it asked each child questions to actively involve him or her in the story while attentively listening and responded to the child’s contributions via its own verbal and non-verbal cues. During each session, the robot learned to personalize the story content it told to each child to promote his or her sustained engagement and boost learning outcomes. Video of each session was analyzed to automatically assess children’s engagement from their expressive cues and electrodermal activity data. Children’s speech samples were also analyzed to assess their lexical and syntax skills. For this purpose, the research project developed and advanced three core technologies: 1) an automatic story analysis tool based on the Index of Productive Syntax (IPSyn) to evaluate the grammatical complexity of children's spontaneous language samples for noun phrase, verb phrase, and sentence structure scales; 2) new computational models to support and generate dynamic, non-verbal communication behaviors such as back channeling and attentive listening cues; and 3) a novel affective personalization algorithm based on reinforcement learning that enabled the robot to adapt its story selection policy from a library of e-books to individual children based on their verbal and expressive behaviors to modulate children’s engagement and enhance their learning gains over repeated encounters.

We recruited 67 English language learners (ELL) and bilingual children between the ages of 4–6 years from local public schools in the Greater Boston area to participate in a 3-month study to evaluate the personalized tutor-companion system. Over the course of the deployment, the robot learned a unique storytelling policy to deliver a personalized story curriculum for each child in the Personalized group. We compared their engagement and learning outcomes to a Non-personalized group with a fixed curriculum robot and to a Baseline group that had no robot intervention. In the Personalization condition, our results show that the policy successfully personalized to each child to boost their engagement and outcomes with respect to learning and retaining more target words (three times greater than baseline) as well as using more target syntax structures as compared to children in the other groups. We also found that children demonstrated higher attention and engagement based on physiological data and non-verbal cues such as body pose over the other groups.

Broader Impact

The broader impact goal of this work is to advance the development and evaluation of personalized AI technologies that can augment the preschool setting and promote children’s language and literacy skill development in an engaging and effective way. We engaged parenting groups and educators in the design of the story content and the robot’s dialogic storytelling behavior. Undergraduate and graduate research assistants were trained in the multidisciplinary aspects of this project: AI algorithms and methods, the design and assessment of education technologies, and early literacy and language development.  Ultimately, we hope work can inspire new tools and practices for early pre-literacy and language education (as well as other domains such as STEM) in the home, classroom, and beyond. The ultimate goal is to develop personalized AI technologies that can help to foster the development, learning, and promotion of academic achievement as well as the wellbeing of children.

Last Modified: 11/27/2018
Modified by: Cynthia Breazeal