Skip to contents

National Science Foundation Logo

Search KDI Site

Banner: Multi-Disciplinary Research at NSF: Accomplishments of the KDI Initiative
 KDI Home    Contact Us   

Image: People Button

Image: Ideas Button

Image: Tools Buttong

 About KDI

 Behind the Scenes

 Taking Stock

 Links and Resources

  

Tools

  Quick Links

IEA Web site

Investigators Test Automatic Essay Scoring in Tutoring Systems

Researchers at New Mexico State University have been testing the patented Intelligent Essay Assessor (IEA) software for its applicability to automated training systems. Their results have demonstrated the promise of IEA as a tool for helping students master course content and improve their writing skills.

Image of IEA scoring exampleThe IEA is a technique for scoring responses to essay questions that is based on a theory called Latent Semantic Analysis (LSA). It assesses the quality of an essay by characterizing the meaning of the words contained in the essay and comparing that characterization with the conceptual relevance and content of essays of known quality. LSA, the theory behind the IEA, and the IEA technology have been presented at a number of conferences and have inspired much interest.

IEA scores on essays are very much like those of human graders. They have been found to agree with those of human experts as much as those experts agree with each other. Other essay grading programs have been developed that assess grammar, spelling, and punctuation rather than content. The IEA software can determine the student's mastery of the material and give students constructive feedback. Unlike other essay grading programs, the IEA software can help students learn.

Peter W. Foltz, one of the professors who developed the IEA, and Adrienne Y. Lee, both of NMSU, were principal investigators on a recent KDI grant. They looked at the potential for using the IEA software as a tool not only for assessing performance but also for helping students learn from automated training systems.

Foltz and Lee developed training systems for use in a series of psychology courses. They evaluated the merits of the IEA software in this context by comparing the relative performance of students who responded to multiple-choice, essay, and reflection questions. The students who responded to the essay questions received feedback from the IEA software and were allowed to rewrite and resubmit their essay as many times as they wanted. Those students mastered the material best and experienced the greatest improvement in their writing skills.

These results are very promising. According to Foltz, "They show that incorporating writing into automated training systems is both feasible and can result in improved learning over traditional assessment methods. Automated writing assessment has the potential to be incorporated into almost any area of training in which a student must reason about a particular content area. This can include school topics, such as biology, physics, and history, as well as such areas as corporate or military training."

Additional NSF-funded research work is under way to refine the application of LSA to training systems and to test additional topics in science in which writing can be incorporated into automated training systems.

 

Back to Top of Page


People | Ideas | Tools
About KDI | Behind the Scenes | Taking Stock | Links and Resources

KDI Home | Contact Us | Site Map
NSF Home | CISE Home | Privacy Statement | Policies | Accessibility

National Science Foundation: Celebrating 50 Years Logo The National Science Foundation
4201 Wilson Boulevard, Arlington, Virginia 22230, USA
Tel: 703-292-5111, FIRS: 800-877-8339 | TDD: 703-292-5090