Register for an account

X

Enter your name and email address below.

Your email address is used to log in and will not be shared or sold. Read our privacy policy.

X

Website access code

Enter your access code into the form field below.

If you are a Zinio, Nook, Kindle, Apple, or Google Play subscriber, you can enter your website access code to gain subscriber access. Your website access code is located in the upper right corner of the Table of Contents page of your digital edition.

Technology

Computers Can Grade Essays As Well As People Can

80beatsBy Veronique GreenwoodApril 23, 2012 10:49 PM

Newsletter

Sign up for our email newsletter for the latest science news

scores2.jpg

RS is the average of scores given by two human readers; all the others are computer programs.

To anyone who's ever written an essay for a standardized test---be it the SAT, the ACT, the GMAT, or others---it should come as no surprise that getting a high-scoring essay is a matter of following a formula. The SAT is not the time to show off your lyrical ability or demonstrate your awareness of the nuances of morality: when the prompt is "Is it better to have loved and lost than never loved at all?" it's hard to argue "It depends" in 25 minutes. Just take a stance, come up with two supporting examples, and hammer that baby out. Turns out, though, that standardized test essays are so formulaic that test-scoring companies can use algorithms to grade them. And before you get worried about machines giving you a bad score because they've never taken an English class, said algorithms give the essays the same scores as human graders do, according to a large study that compared nine such programs with humans readers

. The team used more than 20,000 essays on eight prompts, and you can see in the figure to the right, the mean scores found by the programs and the people were so close that they appear as one line on a chart of the results. Now, that doesn't mean that the programs can tell when a writer has got the facts wrong or has inserted grammatical gibberish, or even whether they've plagiarized someone else's work, though there are programs that can do that. Les Perelman, a writing teacher at MIT, takes pleasure in tricking the program used by Educational Testing Services

, the company that produces the SAT, on all these points. But this level of trickery seems unlikely on the part of the average standardized test taker, sweating out a five-paragraph essay in less than half an hour. And as for human essay graders, they have only a couple minutes to come up with a score. When you're under that kind of pressure, machine-like behavior is the best you can hope for. Image courtesy of Shermis et al.

    3 Free Articles Left

    Want it all? Get unlimited access when you subscribe.

    Subscribe

    Already a subscriber? Register or Log In

    Want unlimited access?

    Subscribe today and save 70%

    Subscribe

    Already a subscriber? Register or Log In