Robo-graders
Robo-graders,
or software that grades essays have been there for some time. But how
good/useful is an algorithm to assess the quality of writing?
Les Perelman, a critic, says these algorithms don’t
understand the meaning (obviously) and just use certain metrics like “length
and the presence of pretentious language”:
“The fallacy underlying this approach is confusing association
with causation. A person makes the observation that many smart college
professors wear tweed jackets and then believes that if she wears a tweed
jacket, she will be a smart college professor.”
Further,
Perelman says that once kids know (or guess) how the algorithm works, they will
write to suit the algorithm without worrying about content! A couple of
students at Harvard demonstrated just that by writing an application that
generates gibberish and yet gets rated highly by the algorithm.
Then
again, as Chaitanya Ramineni, an ETS (Educational Testing Service) researcher,
says, “human scoring suffers from flaws” too. Remember the dread with which
most parents view the marking of board exams?
Another
problem is rooted in the fact that any grading algorithm would have to be
created by analyzing the qualities of good essays (as graded by humans). But
this then means that such an algorithm would inevitably “punish creativity”.
Let this article explain how:
“The computer judges “topical analysis” by favoring “similarity of
the essay's vocabulary to other previously scored essays in the top score
category.”
Ergo,
anything different would, by definition, be considered worse!
But
others do see benefits to algorithmic grading. Andrew Klobucar says that:
“His students are willing to revise their essays, even multiple
times, when their work is being reviewed by a computer and not by a human
teacher.”
Klobucar
feels that happens because a human reviewer feels “corrective, even punitive”
whereas an algorithm feels individualized yet not “personal”. Turns out there
is even a term for this effect of technology on human behavior, the
“disinhibition effect” (Outside education, it’s also seen in how honestly
patients answer questions to software v/s doctors. Guess the eponymous Dr.House
was right in asserting that patients always lie!). Amusingly, other researchers
feel part of the reason why students are willing to revise and resubmit to a
computer is that “the process of improving their writing appeared to take on a
game-like quality, boosting their motivation to get better”!
Maybe
the robo-grader has a place in the education system after all, just not to give
grades.
Comments
Post a Comment