Computers to Grade English Papers in the UK

Status
Not open for further replies.

TruePaige

Diamond Member
Oct 22, 2006
9,874
2
0
http://www.guardian.co.uk/educ...to-mark-english-essays

The owner of one of England's three major exam boards is to introduce artificial intelligence-based automated marking of English exam essays in the UK from next month.

Pearson, the American-based parent company of Edexcel, is to use computers to "read" and assess essays for international English tests in a move that has fuelled speculation that GCSEs and A-levels will be next.

All three exam boards are now investing heavily in e-assessment but none has yet perfected a form of marking essays using computers ? or "robots" ? that it is willing to use in mainstream exams. Academics and leaders in the teaching profession said that using machines to mark papers would create a "disaster waiting to happen".

The Times Educational Supplement (TES) reports today that the Pearson Test of English Academic, an English-­language exam, will launch on 26 October. It includes essay questions and will be used in 20 countries, including the UK, to rate applicants' English skills before they are admitted to university.

Computers have been programmed to scan the papers, recognise the possible right responses and tot up the marks. Pearson claims this will be more accurate than human marking.

John Bangs, head of education at the National Union of Teachers, said that computers could be useful in many areas of assessment but cautioned against their use in English exams: "I'm very concerned that it would constrain the nature of the questions being asked. You won't pick up nuances by machine and it will trigger a trend to answering narrower questions. It could be a disaster waiting to happen."

A Pearson spokesman told the TES that its system produced the accuracy of human markers while eliminating human elements such as tiredness and subjectivity.

Other exam boards said the adoption of computers to mark beyond their current use in multiple choice tests was inevitable. Tim Oates, director of research for Cambridge Assessment, which owns the exam board OCR, said: "It's extremely unlikely that automated systems will not be deployed extensively in educational assessment. The uncertainty is 'when' not 'if'. But all systems need to meet exacting quality criteria and should definitely not be adopted just to make life easier for testing organisations.

"Some approaches look like technology in search of a test, rather than assessment designed to accurately report attainment."

An Edexcel spokesperson said that the board was not planning to use automated marking in mainstream exams such as A-levels and GCSEs. She said that previous trials of the technology in GCSE essay questions had not been expanded.

Reportedly, it is as accurate as human markers. Maybe this is a suitable use of technology for basic assignments, but can we really expect this system to advance enough to handle the subtle nuances of our language with sufficient accuracy?

This quote from above summed it up for me.

"I'm very concerned that it would constrain the nature of the questions being asked. You won't pick up nuances by machine and it will trigger a trend to answering narrower questions. It could be a disaster waiting to happen."
 

SirStev0

Lifer
Nov 13, 2003
10,449
6
81
Are these the type of exams like they give in the US to make sure "teachers are teaching" ... you know the ones that we pad by teaching the test most of the year to make it look like our education isn't as shitty as it is? Or are they professional board test or college entrance tests?
 

TruePaige

Diamond Member
Oct 22, 2006
9,874
2
0
Originally posted by: SirStev0
Are these the type of exams like they give in the US to make sure "teachers are teaching" ... you know the ones that we pad by teaching the test most of the year to make it look like our education isn't as shitty as it is? Or are they professional board test or college entrance tests?

Doesn't sound like too many major exams are going this way yet but...

"The Times Educational Supplement (TES) reports today that the Pearson Test of English Academic, an English-­language exam, will launch on 26 October. It includes essay questions and will be used in 20 countries, including the UK, to rate applicants' English skills before they are admitted to university.

Computers have been programmed to scan the papers, recognise the possible right responses and tot up the marks. Pearson claims this will be more accurate than human marking. "

Does pertain to university admittance.
 

SirStev0

Lifer
Nov 13, 2003
10,449
6
81
Originally posted by: TruePaige
Originally posted by: SirStev0
Are these the type of exams like they give in the US to make sure "teachers are teaching" ... you know the ones that we pad by teaching the test most of the year to make it look like our education isn't as shitty as it is? Or are they professional board test or college entrance tests?

Doesn't sound like too many major exams are going this way yet but...

"The Times Educational Supplement (TES) reports today that the Pearson Test of English Academic, an English-­language exam, will launch on 26 October. It includes essay questions and will be used in 20 countries, including the UK, to rate applicants' English skills before they are admitted to university.

Computers have been programmed to scan the papers, recognise the possible right responses and tot up the marks. Pearson claims this will be more accurate than human marking. "

Does pertain to university admittance.

Interesting.

It sounds like they are attempting to make the exams as objective as possible since it will all be judged on the same criteria by the same reviewer, something that a human reviewer would never be able to do due to the massiveness.

I can't speak for now, as I think they are phasing it out, but when I took the MCAT 3 years ago, the written section was graded by two different people and the two scores were put together to make one total score out of (I believe) 14 points. Basically they hoped that two people who are trying to be as subjective-less as possible put together could be objective. Not exactly the best system if you ask me, especially when you think of how many pairs were necessary to deal with the hundreds of tests needed to be graded.

I can't complain though, I absolutely rocked it and the written turned out to be my best section.
 

Fox5

Diamond Member
Jan 31, 2005
5,957
7
81
Most human reviewers are trained to mechanically grade these types of essays anyway. There's a rigid scoring sheet they use (and are told what to look for) and they're not supposed to deviate.

Still, I hope the computer grading is tested for accuracy before its used.
 

Cogman

Lifer
Sep 19, 2000
10,284
138
106
Of course it would be as accurate as a human grader. I feel have long since felt that psuedo-random number generators had come a long way.
 

Kalmah

Diamond Member
Oct 2, 2003
3,692
1
76
Wait.. the computer can grade essays? So I could write 2 pages of gibberish as long as there are no grammar mistakes and get a 100% ?
 

Kadarin

Lifer
Nov 23, 2001
44,296
16
81
The example cited from Slashdot was a quote from Groucho Marx: "Time flies like the wind, fruit flies like a banana." I wonder what a computer would make of that...
 

SirStev0

Lifer
Nov 13, 2003
10,449
6
81
Originally posted by: Kalmah
Wait.. the computer can grade essays? So I could write 2 pages of gibberish as long as there are no grammar mistakes and get a 100% ?

No, come on now.
 

Fox5

Diamond Member
Jan 31, 2005
5,957
7
81
Originally posted by: SirStev0
Originally posted by: Kalmah
Wait.. the computer can grade essays? So I could write 2 pages of gibberish as long as there are no grammar mistakes and get a 100% ?

No, come on now.

Standardized essays aren't graded on facts or sound arguments (generally) anyway.
 

Elias824

Golden Member
Mar 13, 2007
1,100
0
76
hrm maybe if students can put in subtle code they can force it to give them an Uber A+
 
Status
Not open for further replies.