Students taking their state-mandated tests in Texas this week are being used as guinea pigs for a new artificial intelligence-powered scoring system that will replace the majority of human graders in the region.
The Texas Tribune An “automated scoring engine” that uses natural language processing — the technology that enables chatbots like OpenAI’s ChatGPT to interact with and understand users — is being reported by the Texas Education Agency (TEA). is being introduced to rank open-ended questions in the state by Texas Assessment of Academic Readiness (STAAR) exams. The agency expects the system to save $15-20 million annually by reducing the need for temporary human scorers, with plans to hire fewer than 2,000 graders this year, compared to the 6,000 needed in 2023.
“We wanted to retain as many structured open-ended responses as possible, but they take an incredibly long time to score.”
The STAAR exams, which test students on their understanding of core curriculum between third and eighth grades, were redesigned last year to include fewer multiple-choice questions. It now includes seven times as many open-ended questions, with TEA Director of Student Assessment Jose Rios saying the agency “wants to have as many structured open-ended answers as we can, but they take an incredible amount of time to score. “
According to a slideshow hosted on the TEA’s website, the new scoring system was trained using 3,000 test responses that had already received two rounds of human rating. Some safety nets are also implemented – a quarter of all results the computer classifies will be retrieved by humans, for example, answers that confuse the AI system (including slang or non-English answers). usage).
While the TEA is hopeful that AI will enable it to save buckets of cash, some educators aren’t keen to see it implemented. Lori Rapp, superintendent of the Lewisville Independent School District, said her district saw a “tremendous increase” in constructive responses receiving zero scores when it used the automatic grading system on a limited basis in December 2023. “At this time, we are unable to determine if there is something wrong with the test question or if it is the new automated scoring system,” Rapp said.
AI essay scoring engines are nothing new. A 2019 report Motherboard It turns out they’re being used in at least 21 states with varying degrees of success, though the TEA seems determined to avoid that reputation. The small print on TEA’s slideshow also emphasizes that its new scoring engine is a closed system that is inherently different from AI, in that “AI is a computer that adapts to progressive learning algorithms.” , allowing data to be programmed and essentially teach itself.”
Trying to draw a line between them isn’t surprising—there’s no shortage of teachers frustrated online about how creative AI services are being used to cheat on assignments and homework. Students graded by this new scoring system may find it difficult to accept how they believe the principles of “for you and not for me” are being applied here.