Introduction
Papers graded was a service popularized by Anthropic to provide automated essay scoring on student written assignments. By analyzing essay structure, style, vocabulary use, and grammar, the AI assistant Papa could provide essay scores that aimed to mimic those of human graders. The use of AI for high-stakes assessment also raised questions about potential bias, transparency, and the ability of algorithms to truly understand the complexities and nuances of open-ended writing. This article explores how Papa scoring worked, key issues and debates surrounding its use, and ongoing work towards more equitable and explainable automated scoring systems.
How Papa Scoring Worked
When a student submitted an essay through the Papers Graded portal, Papa would first analyze the text on several key dimensions used in human grading such as organization, style, mechanics (grammar, spelling, punctuation), and how well the response fulfilled the writing prompt or rubric. Based on natural language processing techniques, Papa could identify essay components like the introduction, body paragraphs, and conclusion and evaluate whether the essay flowed in a logical sequence. At the word level, Papa analyzed vocabulary complexity, sentence structure, and grammatical correctness. It also considered how effectively the essay addressed the different parts of the prompt and incorporated relevant examples, analysis, or details.
After this multi-dimensional analysis, Papa would assign a score on whatever scale was desired, such as the common 6-point, 8-point, or 100-point scales used in schools and standardized tests. The scoring model was trained on thousands of examples scored previously by expert human graders, so the goal was for Papa to assign scores in a manner consistent with what people would give. Along with the overall score, Papa provided sub-scores or dimension-level feedback on specific areas of strength and weakness in the writing. It could also give targeted suggestions on how to potentially improve a future draft. The entire scoring process typically took less than 5 minutes per essay.
Issues and Considerations with AI Scoring
While automated scoring solutions aimed to make assessment more accessible and consistent, they also raised various challenges that researchers continue working to address:
Bias – Like any machine learning system, AI scorers are as good as the data used to train them. If that data reflects societal biases around factors like race, gender, cultural background etc., the resulting models may systematically score some groups of students differently in unintended ways. Considerable study has gone into auditing large essay scoring datasets and models for potential unfair treatment.
Transparency – The internal workings of complex neural networks are difficult for people to comprehend, raising issues of explainability and model interpretability. Students and teachers want to understand why a score was given beyond just a number. Papa provided sub-scores and feedback, but the full decision-making process remained opaque.
Contextual Understanding – Capturing the nuanced cultural and rhetorical context around a piece of writing is an immensely difficult task, even for humans. While AI can analyze composition elements, it may miss important implications or inferences that human readers can make based on their lived experiences and general knowledge.
Creativity and Expression – More open-ended assignments aimed at creative or personal expression may challenge the limitations of pattern-matching algorithms. Quantitative metrics alone may not fully capture the qualitative dimensions of insightful, impactful or artful writing.
High-Stakes Testing – Concerns are amplified when AI scoring is used for consequential assessments that impact college admissions or graduation requirements. Students deserve transparent, consistent and bias-free evaluations, with appropriate human oversight and opportunities for appeals.
Addressing the Challenges
Tech companies and researchers have proposed several approaches to address these challenges:
Bias Mitigation – Developing bias identification techniques, collecting more diverse datasets, incorporating fair machine learning techniques, rigorous validation studies on demographically varied groups.
Explainability – Using more transparent models like decision trees, highlighting key features used, offering detailed step-by-step explanations of scores.
Limited Use Cases – Supplementing rather than replacing humans for now in important contexts. Consider low-stakes formative uses before high-stakes summative applications.
Accuracy Improvements – Continuing to expand training data and refining models to better capture more complex writing abilities beyond just structure/mechanics.
Human Oversight – Involving teachers and assessment experts in monitoring scores, flagging exceptions, re-scoring problematic examples, and providing overall quality control.
Appeals Processes – Enabling request for re-evaluation by different scorers (human and/or AI) if students question initial scores. Proactively identify areas needing human judgment.
As the capabilities of AI systems advance while also focusing on issues like inclusiveness and accountability, automated scoring offers potential benefits if implemented cautiously and complemented by human expertise. With diligent research and development, future versions may more closely emulate reliable, nuanced grading by people.
Conclusion
Automated essay scoring sparked both promise and questions around its application, especially for high-stakes purposes. Services like Papers Graded aimed to make assessment more time efficient and standardized, but legitimate concerns surrounded bias, transparency and contextual understanding limitations of early systems. By proactively addressing bias, improving explainability, involving human oversight, and narrowing initial use cases, the field of AI scoring continues progressing towards designs that can reliably, equitably and confidentially support – rather than replace – expert human judgement of student writing abilities. With open challenges also comes opportunity to build assessment tools that provide benefits of consistency without compromising fairness, validity or student needs.
Andrew Stroud