Artificial Intelligence Becomes a Star Law Student, Study Reveals

A groundbreaking academic study has revealed that artificial intelligence has reached a new academic milestone: outperforming many law students on real law school final exams. 

 

Researchers at the University of Maryland Francis King Carey School of Law found that OpenAI’s latest model, known as o3, scored from A+ to B grades on eight recent exams, outperforming earlier generations of generative AI tools and rivalling top-performing students.

 

AI Overtakes Its Predecessors in Legal Reasoning

According to the paper published on SSRN, the O3 model demonstrated significant gains over prior ChatGPT models, which typically received B, C, and even D grades when tested on similar exams between 2022 and 2023. 

 

The O3 model’s improved performance stems from its design as a "reasoning model", which generates tentative answers and revises them internally before producing a final response, mimicking human analytical processes more closely than ever before.

The AI’s final grades included:

• A+ in Constitutional Law, Professional Responsibility, and Property

• A in Income Taxation

• A- in Criminal Procedure

• B+ in Secured Transactions and Torts

• B in Administrative Law

These results suggest that O3 can now match or exceed the academic performance of many law students, particularly in essay-based and multiple-choice questions.

 

Limits of the Machine

Despite its impressive academic capabilities, the study also revealed the AI’s limitations. Its lower grade in Administrative Law was primarily due to its lack of knowledge about the 2024 U.S. Supreme Court ruling in Loper Bright Enterprises v. Raimondo, which overturned the long-standing Chevron deference principle, a doctrine central to administrative law. Because O3’s training data ended before this major ruling, it was unaware of the decision and its implications.

 

Interestingly, when researchers fed the AI with a professor’s notes in one instance, performance declined. The model became "distracted" by excessive context, suggesting that too much input could hinder rather than help its output—a surprising finding for proponents of context-based prompting.

 

Also Read: AI Ethics and Compliance in UAE's Legal Landscape

 

Implications for Legal Education and Academic Integrity

The study’s co-authors, including seven law professors, used their standard grading curves to assess o3’s responses, ensuring the evaluation remained consistent with human standards. While this establishes that AI can now perform at the level of top law students, it also reignites debates around academic integrity and potential misuse.

 

The researchers are already planning follow-up experiments to see whether AI could be used for academic cheating. They aim to make the AI intentionally include minor spelling and grammar mistakes, making it harder to distinguish from a human student—a tactic that could be exploited if safeguards are not enforced.

 

Legal Sector Braces for Further Disruption

This study marks another step in the growing disruption of the legal profession by generative AI. While previous research noted that AI could pass the bar exam and accelerate legal writing, the quality of that writing was often considered "mediocre." This latest development suggests AI may now be capable of producing top-tier legal analysis, challenging the role of traditional legal education and early-career legal work.

 

As law firms increasingly adopt AI tools to streamline legal research, contract analysis, and brief writing, experts warn that the legal industry must adapt quickly, not just in practice but also in regulatory frameworks and ethics training.

 

Expert Insight: Legal Education Must Evolve

Sunil Ambalavelil, Chairman of Kaden Boriss and a seasoned legal professional based in the UAE, offered this perspective: 

• “This study is both exciting and cautionary. While it validates the power of artificial intelligence in legal reasoning, it also challenges our current models of legal education and professional development.” 

• “In jurisdictions like the UAE, where innovation is welcomed but regulation is evolving, it's critical that we set clear boundaries on the ethical use of AI in academic and professional settings.” 

• “Firms, educators, and regulators must work together to preserve the integrity of legal practice while embracing tools that enhance human capability.”

 

Final Word

As AI systems become indistinguishable from human learners, institutions must bolster their honour codes, redesign assessment formats, and implement AI-detection mechanisms to preserve academic standards.

 

For any enquiries or information, contact info@thelawreporters.com or call us on +971 52 644 3004. Follow The Law Reporters on WhatsApp Channels.