California Bar discloses AI was used to develop some questions in problem-plagued February exam

The State Bar of California has revealed that some multiple-choice questions in a troubled bar exam were created with the help of artificial intelligence. In a statement on Monday, the legal licensing body announced it will seek to adjust test scores for February bar exam takers with the California Supreme Court.

Mary Basick, assistant dean at the University of California, Irvine, Law School, expressed shock over the situation, calling it “unbelievable” that questions were developed by non-lawyers using AI.

The February exam faced issues as test-takers encountered problems like online platform crashes, essay saving difficulties, screen delays, and error messages. The State Bar disclosed that 100 of the 171 multiple-choice questions were from Kaplan Exam Services, while 48 came from a first-year law students exam. Additionally, 23 questions were developed with artificial intelligence by ACS Ventures.

Leah Wilson, the State Bar’s executive director, affirmed confidence in the fairness and accuracy of the multiple-choice questions to evaluate test-takers’ legal competence. Katie Moran, an associate professor at the University of San Francisco School of Law, described the situation as a “staggering admission” and criticized the State Bar for using non-lawyers and AI to create and approve exam questions.