California Senate Moves to Curb Lawyers’ Use of Artificial Intelligence

The California Senate has passed a bill that would impose strict obligations on lawyers using artificial intelligence, requiring them to verify the accuracy of all AI-generated materials submitted in court, including case citations and factual information.

 

The legislation, believed to be among the first of its kind at state level, has now been sent to the California State Assembly for consideration.

 

Under the proposal, lawyers would be required to take “reasonable steps” to confirm the reliability of any content produced using AI, correct false or hallucinated outputs, and remove biased material before relying on it in legal work. Attorneys would also be prohibited from inputting confidential, personally identifiable or other non-public information into publicly available generative AI tools.

 

The bill further restricts arbitrators handling out-of-court disputes from delegating decision-making to generative AI or relying on AI-produced information outside the official case record without first notifying the parties involved.

 

Courts across the United States have increasingly confronted the risks posed by AI in legal practice, particularly as chatbots and specialised tools have been linked to fabricated case law and factual errors. In dozens of cases, lawyers and self-represented litigants have faced sanctions or reprimands for submitting unverified AI-generated material.

 

Introduced by Democratic Senator Tom Umberg, chair of the California Senate’s judiciary committee, the bill—known as SB 574—aims to establish clear guardrails as AI becomes more prevalent in the legal system. Umberg said the measure is intended to protect client confidentiality and ensure that legal decisions remain the responsibility of humans, not algorithms.

 

The proposal is part of a broader push in California to regulate artificial intelligence, including recent legislation requiring major companies to disclose how they plan to mitigate potentially catastrophic risks posed by advanced AI models.

 

According to a Senate analysis, the bill builds on existing professional obligations already binding on lawyers, such as the requirement that court filings be grounded in existing law. It is also modelled on California’s rules governing AI use by judges and court staff, as well as a recent appeals court decision that fined a lawyer $10,000 for submitting fabricated information and stressed the duty to personally verify all citations.

 

For any enquiries or information, contact ask@tlr.ae or call us on +971 52 644 3004. Follow The Law Reporters on WhatsApp Channels.