US Law Firm Apologizes After AI Hallucinations Made It To Legal Filing

Authored by Brayden Lindrea via CoinTelegraph.com,

Wall Street law firm Sullivan & Cromwell has apologized to a federal judge after submitting a court filing that contained around 40 incorrect citations and other errors caused by AI hallucinations.

“We deeply regret that this has occurred,” Andrew Dietderich, co-head of Sullivan & Cromwell’s global restructuring team, wrote Friday in a letter to Chief Judge Martin Glenn of the US Bankruptcy Court for the Southern District of New York.

“The Firm and I are keenly aware of our responsibility to ensure the accuracy of all submissions including under Local Bankruptcy Rule 9011-1(d), and I take responsibility for the failure to do so,” he said of an emergency motion filed nine days earlier.

Excerpt from Andrew Dietderich’s letter to Chief Judge Martin Glenn. Source: Sullivan & Cromwell

The incident highlights the risk AI tools can pose in high-stakes professional work without proper oversight. A database managed by legal technologist Damien Charlotin has recorded 1,334 incidents of AI hallucinations in court filings around the world, including more than 900 in the US.

Charlotin pointed out that most of these hallucinations involve fabricated citations, though AI-generated legal arguments have also occasionally been identified.

Dietderich said Sullivan & Cromwell has policies in place for the use of AI tools, which include a review of the citations it uses, but said the policies weren’t followed.

“Regrettably, this review process did not identify the inaccurate citations generated by AI, nor did it identify other errors that appear to have resulted in whole or in part from manual error.”

Sullivan & Cromwell is one of the largest law firms in the US by revenue, ranking 30th on the AmLaw Global 200. The firm also represented crypto exchange FTX in its bankruptcy case.

Sullivan & Cromwell is conducting an internal investigation

Dietderich said the law firm took “immediate remedial measures,” including a full review of the circumstances that led to the errors. 

The firm is also “evaluating whether further enhancements to its internal training and review processes are warranted,” Dietderich said.

Dietderich also noted that the errors were spotted by a rival law firm.

“I also called Boies Schiller Flexner LLP on Friday to thank them for bringing this matter to our attention and to apologize directly to them as well,” he said.