AI Hallucinations in Legal Filings: A Growing Concern for Lawyers

The Rising Risk of AI-Generated Errors in Law

Artificial intelligence (AI) is revolutionizing the legal industry, offering faster research and document drafting. However, its tendency to fabricate information, known as LLM hallucination (Large Language Model hallucination), is creating serious issues. These neural network errors can mislead legal professionals and jeopardize cases.

Morgan & Morgan’s Warning to Lawyers

Morgan & Morgan, a leading U.S. personal injury law firm, recently sent an urgent message to its lawyers: AI-generated legal fiction can cost you your job. The warning followed an incident where a federal judge in Wyoming threatened to sanction two of its attorneys for citing non-existent cases in a lawsuit against Walmart. One lawyer admitted he unknowingly used an AI tool that fabricated legal precedents.

AI Hallucinations: A National Legal Issue

This is not an isolated case. Over the past two years, at least seven legal cases across the U.S. have involved false AI-generated citations. Courts have questioned or sanctioned attorneys who relied on incorrect AI outputs, raising alarms about AI’s role in legal research.

The Walmart lawsuit is particularly notable due to the involvement of a major law firm and a large corporation. However, similar incidents have appeared in various lawsuits, emphasizing the risks AI presents in litigation.

Generative AI: A Double-Edged Sword

Law firms are increasingly integrating AI tools to expedite research and drafting. A Thomson Reuters survey revealed that 63% of lawyers have used AI at work, with 12% using it regularly. Despite its efficiency, AI is known for confidently generating false information based on statistical patterns rather than factual verification.

Legal ethics demand that attorneys verify their court filings. The American Bar Association (ABA) has reinforced this obligation, warning its 400,000 members that even unintentional AI-generated misstatements can lead to disciplinary actions.

Legal Consequences of AI Mistakes

Andrew Perlman, dean of Suffolk University’s Law School, calls unverified AI-generated citations “pure incompetence.” Courts are taking action against such errors:

In June 2023, a Manhattan judge fined two New York attorneys $5,000 for submitting AI-generated fake citations in a personal injury lawsuit.

A New York federal judge nearly sanctioned Michael Cohen, Donald Trump’s former lawyer, after he submitted false AI-generated legal references.

A Texas judge fined an attorney $2,000 and required him to attend an AI training course after citing nonexistent legal cases.

A Minnesota judge discredited a misinformation expert for including fake AI-generated citations in a case involving a deepfake parody of Vice President Kamala Harris.

The Need for AI Literacy in Law

Legal AI expert Harry Surden from the University of Colorado emphasizes that lack of AI literacy is the real problem. While AI errors are making headlines, lawyers have always made mistakes in filings. The key difference is that AI can generate plausible but false legal information at an unprecedented scale.

To mitigate these risks, legal professionals must:

Understand AI’s strengths and weaknesses.

Double-check all AI-generated legal references.

Use AI as a research assistant, not a replacement for legal expertise.

Follow court guidelines on AI usage in legal filings.

Conclusion: AI in Law Requires Caution

AI is a powerful tool, but its potential to fabricate legal facts makes it a liability if misused. The rise of AI hallucinations in legal filings underscores the need for rigorous verification and AI literacy. Lawyers must take responsibility for their filings, ensuring that AI enhances rather than compromises legal practice.

As AI continues to evolve, the legal industry must adapt responsibly. Failing to verify AI-generated content could lead to severe legal and professional consequences.