AI 'Hallucinations' in Court Papers Spell Trouble For Lawyers (reuters.com) 73
An anonymous reader shares a report: U.S. personal injury law firm Morgan & Morgan sent an urgent email this month to its more than 1,000 lawyers: Artificial intelligence can invent fake case law, and using made-up information in a court filing could get you fired. A federal judge in Wyoming had just threatened to sanction two lawyers at the firm who included fictitious case citations in a lawsuit against Walmart. One of the lawyers admitted in court filings last week that he used an AI program that "hallucinated" the cases and apologized for what he called an inadvertent mistake.
AI's penchant for generating legal fiction in case filings has led courts around the country to question or discipline lawyers in at least seven cases over the last two years, and created a new high-tech headache for litigants and judges, Reuters found. The Walmart case stands out because it involves a well-known law firm and a big corporate defendant. But examples like it have cropped up in all kinds of lawsuits since chatbots like ChatGPT ushered in the AI era, highlighting a new litigation risk.
AI's penchant for generating legal fiction in case filings has led courts around the country to question or discipline lawyers in at least seven cases over the last two years, and created a new high-tech headache for litigants and judges, Reuters found. The Walmart case stands out because it involves a well-known law firm and a big corporate defendant. But examples like it have cropped up in all kinds of lawsuits since chatbots like ChatGPT ushered in the AI era, highlighting a new litigation risk.
I don't like lawyers. Get rid of them (Score:1)
Re:I don't like lawyers. Get rid of them (Score:5, Insightful)
Lawyers may lie about what their clients did or did not do, but AI lies about what is and is not a law in the first place.
Re: (Score:3)
AI lies about what is and is not a law in the first place.
So do human lawyers. But when human lawyers do it there can be repercussions.
Re:I don't like lawyers. Get rid of them (Score:5, Informative)
Everybody hates lawyers till someone does you wrong.
The reason you hate lawyers is not because of what the lawyers do, but because of what their clients order them to do. When they are total scum, it is at the request of the scum that hired them.
Trust me it is your ex-wife that asked for your dog, not her lawyer. Elon Musk that denies TESLA has brake problem in china and sues the widow of his own customers for slander, not the chinese lawyers he hired.
And it was the restaurant that told their lawyers to say 'Boneless Wings are a type of dish, not a description of the chicken wings.', not their lawyers.
Re:I don't like lawyers. Get rid of them (Score:4, Interesting)
Re:I don't like lawyers. Get rid of them (Score:4, Informative)
> public defender
You had a public defender, not a lawyer. Public defenders are deliberately underpaid and overworked to provide the minimum legal requirement for representation. No matter how dedicated and passionate about real justice an individual public defender may be, their actual job is only to legitimize state persecution.
Next time get an actual lawyer, if you can afford one.
=Smidge=
Re:I don't like lawyers. Get rid of them (Score:4, Interesting)
Re:I don't like lawyers. Get rid of them (Score:4, Insightful)
I do not understand how a poor person should not have the money to defend himself/herself like Trump can? Throwing hundreds of millions of dollars into bullshitting pieces of shit that lie for you. I want that.
At risk of being glib, the answer is pretty obvious: society doesn't want to pay for it.
Voters are more supportive of "tough on crime" politicians than of those who would increase funding for public defender offices.
Re: (Score:3)
Yup, you rarely get a good public defender. Overworked, underpaid, unappreciated.
However the one criminal case I was a juror on, the public defender was great. He did his homework, he appeared experienced, he behaved professionally the entire time. And he won the case. The prosecutor on the other hand was inept, had not built a strong case, didn't even have a weak case, and everyone in the jury was surprised when she rested her case because we had kept waiting for some solid evidence to show up.
Re: (Score:2)
Re: (Score:2)
Well, at the time the DA was under intense scrutiny for a major prosecutorial screw up (satanic rituals hysteria). My thinking at the time was that there was pressure in the office to prosecute harder and get convictions to bolster him at the next elections.
Re: (Score:2)
Re: (Score:1, Troll)
> public defender
You had a public defender, not a lawyer. Public defenders are deliberately underpaid and overworked to provide the minimum legal requirement for representation. No matter how dedicated and passionate about real justice an individual public defender may be, their actual job is only to legitimize state persecution.
Next time get an actual lawyer, if you can afford one.
=Smidge=
I personally know a public defender. He is one of the most dedicated lawyers I have met. So not *all* lawyers are assholes. I will admit that he's overworked and underpaid. But he *does* care about his clients, and fights for them in court.
Difficulty: Massachusetts. Offer probably not valid in red states.
Re:I don't like lawyers. Get rid of them (Score:4, Insightful)
A system in which victory in court depends on your wealth shouldn't be mistaken for justice.
Re: (Score:2, Informative)
Re: (Score:2)
Re: (Score:1)
Re:I don't like lawyers. Get rid of them (Score:5, Informative)
That is not why I hate lawyers at all. I understand defending a criminal no matter what they have done. I understand arguing for their client.
What I hate about lawyers is their blatant overcharging, making a system that is needless complex, so they can charge more. They produce piles of documents to confuse the situation. They are basically grifters who's profession is to game the legal system in order to charge people as much as possible.
Examples I have experienced,
1. Sold a house to my trust, the lawyer charged for both writing up the document on one side and checking it on the other. Didn't they check it as part of writing it? Did they had to make sure I wasn't ripping myself off?
2. When I was doing a gifting to the trust, said it was a big job, because they needed to check that I still owned my own house, really I think I would have known if I sold it, and it was that would only effect me anyway.
3. Get there legal aids to write up the documents and charge like they did it.
4. Handed in a submission to the court, was rejected because it was printed double sided, the poor judge couldn't handle that. I had to print it again.
Look I understand there probably are ethical lawyers, that charge there clients a reasonable fee (probably public defenders) but I have yet to meet one.
But the whole profession seems to be broken, its so expensive that only the rich can afford reasonable representation.
But even your argument that they are only doing what their client asks them is no excuse, its like saying not the paid assassins fault they are only doing what their client asks. Yes I know what lawyers do is legal, but legal and ethical are not the same thing. But as I said that is not really my problem with lawyers.
Re: (Score:2)
"Did they had to make sure I wasn't ripping myself off?"
They had to prove that you weren't committing fraud (and also that you weren't implicating them in a fraud).
There were two separate tasks that would otherwise be done by two separate attorneys. I'd be suspicious they they were not billed separately.
"really I think I would have known if I sold it"
They needed to supply proof that you weren't committing fraud (and also that you weren't implicating them in a fraud). The word of a party interest in both sid
Re: (Score:2)
The complexity comes from legistlators; and not all of them are lawyers. Laws can be made simpler, have fewer loopholes and fewer deliberately ambiguous portions. The point of being a laywer is being able to decipher all that mess for the clients.
Re: (Score:2)
I've dealt with some lawyers, in a non-criminal way, and they were great and worth the fee. The law is complex and you need experts to help you deal with it.
Re: (Score:2)
The reason you hate lawyers is not because of what the lawyers do, but because of what their clients order them to do.
Only an unethical (or very, very stupid) lawyer will do what their client orders them to do, laws and ethics be damned. Most lawyers follow most laws and ethics. That being said, they will come as close as they think they can get away with to breaking those laws and ethics, while still maintaining a veneer of respectability. That's why you hire a lawyer: because they know what they can get away with and what they can't.
Re: (Score:3)
Trust me it is your ex-wife that asked for your dog, not her lawyer.
No dude. I will NOT trust you. I have experience in this matter. My wife and I were having a more or less amicable divorce until her sisters pushed her to get a lawyer.
That lawyer stood in front of a judge and claimed that I physically abused my wife throughout our marriage.
The judge asked for proof and the lawyer could not provide any. The judge then asked my wife, and she told the judge directly that I was never abusive. Thank god she had more integrity than the lawyer.
The world is not like you think it i
Re: (Score:3)
Everybody hates lawyers till someone does you wrong.
The reason you hate lawyers is not because of what the lawyers do, but because of what their clients order them to do. When they are total scum, it is at the request of the scum that hired them.
Trust me it is your ex-wife that asked for your dog, not her lawyer. Elon Musk that denies TESLA has brake problem in china and sues the widow of his own customers for slander, not the chinese lawyers he hired.
And it was the restaurant that told their lawyers to say 'Boneless Wings are a type of dish, not a description of the chicken wings.', not their lawyers.
I think the problem is that most people only encounter the scummiest of lawyers, the ambulance chaser, Lionel Hutz type shysters, people behind the "YOU CAN SUE" ads. So this is the image most people conjure up when talking about lawyers.
They make up the overwhelming minority of lawyers, most lawyers will being doing specialities like finance, property, contract, et al. and are generally straight talking, honest, if not a little anally retentive but that's not a bad thing when you need someone to make su
Re: (Score:1)
Re:I don't like lawyers. Get rid of them (Score:4, Informative)
"The first thing we do is, let's kill -9 all the lawyers."
- W. Shakespeare
Re: (Score:2)
Often forgotten that the line is said by a villain who wants to get rid of the lawyers so he can get away with mischief.
Re: I don't like lawyers. Get rid of them (Score:2)
AI can replace the entire legal system today. It will be full of inaccuracies and hallucinations, and outcomes can be manipulated by bad actors. But that's kinda like the system we already accept.
Faster. Cheaper. Fairer. Pick any zero of thoses.
Re: (Score:2)
Re: (Score:2)
I dunno. A good lawyer is rare and you want to keep them. In my view, in ANY job, if you need to use AI then you are not good at your job. A lawyer that used AI as a shortcut and then did not cross check every word of the output is a bad lawyer.
Tell your Paralegal not to trust AI (Score:5, Insightful)
ChatGPT, for example, is shockingly good at creating citations that appear genuine. It'll give you a case that seems to be what you're looking for, real parties that may even have been in a real case against one another, content that is basically what you're looking for... but it won't be real. The citation LOOKS real, but it's a few digits off. When you go to try to find it, you might find the case that it's masquerading as, but the content is totally different. You can even ask ChatGPT if it is SURE that this case is real, and it will cheerfully assert that it is!
At the end of the day the Lawyer is responsible for the work of their staff, so it still falls to them to not let this happen. But I feel bad for all the Paralegals or Legal Assistants who have had these AI tools shoved in their face and been told to use them because the firm spent big bucks. Still... someone at SOME level should be looking up these cases before putting them in the pleadings.
Re:Tell your Paralegal not to trust AI (Score:5, Insightful)
ChatGPT, for example, is shockingly good at creating citations that appear genuine. It'll give you a case that seems to be what you're looking for, real parties that may even have been in a real case against one another, content that is basically what you're looking for... but it won't be real. The citation LOOKS real, but it's a few digits off. When you go to try to find it, you might find the case that it's masquerading as, but the content is totally different. You can even ask ChatGPT if it is SURE that this case is real, and it will cheerfully assert that it is!
At the end of the day the Lawyer is responsible for the work of their staff, so it still falls to them to not let this happen. But I feel bad for all the Paralegals or Legal Assistants who have had these AI tools shoved in their face and been told to use them because the firm spent big bucks. Still... someone at SOME level should be looking up these cases before putting them in the pleadings.
This is kind of what current gen AI does with everything. If it can't come up with a direct answer, it'll make something up and be very convincing about it. With code it's typically easy to catch, but there are a lot of folks used to just trusting whatever the computer tells them and don't question it. Leading to exciting stories like this one. What I want to know is if it's actually impossible to build these things in a way where "I can't find that information" is a possible output, or if companies have chosen *NOT* to implement that as a possible response simply so they don't run into edge cases where someone pays a bunch of money, then gets a 95% return of "I can't find that information." I have this sneaking suspicion it's the latter, but don't have the data to back up that hunch.
Re: (Score:2)
it'll make something up and be very convincing about it.
Its quite interesting because many of the approaches suffer from the same limitation of holding it together for short bits/areas/sentences and everything flows from one element to the next, but when you back up and look at the big picture it can’t hold it together. No matter if it’s chatGPT and logically falling apart at anything over a paragraph or midjourney and object scale, larger forms, or any number of real world details that is logically evident from the situation but this common sense i
Re: (Score:2)
it'll make something up and be very convincing about it.
Its quite interesting because many of the approaches suffer from the same limitation of holding it together for short bits/areas/sentences and everything flows from one element to the next, but when you back up and look at the big picture it can’t hold it together. No matter if it’s chatGPT and logically falling apart at anything over a paragraph or midjourney and object scale, larger forms, or any number of real world details that is logically evident from the situation but this common sense is missing. We need an actually intelligent algorithm, or set of them, integrated to adhere to real world common sense on larger scales.
Today, we don't know how to do that. These are guessing algorithms, not logic algorithms. They seem intelligent to management types though, because they can sort of string sentences together in a somewhat logical way. Which is, admittedly, better than some of us do in a business setting.
Re: (Score:2)
But I feel bad for all the Paralegals or Legal Assistants who have had these AI tools shoved in their face and been told to use them because the firm spent big bucks.
Is that really what is happening though? It seems strange to me that we keep hearing multiple examples of lawyers getting severely sanctioned for use of AI generated fakery and yet lawyers seemingly keep on presenting this sort of thing to courts. It seems extremely likely that every lawyer now knows to avoid AI tools like the plague because of the personal risk to them that its use will likely cause so why would they spend "big bucks" on a tool that is going to get them fined if not disbarred and leave th
Re: (Score:2)
First there are a lot of lawyers so its likely that some are going to use AI, they don't seem to be that smart to me. Second if all lawyers do not to use AI should they have informed there paralegals not to use it either. Also shouldn't they be checking their paralegals work anyway after all they are not lawyers and the lawyer is responsible for them.
I'm sure Grok 3 is better (Score:2)
I mean, they have thousands of GPUs, so there's no problem anymore. It works just as well as Full Self Driving, which is definitely not in forever beta.
Eh... (Score:4, Insightful)
They should be sanctioned (Score:5, Insightful)
Lawyers in these cases should be sanctioned and whatever other punishments are usual for lying to the court. "My computer did it" is worse than "my dog ate my homework" on the hierarchy of unacceptable excuses. Especially after multiple incidents have been widely reported. Anyone trying the "I thought it was just a fancy web search" is either lying or incompetent. Not that it ever should have been an acceptable excuse. There are dedicated legal databases for such things. Any lawyer using a general web search (the thing they claim they think the LLM is replacing) for case law is a bad lawyer.
Any clients should be examining their bills very closely. Were you billed for 4 hours of senior partner time when the work was a 1st year spending 15 minutes with ChatGPT?
Re: (Score:1)
Re:They should be sanctioned (Score:5, Insightful)
I could forgive the first lawyer who fell into the trap in a highly publicized case about ... a year ago or so now? ChatGPT was new, it was overhyped, no one really understood the limitations yet, and lawyers are just human.
Today, though? It's like not knowing you shouldn't use your phone while driving.
Re: (Score:2)
Where convenience converges with no scruples. If we make it easy then they'll come pouring out of the woodwork. Take-what-can-be-taken attitude.
Re: (Score:3)
Many lawyers have been sanctioned. In fact, it's made the news so much that no lawyer should be using AI at all.
I mean, it was bad enough when the lawyer in the airline case used AI - the court sanctioned him, forced him to pay the defense's legal costs for wasting their time researching non-existent cases, and threw the case out with prejudice. The judge did tell the plaintiff to take the damages and sue their lawyers.
And this has happened many more times. So many times it's not even funny. It was so bad,
Re: They should be sanctioned (Score:2)
Re: (Score:1)
Disbarred, I should hope.
> "My computer did it" is worse than "my dog ate my homework"
I mean, if the results were satisfactory, were they planning to give the AI credit and pay it? Because you can't have responsibility and not have it too. Credit and blame go together. Either the AI is a tool that you use and you are responsible, or not.
Re: (Score:1)
Re: (Score:2)
thousands of people sitting in cubes clicked on the one out of eight responses that looked least retarded
On all the Captchas that ask me to click on all the pictures of vehicles, I always skip the bicycles. I can only hope that this makes it into our traffic laws at some point.
Re: (Score:2)
I always skip the bicycles. I can only hope that this makes it into our traffic laws at some point.
It already has. Look up the Idaho stop that has been trending for over a decade
I do love the lawyer shows.. and the dilemas. (Score:2)
Re: (Score:2)
That says more about the job screenwriters than real life. As a lawyer, a TV show about my job would be very, very, boring. But if you wrote in some major ethical dilemmas and made my behavior problematic, you could make it a LOT more interesting.
Re: (Score:3)
No, really? (Score:3)
I'm glad this happening. People need transparency about how unreliable AI is.
Never mistake gossip for gospell.
Re: (Score:2)
Why AI won't replace humans? Responsibility (Score:2)
You can't hold computers accountable. It's not a thing that can care about accountability. You can hold humans accountable. This is why AI won't replace humans. The HUMAN lawyers can be punished for making professional errors, a computer AI can't be.
Re: (Score:1)
Re: (Score:2)
But here is the problem with self-driving cars: one will eventually crash. When a human driver crashes and kills people, well that's a tragedy but it's a one-off. You punish that one person, and life goes on. But when a self-driving vehicle crashes, it's no longer a one-off. You've got thousands, or millions, of the exact same cars on the road with the exact same software. If another car somewhere else makes the same mistake and kills MORE people, now you have a major recall problem. You aren't just talking
Re: (Score:2)
To make the legal system fair and blind. (Score:2)
Re: (Score:2)
Re: (Score:2)
From a Lawyer (Score:5, Informative)
I am a lawyer. I don't do personal injury, but I did work part time for a personal injury firm in law school so I know the basics of the practice area.
Morgan & Morgan is what is known as a "mill." They offer low-end service on mostly low-end cases and make money through volume. Further, my understanding is they are more of a referral network and advertising collective than a true law firm, so there's not much oversight over individual attorney work product. So it's not surprising there are attorneys affiliated with Morgan and Morgan who are looking for any shortcut possible to cut costs and overhead. Why pay a Jr. associate $50-100k a year to churn out pleadings when you can just use Chat GPT?
Setting aside the duty of oversight and ethical concerns, this is mostly made possible because the personal injury mill business involves extremely repetitive claims. You aren't looking at novel or complex legal issues. The complaints to initiate suits are almost all copy paste with the names and dates changed. Most of it CAN be done by AI without much trouble. The problem is that signing a pleading means you are agreeing to be held accountable for what it says, including the ethical duty of candor to the court. Whether it was a Jr. Associate who made up the case or AI, failing to check the work product or another is a dereliction of duty.
The legal profession already has a method to police this. It's called sanctions. The court can directly fine lawyers for this sort of conduct. I suspect it will get more common until AI cite checking gets good enough that these low-level AI generated pleadings fly under the radar. What I fear even more is when we get judges using AI to write their opinions.
Re: (Score:1)
Re: (Score:2)
I am a tax lawyer, so I my work isn't in opposition to engineers at all. In fact, engineers have been very helpful to my work. Many things like R&D tax credits depend on engineers doing their thing.
Sloppy lawyering (Score:2)
I paid for legal incorporation documentation from a local lawyer many years ago and when I read it, there was a large part that clearly was unrelated leftovers from a cut/paste. I called the lawyer on it and he just said to cross it out!
Sounds like the AI-enabled lawyers have the same problem--sloppy work.
Weekly reminder needed (Score:2)
This is one of those things where they'll need to send out a reminder at least once a week. I don't know why, but the dullards keep thinking that AI is some magical thing that knows everything, even when you tell them and show them that it generates garbage.
Going To Use AI Anyway (Score:3)
Lawyers in particular have been famously made well aware that AI hallucinates not only case law, but every single citation that it ever says. The latter being all-fake all-the-time always. This has been in the courts and newspapers and legal newsletters and journals and bar associations and EVERYWHERE for lawyers FOR YEARS at this point.
There is no way that any lawyer is unaware that AI is (to say the least) not safe to use in court. And yet, this is not stopping them.
The only thing that will make any impact is if each time they are caught using fake AI-generated crap in court, they permanently lose their law license. Just as if they had personally deliberately and with malicious intent decided to make up random fake shit and try to tell the judge it's case law.
Burn them.
Re: (Score:2)
This can't happen with AI/LLM written software (Score:4, Funny)
Just remember the triumphal success of Elizabeth Holmes of Theranos or Sam Bankman-Fried at FTX. You could be as rich and successful as those founders are now if you invest soon enough, or stake your business on AI/LLM systems that guarantee success. The only way to loose is not do AI/LLM. Trust me.