Can AI Developers Be Held Liable for Negligence? (lawfaremedia.org) 54
Bryan Choi, an associate professor of law and computer science focusing on software safety, proposes shifting AI liability onto the builders of the systems:
To date, most popular approaches to AI safety and accountability have focused on the technological characteristics and risks of AI systems, while averting attention from the workers behind the curtain responsible for designing, implementing, testing, and maintaining such systems...
I have previously argued that a negligence-based approach is needed because it directs legal scrutiny on the actual persons responsible for creating and managing AI systems. A step in that direction is found in California's AI safety bill, which specifies that AI developers shall articulate and implement protocols that embody the "developer's duty to take reasonable care to avoid producing a covered model or covered model derivative that poses an unreasonable risk of causing or materially enabling a critical harm" (emphasis added). Although tech leaders have opposed California's bill, courts don't need to wait for legislation to allow negligence claims against AI developers. But how would negligence work in the AI context, and what downstream effects should AI developers anticipate?
The article suggest two possibilities. Classifying AI developers as ordinary employees leaves employers then sharing liability for negligent acts (giving them "strong incentives to obtain liability insurance policies and to defend their employees against legal claims.") But AI developers could also be treated as practicing professionals (like physicians and attorneys). "{In this regime, each AI professional would likely need to obtain their own individual or group malpractice insurance policies." AI is a field that perhaps uniquely seeks to obscure its human elements in order to magnify its technical wizardry. The virtue of the negligence-based approach is that it centers legal scrutiny back on the conduct of the people who build and hype the technology. To be sure, negligence is limited in key ways and should not be viewed as a complete answer to AI governance. But fault should be the default and the starting point from which all conversations about AI accountability and AI safety begin.
Thanks to long-time Slashdot reader david.emery for sharing the article.
I have previously argued that a negligence-based approach is needed because it directs legal scrutiny on the actual persons responsible for creating and managing AI systems. A step in that direction is found in California's AI safety bill, which specifies that AI developers shall articulate and implement protocols that embody the "developer's duty to take reasonable care to avoid producing a covered model or covered model derivative that poses an unreasonable risk of causing or materially enabling a critical harm" (emphasis added). Although tech leaders have opposed California's bill, courts don't need to wait for legislation to allow negligence claims against AI developers. But how would negligence work in the AI context, and what downstream effects should AI developers anticipate?
The article suggest two possibilities. Classifying AI developers as ordinary employees leaves employers then sharing liability for negligent acts (giving them "strong incentives to obtain liability insurance policies and to defend their employees against legal claims.") But AI developers could also be treated as practicing professionals (like physicians and attorneys). "{In this regime, each AI professional would likely need to obtain their own individual or group malpractice insurance policies." AI is a field that perhaps uniquely seeks to obscure its human elements in order to magnify its technical wizardry. The virtue of the negligence-based approach is that it centers legal scrutiny back on the conduct of the people who build and hype the technology. To be sure, negligence is limited in key ways and should not be viewed as a complete answer to AI governance. But fault should be the default and the starting point from which all conversations about AI accountability and AI safety begin.
Thanks to long-time Slashdot reader david.emery for sharing the article.
Re: (Score:2)
This will simply result in endless litigation trying to score jackpots from deep pockets. Why don't we just put a warning label on every AI engine saying "use at your own risk, output may not be what you want". AI is in its infancy, it is going to have a HUGE number of errors in it. AFAIK no significant AI engine in existence can be blindly used without humans supervising its decisions. Consider AI as making suggestions, and the YOU decide whether or not to accept that suggestion.
Re: (Score:2)
Coders will insist on liability insurance as part of their employment package.
There will be less money for innovation and more money for lawyers and bureaucrats.
More development will move overseas.
Re: Who decides what an 'unreasonable risk ' is? (Score:2)
Legislation may have to change in order to define what's permitted in the EULA as well as requuring sign-off by a licensed software engineer.
Re: (Score:2)
Coders will insist on liability insurance as part of their employment package.
There will be less money for innovation and more money for lawyers and bureaucrats.
More development will move overseas.
Must be why we have a shortage of home builders and doctors in this country. Becasue of liability insurance. Everyone is moving overseas.
Re: Who decides what an 'unreasonable risk ' is? (Score:2)
Everyone wants to be an influencer.
Re: (Score:2)
A counter argument is that if the tech is known to be buggy and not ready for prime time (as you clearly believe to be the case), but the com
Re: (Score:2)
Bridge builders don't have the same kind of limited liability licenses (for good reason), but if the software causes too much damage, the limited liability license isn't going to help either.
Re: (Score:2)
Re: (Score:2)
What you are suggesting be done *has* been done in software since it's infancy.
The products are not warranted for any use.
Why wouldn't the same model work here?
You'll have to answer that before we get into your diatribe about bridges and AI being the same thing.
Re: You can already go to jail (Score:2)
The key is your software has to cause damages. If it does, then you can be sued.
Re: (Score:2)
Quick reminder: you can already be held liable for damage your software causes. A prime example is the software engineers at Volkswagen, who wrote the code to cheat emissions. This guy is another example [darkreading.com]. Occasionally drug website developers are arrested. The key is your software has to cause damages. If it does, then you can be sued.
And this guy is talking about negligence, not "I used your AI model to try to beat the market and lost money." Negligence is a lot different than just it didn't work like I thought it would.
Re: You can already go to jail (Score:2)
Re: (Score:1)
"This is not enough. The risk is that the output may not be what we want you to hear."
-- From a totalitarian state or some equivalent though-police department.
Re: (Score:2)
How about a regulatory authority? (Score:2)
Ask 10 people and you'll get 10 different answers.
Ask the FAA and you'll get one answer, and it'll be a precise answer.
The FAA has a bunch of codified rules for software development and testing. As I understand it, the intent was that if the code was developed and tested according to FAA rules, then the manufacturer would not be held liable for problems; specifically, the airplane manufacturer could not be sued for building an unsafe plane, because the design processes were the best standards of the industry.
(And note that the NTSB and FAA analyze crashes
Re: (Score:2)
I think the same thing will happen with autodrive in cars: if the autodrive software is statistically safer than a human driver, then the manufacturer can't be sued for fatal crashes.
They'll have to nail down "human driver" though. Are we talking about a 50th percentile, "average" driver? 5th percentile "drunk, distracted, and speeding"? 95th percentile "never had an accident"?
Then, of course, ideally "best practices" will eventually take over, where you stop competing against the "human driver" and start needing to meet "industry best practices" - IE not zero accidents, but a certain constant effort to reduce them.
Re: Who decides what an 'unreasonable risk ' is? (Score:2)
No human involved. I do wonder what copilot would say though.
CEOs, sure (Score:3)
Those making the business decisions are the ones responsible.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Those making the business decisions are the ones responsible.
That’s odd. The CEOs keep pinning responsibility on someone named “Fiduciary”, who’s apparently one HELL of a demanding bitch.
The next big thing in AI (Score:2)
Artificial Insurance.
Re: (Score:2)
We don't blame weather forcasts for being wrong (Score:1)
AI is random noise, statistics, and a predictive algorithm that guesses at what the inputted information is.
If there's a 90% chance of rain it's not like I'm mad when it stays a nice sunny day.
AI is great when you straight up have the original data, and can compare it. Until recently, we couldn't even get pictures with people with the correct number of fingers. Why in the world would you blame the AI makers for mistakes? It's only as good as the data going in, if your data doesn't exist, the AI will guess.
Re: (Score:2)
I'd only blame the "AI maker" in the sense of it being the business or company making it and putting out unreasonable claims. Not the developers, who as somebody else said, may not have the decision to deploy it or not. Same deal if a team was developing a plain and the business side decided to push it out the door before all the problems had been worked out.
Re: (Score:2)
> We don't blame weather forcasts for being wrong
The moment you assume people are rational, you lose.
https://www.unilad.com/news/wo... [unilad.com]
https://lauralaw.net/the-law-o... [lauralaw.net]
All programmers should be liable (Score:2)
Re: (Score:2)
Well, no. But other engineering fields provide guidelines. Essentially, if you pay for a professionally made thing, you have a reasonable expectation of it being fit for purpose and having no severe flaws. Bu there are degrees, even in established engineering disciplines and it depends on what you can reasonably expect from a product. The problem is "free" stuff, whether really free (FOSS) or just free do download and use. Obviously, even something you get for free should either be labelled "experimental, m
Re: (Score:2)
I'm entirely unsure what your argument is.
We *do* have guidelines for safety critical software. I know. I used to write it. So that part of your argument is rubbish.
As far as free software, it's like finding a random piece of fried chicken by the road side and thinking it's okay to eat, not from some supermarket.
What is your point?
Re: (Score:3)
There's a reason this requirement isn't in place for computers / IT: It's not a static system, and the variables introduced into it may not even exist for literal decades after it's brought into service. Placing liability on the designers of the system for a problem that may not exist until well after the original designers have retired isn't going to get you a lot of people volunteering to go in
Re: (Score:2)
It would lead to a world where little to no software innovation occurred.
Software is already held accountable in situations where it matters, think trains, FAA, etc.
Your proposal just serves to get more lawyers rich.
You wouldn't happen to be a lawyer, would you?
What's he smoking? (Score:2)
You can't even get that form of liability coverage from developers who don't program AI, so you think an AI developer should put his/her head in the guillotine? Why don't you just let the developers take out liability insurance and call it aday?
Re: (Score:2)
You expect all the Open Source developers, as well as anyone else who wants to release software take out liability insurance?
Nope, sorry!
You'll get a lot of people anonymously releasing software then, on darknets if necessary.
Nobody ever sued Microsoft (Score:2)
for spreadsheet errors, lost Outlook emails, etc. Why would dodgy AI be any different?
Re: (Score:2)
It's right in their EULA, that's why.
They state is isn't fit for any purpose whatsoever, and it's your fault if you use it at all. At least that's the gist.
It will be different for AI if they make it different.
There's no logic necessary. Laws don't need to cleave neatly along non-law boundaries. They can define AI, so they can separate is as a category.
Why should they? (Score:2)
Software vendors can release any old POS code and cause untold damage with zero consequences. Why would AI developers be treated any different?
I content that both should be held liable. It's high time Microsoft should answer for bugs in Windows that allows malware to exfiltrate personal data for example, and officers of companies that release reckless software face jail time.
Similarly, companies that use "algorithms" to moderate their sites, like Youtube or Amazon, should be held accountable for whatever th
When Twitter became 'news' (Score:2)
I remember when Twitter became a 'news' source: Competing networks rushed to publish the latest meme, which usually lacked an eye-witness and/or facts. Will broadcast networks repeat the same pissing-contest using AI?
We already know LLMs hallucinate and worse, they feed on their on own garbage, resulting in a toxic 'personality' such as Microsoft Tay. Still, reporting and journalism, is seen as the perfect job for LLMs, turning many words into few. Such LLMs will also have to include the bias of broad
Re: (Score:2)
Re: (Score:2)
And also in that, you get all the uninformed opinions and anti-facts that have been posted in that Slashdot discussion.
Nicely presented as facts, all polished and turd like, in this case.
Make Comp Sci associate professors responsible (Score:1)
Associate Computer Science professors should be criminally charged when their students go on to write shitty code that ends up exposing PII (like FB keeping billions of passwords in the clear, or companies CONSTANTLY getting hacked).
Instead of those companies bearing the burden of having good practices, and not have to get their customers "one free year of LifeLock" (a negative value given the information leaked or hacked) the people who were negligent in teaching the programmers should be charged with that
Re: (Score:2)
Let's just sue the parents of the people who did the work, too.
If they hadn't spawned the kid, they would never have made the mistake!
AI Developer payroll (Score:2)
..But AI developers could also be treated as practicing professionals (like physicians and attorneys). "{In this regime, each AI professional would likely need to obtain their own individual or group malpractice insurance policies."
Sure. They could treat AI developers like that..if they want to see those job titles come with seven-figure starting salaries.
Sometimes we forget why we pay those “practicing” professionals so much money. A good part of a doctors salary funds pay-to-play insurance policies.
Yes (Score:2)
If you make a tool to give information and it provides wrong information, then that is like a drill that doesn't, well, drill, you should get your money back if you bought that.
Lawyers push for more money to lawyers (Score:2)
Of course a law blog would advocate an idea which will bring lawyers more work, even if this idea doesn't make any sense.
Don’t Forget the Real Culprits: Politicians (Score:2)
Sure, AI developers should be held accountable for negligence, but let's not forget the politicians who've been ignoring decades of scientific warnings about climate change. They're the ones creating the pressure cooker environment where radical, untested tech becomes the "solution" to crises they failed to address. If policymakers hadn’t consistently dropped the ball, maybe we wouldn’t be in a position where we're depending on AI—or any other cutting-edge technology—as a last-minute