Delta Sues CrowdStrike Over Software Update That Prompted Mass Flight Disruptions (reuters.com) 33
An anonymous reader quotes a report from Reuters: Delta Air Lines on Friday sued cybersecurity firm CrowdStrike in a Georgia state court after a global outage in July caused mass flight cancellations, disrupted travel plans of 1.3 million customers and cost the carrier more than $500 million. Delta's lawsuit filed in Fulton County Superior Court called the faulty software update from CrowdStrike "catastrophic" and said the firm "forced untested and faulty updates to its customers, causing more than 8.5 million Microsoft Windows-based computers around the world to crash." [...]
Delta, which has purchased CrowdStrike products since 2022, said the outage forced it to cancel 7,000 flights, impacting 1.3 million passengers over five days. "If CrowdStrike had tested the faulty update on even one computer before deployment, the computer would have crashed," Delta's lawsuit says. "Because the faulty update could not be removed remotely, CrowdStrike crippled Delta's business and created immense delays for Delta customers." Delta said that as part of its IT-planning and infrastructure, it has invested billions of dollars "in licensing and building some of the best technology solutions in the airline industry."
Delta, which has purchased CrowdStrike products since 2022, said the outage forced it to cancel 7,000 flights, impacting 1.3 million passengers over five days. "If CrowdStrike had tested the faulty update on even one computer before deployment, the computer would have crashed," Delta's lawsuit says. "Because the faulty update could not be removed remotely, CrowdStrike crippled Delta's business and created immense delays for Delta customers." Delta said that as part of its IT-planning and infrastructure, it has invested billions of dollars "in licensing and building some of the best technology solutions in the airline industry."
The should have read the fine print (Score:2)
Damages never more than the software costs.
Re: The should have read the fine print (Score:2)
These exclusions donâ(TM)t apply to things like gross negligence or fraud, and whether or not the event was one of those is a question of fact for a jury to decide.
Re: (Score:2)
Delta is demanding from its contractor reparations for damages that it's refusing its customers. When they reject claims for which they have responsibility, how can they go after their contractors on the same basis?
Delta is not the only airline to use CrowdStrike, and they all had outages on July 19. Delta, however, is the only one that couldn't recover until July 24.
Crowdstrike's test and deployment processes to me look like gross negligence: their business is having companies ent
Re: The should have read the fine print (Score:2)
The process that delta uses may not be relevant. What matters is what Crowdstrike said its processes are, what the contract between delta and Crowdstrike says, and what a reasonable person would do based on those assurances. Crowdstrike publicly admitted that it did not test the update. If the contract between delta and Crowdstrike says that Crowdstrike shall test all updates before deployment on infrastructure that duplicates delta infrastructure because delta doesnt want to do that testing itself, Crowdst
Re: (Score:2)
I didn't follow this one beyond the basics. Did crowdstrike really admit in public or to Delta lawyers that they didn't test before releasing??
The Person I Would Hate to Be (Score:3, Interesting)
Re: (Score:2)
It's possible that the defendant's lawyers may try to cast shadow on the coder, but the court is likely to consider all aspects of the event, including very poor processes by the company at large. After all, any revision and release processes need to take into account problematic or dangerous updates, and so while you might be able to blame a coder or a code group of buggering it up, the update was green lit, suggesting there are severe systemic issues.
also the lack of update control for that part that (Score:2)
also the lack of update control for that part that crashed.
Re:The Person I Would Hate to Be (Score:4, Insightful)
Re: (Score:2)
I imagine their name will come out at some point during the litigation of this case, and at that point their career and any semblance of a decent life will be over.
Only truly stupid companies wouldn't hire the person. A person is incapable of making a mistake like this. A mistake like this requires a fundamental lack of quality control systems and processes. There is a name you can directly blame. George Kurtz, CEO and Founder of a company who demonstrated that they don't take any basic QA/QC precautions.
Yes, test it - before putting it into service. (Score:2, Insightful)
I'm talking to you, Delta.
Re:Yes, test it - before putting it into service. (Score:5, Informative)
IIRC part of this issue was that Clownstroke does not let you do controlled updates like that.
Re: (Score:2)
https://www.crn.com/news/secur... [crn.com]
Re: (Score:1)
IIRC part of this issue was that Clownstroke does not let you do controlled updates like that.
That just pushes the fault back a step: Delta should never have gotten into such an arrangement.
Re:Yes, test it - before putting it into service. (Score:4, Informative)
I'm talking to you, Delta.
That's not how Crowdstrike channel updates work. Not for Delta not for anyone else. That said there was someone who should have tested something before putting it in service, but you're addressing the wrong side of the court room for this.
Valid if it was just Delta that shit itself. (Score:2)
Re: (Score:2)
Re: (Score:2)
Everyone should be testing before deploying. That fact that it hit so many in modern times shows up the bad practice creeping in.
Re: (Score:3)
https://www.crn.com/news/secur... [crn.com]
Bad practices that vendors force on you.
Re: (Score:2)
EULA (Score:2)
One could guess that the EULA says the software does not have to work and you use it at your own risk.
https://www.crowdstrike.com/en... [crowdstrike.com]
Re: (Score:3)
Re: (Score:2)
A Contact you can't negotiate is called a "contract of adhesion"
"For a contract to be treated as a contract of adhesion, it must be presented on a standard form on a "take it or leave it" basis, and give one party no ability to negotiate because of their unequal bargaining position."
https://en.wikipedia.org/wiki/Standard_form_contract#Contracts_of_adhesion
This is what we see for most consumer contracts.
That said, if two companies are large enough they will negotiate a custom contract. Usually the largest co
Re: (Score:2)
6. No Warranty.
6.1 Disclaimer. THE SOFTWARE AND ALL OTHER CROWDSTRIKE OFFERINGS ARE PROVIDED “AS-IS” AND WITHOUT WARRANTY OF ANY KIND. CROWDSTRIKE AND ITS AFFILIATES DISCLAIM ALL OTHER WARRANTIES, WHETHER EXPRESS, IMPLIED, STATUTORY OR OTHERWISE.
Sounds fairly straight forward. Did Delta not read the agreement?
Make Microsoft accountable too. (Score:2)
Re: (Score:2)
Forced reboots are why many people use unsupported operating systems as they no longer reboot for updates disrupting workflows.
Microsoft doesn't force any reboots. Microsoft displays a warning on the screen that during the next reboot or shutdown an update will apply. Microsoft provides an offline timer to apply updates out of hours when the PC isn't use. Microsoft provide APIs that allow state recovery on reboot, as well as APIs to lock out reboots preventing work being lost.
If you are losing work you definitely have someone to blame, but it's not Microsoft. Start with your own IT department (yes ours forces reboots on our machine
Re: (Score:2)
If your systems are so fragile you can't reboot for an update without going offline you need to look more closely at your systems.
That's called a "single point of failure" and is considered a "bad thing".
At early startups, sure, we crossed our fingers and had plenty of outages but as the companies grew and matured we eliminated every spof we knew about or stumbled into the hard way.
Why are mature companies running known spof critical systems?
Re: (Score:2)
Of course, Crowdstrike does not have that kind of money lying around
Crowdstrike is worth $75billion. Not only do they have $4bn in free cash laying around, they could very easily raise more capital.
Re: (Score:2)
Their annual recurring (I.E. subscription) revenue in 2023 was almost 3.5 billion, with a gross profit margin of 75-80% on that income.
They could (maybe with indigestion) stomach a 5.5 billion loss after several years similar to 2023 in recent history.
backups (Score:2)
How did that Clownstrike failure sneak into Delta's backups/snapshots? Asking for a friend.
Problem is lack of transparency? (Score:2)
Maybe the worst thing is that CS's practices came AS A COMPLETE SURPRISE to everyone,
How many other companies out there are you currently relying on? What are their practices?
Delta, like many, took a decision to trust CS and whatever claims CS made about their products. Well, trust works for the good guys. But it's the bad guys that are the problem. And as the old Marx Brothers joke goes, if you can fake that then you've got it made. The bad companies are deceiving you into blindly trusting them.
The only an