Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
The Courts Software

Delta Sues CrowdStrike Over Software Update That Prompted Mass Flight Disruptions (reuters.com) 44

Posted by BeauHD from the fasten-your-seatbelts dept.
An anonymous reader quotes a report from Reuters: Delta Air Lines on Friday sued cybersecurity firm CrowdStrike in a Georgia state court after a global outage in July caused mass flight cancellations, disrupted travel plans of 1.3 million customers and cost the carrier more than $500 million. Delta's lawsuit filed in Fulton County Superior Court called the faulty software update from CrowdStrike "catastrophic" and said the firm "forced untested and faulty updates to its customers, causing more than 8.5 million Microsoft Windows-based computers around the world to crash." [...]

Delta, which has purchased CrowdStrike products since 2022, said the outage forced it to cancel 7,000 flights, impacting 1.3 million passengers over five days. "If CrowdStrike had tested the faulty update on even one computer before deployment, the computer would have crashed," Delta's lawsuit says. "Because the faulty update could not be removed remotely, CrowdStrike crippled Delta's business and created immense delays for Delta customers." Delta said that as part of its IT-planning and infrastructure, it has invested billions of dollars "in licensing and building some of the best technology solutions in the airline industry."

Delta Sues CrowdStrike Over Software Update That Prompted Mass Flight Disruptions More | Reply

Delta Sues CrowdStrike Over Software Update That Prompted Mass Flight Disruptions

Comments Filter:

  • Damages never more than the software costs.

    • These exclusions donâ(TM)t apply to things like gross negligence or fraud, and whether or not the event was one of those is a question of fact for a jury to decide.

      • Re: (Score:2)

        by DingerX ( 847589 )
        The sticky parts here:
        Delta is demanding from its contractor reparations for damages that it's refusing its customers. When they reject claims for which they have responsibility, how can they go after their contractors on the same basis?
        Delta is not the only airline to use CrowdStrike, and they all had outages on July 19. Delta, however, is the only one that couldn't recover until July 24.

        Crowdstrike's test and deployment processes to me look like gross negligence: their business is having companies ent

        • The process that delta uses may not be relevant. What matters is what Crowdstrike said its processes are, what the contract between delta and Crowdstrike says, and what a reasonable person would do based on those assurances. Crowdstrike publicly admitted that it did not test the update. If the contract between delta and Crowdstrike says that Crowdstrike shall test all updates before deployment on infrastructure that duplicates delta infrastructure because delta doesnt want to do that testing itself, Crowdst

          • I didn't follow this one beyond the basics. Did crowdstrike really admit in public or to Delta lawyers that they didn't test before releasing??

            • I'm not sure if they publicly admitted it like the OP stated, but it would be hard to argue to anyone with even half a brain that they did test. When your update fails in virtually 100% of the cases, you're going to have to show some extraordinary proof that you tested it and the test(s) passed.

          • Re: (Score:2)

            by mspohr ( 589790 )

            Does the contract really have to say that Corowdstrite must test updates?
            Isn't that just assumed?
            I mean, isn't it just basic software development practice to actually test software before you release it?

        • Delta is not the only airline to use CrowdStrike, and they all had outages on July 19. Delta, however, is the only one that couldn't recover until July 24.

          That's one of the strongest parts of the defense against the bulk of the lawsuit.

          The hours immediately after the issue? Absolutely the company is on the hook there, and their insurance is already paying out a fortune for those claims. No question about the liability there.

          But after that? While the core systems were down for hours airlines had to move things around. All the other airlines were out for about an additional day to reposition their flight crews and aircraft. If you want to look at what is rea

        • When they reject claims for which they have responsibility, how can they go after their contractors on the same basis?

          I don't know why they decided to reject those claims, but I'm pretty sure I know why they're going after Cloudstrike this way. The suit is being put together by lawyers and they're going to start off demanding everything they can think of so that they can use the more outlandish claims as bargaining chips.

  • The Person I Would Hate to Be (Score:3, Interesting)

    by crunchy_one ( 1047426 ) on Saturday October 26, 2024 @09:17AM (#64895649)
    I would not like to be the person who authored the bad patch at CrowdStrike. Not one bit. I imagine their name will come out at some point during the litigation of this case, and at that point their career and any semblance of a decent life will be over. Never mind that they worked for a company that clearly had just about the worst release process, if you can call it that, on the planet. This situation calls for a scapegoat and that lone coder will be it.

    • It's possible that the defendant's lawyers may try to cast shadow on the coder, but the court is likely to consider all aspects of the event, including very poor processes by the company at large. After all, any revision and release processes need to take into account problematic or dangerous updates, and so while you might be able to blame a coder or a code group of buggering it up, the update was green lit, suggesting there are severe systemic issues.

    • Re:The Person I Would Hate to Be (Score:4, Insightful)

      by bjoast ( 1310293 ) on Saturday October 26, 2024 @10:21AM (#64895727)
      This guy's name may come out, yes, but I think you are exaggerating the industry's perception of who was responsible for this disaster. This was caused by a systemic problem linked to CrowdStrike's inadequate integration testing, not just the result of one developer's mistake. Based on the information available, most people should understand that there exists no reason to blacklist him from the profession, as you seem to believe.

    • I imagine their name will come out at some point during the litigation of this case, and at that point their career and any semblance of a decent life will be over.

      Only truly stupid companies wouldn't hire the person. A person is incapable of making a mistake like this. A mistake like this requires a fundamental lack of quality control systems and processes. There is a name you can directly blame. George Kurtz, CEO and Founder of a company who demonstrated that they don't take any basic QA/QC precautions.

    • "I would not like to be the person who authored the bad patch at CrowdStrike."

      I would not like to be the person who authored a parser with no input validation,
      I would not like to be the person who put code with no error handling in an environment where an error can bring down the entire machine.

      There are many things that had to go wrong, and a bad patch is neither necessary nor sufficient to cause this scenario.

    • Most software development teams in 2024 have plenty of change management processes to ensure that a single incompetent or rogue developer can't single-handedly push an update that breaks the system. It's extremely common for all software changes (and yes, that includes config) to require approval from multiple other developers, pass many unit tests, and in some cases pass automated integration tests. In addition to that, the company could have deployed the changes in waves instead of all at once. A failu

  • I'm talking to you, Delta.

  • But it was everyone. Crowdstrike was clearly the problem.

  • One could guess that the EULA says the software does not have to work and you use it at your own risk.

    https://www.crowdstrike.com/en... [crowdstrike.com]

    • Which is great! We can strike EULA's out of law, as they should be. A contract you can't negotiate isn't a contract.

      • Re: (Score:2)

        by hwstar ( 35834 )

        A Contact you can't negotiate is called a "contract of adhesion"

        "For a contract to be treated as a contract of adhesion, it must be presented on a standard form on a "take it or leave it" basis, and give one party no ability to negotiate because of their unequal bargaining position."

        https://en.wikipedia.org/wiki/Standard_form_contract#Contracts_of_adhesion

        This is what we see for most consumer contracts.

        That said, if two companies are large enough they will negotiate a custom contract. Usually the largest co

        • Honest question: does the law state that it's legal for the creator of the contract of adhesion to alter the terms whenever they want as a take-it-or-leave-it opt-out agreement and that all changes are legally binding (assuming they don't violate any other laws)?

    • 6. No Warranty.

      6.1 Disclaimer. THE SOFTWARE AND ALL OTHER CROWDSTRIKE OFFERINGS ARE PROVIDED “AS-IS” AND WITHOUT WARRANTY OF ANY KIND. CROWDSTRIKE AND ITS AFFILIATES DISCLAIM ALL OTHER WARRANTIES, WHETHER EXPRESS, IMPLIED, STATUTORY OR OTHERWISE.

      Sounds fairly straight forward. Did Delta not read the agreement?

      • > Sounds fairly straight forward

        That's not how the law works.

        Unconscionable terms and acts of fraud and gross negligence are frequently found to be outside of contract terms.

        It's far from straightforward. Partially self-serving on the part of lawyers.

      • They can disclaim statutory warranties all they want, but the law says that they still apply.
  • Forced reboots are why many people use unsupported operating systems as they no longer reboot for updates disrupting workflows.

    • Forced reboots are why many people use unsupported operating systems as they no longer reboot for updates disrupting workflows.

      Microsoft doesn't force any reboots. Microsoft displays a warning on the screen that during the next reboot or shutdown an update will apply. Microsoft provides an offline timer to apply updates out of hours when the PC isn't use. Microsoft provide APIs that allow state recovery on reboot, as well as APIs to lock out reboots preventing work being lost.

      If you are losing work you definitely have someone to blame, but it's not Microsoft. Start with your own IT department (yes ours forces reboots on our machine

    • If your systems are so fragile you can't reboot for an update without going offline you need to look more closely at your systems.

      That's called a "single point of failure" and is considered a "bad thing".

      At early startups, sure, we crossed our fingers and had plenty of outages but as the companies grew and matured we eliminated every spof we knew about or stumbled into the hard way.

      Why are mature companies running known spof critical systems?

  • How did that Clownstrike failure sneak into Delta's backups/snapshots? Asking for a friend.

  • Maybe the worst thing is that CS's practices came AS A COMPLETE SURPRISE to everyone,

    How many other companies out there are you currently relying on? What are their practices?

    Delta, like many, took a decision to trust CS and whatever claims CS made about their products. Well, trust works for the good guys. But it's the bad guys that are the problem. And as the old Marx Brothers joke goes, if you can fake that then you've got it made. The bad companies are deceiving you into blindly trusting them.

    The only an

  • CrowdStrike was at fault, but Delta owns the blame here. Delta had no way to recover in a timely manner; Delta didn't demand a Test QA before promoting changes to Prod. A $5 "break glass" USB could have been in a ziplock-type bag attached to each PC to boot to a base image and get back online; there are dozens of remote management recovery solutions available.

  • They are both Delaware corporations, I would think the first thing CrowdStrike is going to do is point out this obvious fact. Delta is obviously going for "Home Field Advantage" since their headquarters is in Atlanta, but the correct state for this suit should be Delaware.

Slashdot Top Deals

Try `stty 0' -- it works much better.

Close