Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
×
The Courts Bug United Kingdom

Despite 16-Year Glitch, UK Law Still Considers Computers 'Reliable' By Default (theguardian.com) 96

Long-time Slashdot reader Geoffrey.landis writes: Hundreds of British postal workers wrongly convicted of theft due to faulty accounting software could have their convictions reversed, according to a story from the BBC. Between 1999 and 2015, the Post Office prosecuted 700 sub-postmasters and sub-postmistresses — an average of one a week — based on information from a computer system called Horizon, after faulty software wrongly made it look like money was missing. Some 283 more cases were brought by other bodies including the Crown Prosecution Service.
2024 began with a four-part dramatization of the scandal airing on British television, and the BBC reporting today that its reporters originally investigating the story confronted "lobbying, misinformation and outright lies."

Yet the Guardian notes that to this day in English and Welsh law, computers are still assumed to be "reliable" unless and until proven otherwise. But critics of this approach say this reverses the burden of proof normally applied in criminal cases. Stephen Mason, a barrister and expert on electronic evidence, said: "It says, for the person who's saying 'there's something wrong with this computer', that they have to prove it. Even if it's the person accusing them who has the information...."

He and colleagues had been expressing alarm about the presumption as far back as 2009. "My view is that the Post Office would never have got anywhere near as far as it did if this presumption wasn't in place," Mason said... [W]hen post office operators were accused of having stolen money, the hallucinatory evidence of the Horizon system was deemed sufficient proof. Without any evidence to the contrary, the defendants could not force the system to be tested in court and their loss was all but guaranteed.

The influence of English common law internationally means that the presumption of reliability is widespread. Mason cites cases from New Zealand, Singapore and the U.S. that upheld the standard and just one notable case where the opposite happened... The rise of AI systems made it even more pressing to reassess the law, said Noah Waisberg, the co-founder and CEO of the legal AI platform Zuva.

Thanks to Slashdot reader Bruce66423 for sharing the article.
This discussion has been archived. No new comments can be posted.

Despite 16-Year Glitch, UK Law Still Considers Computers 'Reliable' By Default

Comments Filter:
  • Criminal fraud? (Score:5, Interesting)

    by whoever57 ( 658626 ) on Saturday January 13, 2024 @04:45PM (#64156441) Journal

    The Post Office "Investigators" knew that there were many reports of problems with the Horizon system.

    They lied to postmasters about the number of cases.

    They presented falsehoods to get the postmasters to compensate the Post Office.

    Isn't that fraud (obtaining money by deception)? How about some criminal prosecutions for the "investigators"?

    There is also a case where a postmaster was convicted on killing his wife, with the motive being the apparent shortfall in the accounts (really, bugs or deliberate theft by Fujitsu) and, in that case, where it benefited the Post Office, the Post Office investigator said that the time that claims of shortfalls being due to the Horizon system were very common -- in direct conflict with what they were telling other postmasters, in order to obtain confessions and defraud them of money.

    • Though it feels like they are taking a long time about it. Let's hope for some senior people get heavily punished.

    • The pattern of fake cases would've been pretty obvious pretty quick but it doesn't sound like the investigations stopped. I don't know about British politics but here in the States if you saw something like that it would be a political purge of some kind.

      That said our postal service is heavily politicized right now because our right wing party is undermining it to privatize it.
      • by MeNeXT ( 200840 )

        but here in the States if you saw something like that it would be a political purge of some kind.

        Really? Did you see the politics in the Sates? Take your head out of the sand.

      • by VeryFluffyBunny ( 5037285 ) on Sunday January 14, 2024 @03:05AM (#64157229)
        The Post Office was effectively privatised, i.e. set up to run as a privatised company in 1987. The "Post Masters" that were prosecuted were essentially franchisees, independent businesses subcontracting to Post Office Counters LTD.

        This scandal is political only in the sense that the Crown Prosecution Service & the UK courts abandoned the presumption of innocence thereby denying the defendants the right to a fair trial. It smacks more of incompetence than conspiracy but then there may well have been conspiracy in the cover-up. It's always the cover-up where the worst crimes are committed.
        • Political also in the sense that the Post Office is its own prosecuting authority in England (and possibly Wales, not Scotland), and brought most of the prosecutions itself. It looks like they are going to lose their prosecuting powers in England. I haven't been following proceedings in the Senedd so I don't know what is happening in Wales.

        • by jvkjvk ( 102057 )

          >It's always the cover-up where the worst crimes are committed.

          Hardly. However, it's often times the only thing that gets prosecuted. Sorry but the crimes are generally much worse that lying.

        • by AmiMoJo ( 196126 )

          Another political aspect is that a few years back the government changed the law so that people who are wrongly convicted and sent to prison can only get compensation if they can prove their innocence. They might be let out because the conviction is based on evidence that is later refuted, as is the case here were the software has been discredited, but they have to reach the an almost impossible standard of proof of innocence to get any compensation.

          As such most of the affected people will have to sue to ge

  • What about the software running on those computers?
    • UK can stop spending money on software upgrades.

      What they have is presumed to be reliable.

      (dumb judges don't like difficult cases)

  • by Revek ( 133289 ) on Saturday January 13, 2024 @04:52PM (#64156455)
    Watch Mr Bates vs The Post Office: The Real Story I watched this just yesterday and its criminal how these people were treated.
    • by jhoegl ( 638955 )
      What should be happening is, the software puts out suspicion, and they investigate. Not rely on it to prosecute.

      How lazy and terrible.
      • This sounds like one of those Mr. Buttle vs Mr Tuttle errors; "Sorry for the mess. Now go and try to mend your broken lives with the meagre compensation we're willing to pay you. In order to receive said compensation, you must sign this document, absolving us of all future responsibility."
  • "Let me put it this way...The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error." - HAL 9000
  • ... render their reliable judgements. They will certainly find that computers are, indeed, inerrant.
  • by bloodhawk ( 813939 ) on Saturday January 13, 2024 @05:03PM (#64156471)
    computers are reliable and that is a good default position. What is not reliable is the companies and people writing software for them.
    • by RightwingNutjob ( 1302813 ) on Saturday January 13, 2024 @05:13PM (#64156497)

      Any piece of equipment or software is a piece of shit Potemkin artifact made of duct tape and bailing wire by the lowest bidder, unless exhaustively tested to conform to specification.

      • by LindleyF ( 9395567 ) on Saturday January 13, 2024 @05:27PM (#64156521)
        • Could you point actual fraud made with electronic voting. And please do not come with Venezuela as an example. The fraud there had nothing to do with the voting machines.

          • You totally missed the joke.
          • Every time these voting machines are scrutinized by experts, lots of vulnerabilities are found.

            More importantly, we do not need them. We can accurately hand-count paper ballots, and they offer less potential for malfeasance.

            This is one thing that security-minded people on "both sides" of the aisle should be able to agree on, unlike mail-in ballots — and for the record, I am in favor of absentee voting.

            • It doesn't even matter if computer voting were more reliable. It's not auditable; you can't have someone verify that the code is bug-free and wasn't swapped out nor overridden by the hardware, which you also can't verify. Also it's a lot easier to argue that someone swapped out some thumb drives rather than that they created and hid in their pocket 1,500,000 kg of paper.

              Elections don't just have to be accurate, people have to trust them.

              • As if you could audit paper ballots. Once you cast it, you cannot be sure of what is done with it. Someone with a small truck could swap 3,500 kg of paper, which would amount to 1,5 million votes, if each ballot was an ISO A5 piece of paper (It must be enough, I think. I've never seen a paper ballot.). The people counting votes could miscount them intentionally, or forced to do so. After the votes are counted manually, do you think they are summed up manually again? What if someone tamper the program that m

                • As you said, someone with a truck vs someone with a pocket. It's also harder to sneak a truck into a voting booth than a pocket. Everything that makes paper a heavy, bulky, expensive, annoying pain in the ass also works as a minor security improvement. My area uses machine-counted paper ballots, with occasional manual recounts.

            • by flink ( 18449 )

              You can do both. Have the evoting machine spit out a marked paper ballot. The voter can the visually inspect this ballot for accuracy before handing it in. If there is any question about the accuracy of the electronic total, just tally the paper ballots. They can even be made machine readable to speed recounts.

    • by Calydor ( 739835 )

      Reliable is fine. Assuming they're infallible is not.

    • They generally seem to be better behaved than the software that runs on them; but computers absolutely aren't reliable(especially the ones that skip things like ECC and storage medium redundancy). What's even worse is that(unlike software, which at least in principle can be correct, even if it's generally uneconomic to write it at the level of formal verification and people don't bother) hardware fails unpredictably. Some particularly bad designs or defective components can make certain failures so overwhel
      • reliable doesn't mean perfect or infallible. when correctly managed with proper redundancy, logging, auditing etc there is almost zero concern of them making errors that affect the results. However the software running on them in any large system will have a large amount of known and unknown errors and trusting the software is doing exactly the right thing is infinitely harder and more expensive.
      • by Calydor ( 739835 )

        It's no different from saying that any given person is reliable. It means they are known to not be prone to lying or embellishing events. It does not mean that they can't misremember something. Basically; unless someone speaks directly against them you can probably assume that what they say is the truth without further digging.

        And yes, such assumptions are necessary even in a trial. Otherwise you start having to verify if the witness tells the truth, and if the guy verifying is telling the truth about the w

        • by jabuzz ( 182671 ) on Sunday January 14, 2024 @08:24AM (#64157487) Homepage

          If a witness says he saw a person in New York at 10am and on the same day he also saw a different person in Chicago at 11am we can reasonably assume he is a liar and his testimony should be ignored.

          In the Horizon scandal the *ONLY* evidence that the money was missing was the computer system and the Post Office/Fujitsu claimed it was 100% bug free and reliable. *NO* software system is 100% bug free and reliable so they obviously lied.

          Hell in one the cases 30,000GBP of stamps where suddenly missing from a Post Office branch. This is when a first class stamp cost less than 50p so you are talking well over 60,000 stamps magically went missing. That is a huge quantity of physical stuff to go missing and they could not provide any evidence the Post Office had even been delivered that amount of stamps which is over a years supply gone missing. The Judges should have been throwing that sort of evidence out as preposterous.

    • by kmoser ( 1469707 )
      So if I create a computer with proprietary hardware and code that cannot be easily reverse engineered, and make it print "the Queen of England is guilty of jaywalking" then it is assumed to be correct until proven otherwise? What could possibly go wrong?
    • There have been lots of instances of hardware and firmware bugs.

  • by seebs ( 15766 ) on Saturday January 13, 2024 @05:05PM (#64156473) Homepage

    (source: https://infosec.exchange/@Goss... [infosec.exchange])

    Been watching the Post Office scandal drama that ITV made. It’s really good.

    One big notable so far - they introduce the CEO in episode two.. skipping over the CEO before her, who was actually in charge when all this kicked off.

    He went on to be CEO of ITV.

    • The show may have been misleading but the underlying system and problems with it were definitely not. I mean the Wikipedia entry listing just the court cases and the outcomes is more detailed that those for whole countries and their histories https://en.wikipedia.org/wiki/... [wikipedia.org]. There was a *LOT* which happened here, and very little of it was good.

  • by Bruce66423 ( 1678196 ) on Saturday January 13, 2024 @05:38PM (#64156553)

    Fujitsu wrote the system and were actively involved in its maintenance. This 'maintenance' included direct, unaudited, access to the live data files of a financial system. I believe it was the revelation that this was happening that finally derailed the whole pattern of abuse; if you ever discover that your company allows this to critical, court ready data run away, very very fast.

    • by flink ( 18449 )

      This was pretty routine when I worked at a claims clearinghouse. We had routine access to claims and remits at 100s of hospitals. Not saying it was good, but it was a common practice when outsourcing this stuff.

    • That is silly. Name and shame what actually cause the problem: A lowest bidder, ill defined contract by someone who had no idea how to define what they wanted, for a poorly run project.

      It doesn't matter who is the vendor. There's a reason why government run IT projects end in a colossal disaster over and over again while the same contractors have zero problems delivering for other projects. Pointing the finger at any one person or company is completely and utterly failing to address the cause of the problem

  • by NomDeAlias ( 10449224 ) on Saturday January 13, 2024 @05:44PM (#64156565)
    The defense should be able to review every single line of code.
  • And "Phoenix" (Score:5, Informative)

    by ve3oat ( 884827 ) on Saturday January 13, 2024 @06:07PM (#64156605) Homepage
    And then there is the Canadian federal government's Phoenix pay system (https://spectrum.ieee.org/canadian-governments-phoenix-pay-system-an-incomprehensible-failure) designed and implemented by IBM without proper testing (no parallel operation with the old, existing system which was fault-free) and which inflicted financial damage and emotional anxiety upon thousands of federal civil servants, and which is not fixed even yet!
    I am so glad that the department that I retired from was not, at least at that time, using Phoenix.
    • by Anonymous Coward

      I'm posting anon because I don't want my wife to lose her job. We can't lay all the blame at IBM's feet: yeah, the system is shot and a lot of it is their fault, but the problem is they aren't the whole issue. The biggest problem was Judy Foote, who was Minister of Public Services and Procurement Canada, who ignored IBM's pleas to delay the rollout of Phoenix because they were still fixing bugs and various issues. She was desperate to get it launched under her tenure as Minister, and the delay would have me

  • by Mirnotoriety ( 10462951 ) on Saturday January 13, 2024 @06:08PM (#64156609)
    Report on the EPOSS PinICL Task Force [postoffice...iry.org.uk]:

    ‘This extract from EPOSSCore.d11 has been written to reverse the sign of a number and is equivalent to the command : d = - d’

    -------

    Public Function ReverseSign(d)

    If d
    Else d = d - (d * 2)

    End

    If ReverseSign = d

    End Function

    -------

    ‘Whoever wrote this code clearly has no understanding of elementary mathematics or the most basic rules of programming.’
  • and FDIV
  • by Bruce66423 ( 1678196 ) on Saturday January 13, 2024 @07:12PM (#64156727)

    Scary

    https://www.bbc.co.uk/news/uk-... [bbc.co.uk]

    Post Office executives appear to have lied on camera. The only real solution to this is to have all statements by such people to be made as legal depositions so that if they are later proved to have lied, they can be quickly imprisoned for perjury.

    • by ufgrat ( 6245202 ) on Sunday January 14, 2024 @05:55AM (#64157341)

      This is the real issue-- It's not that the computer was wrong, and people went to jail-- which did happened.

      It's that Fujitsu and the Royal Mail KNEW there was a problem with their software, and they covered it up for TWENTY YEARS. This wasn't the work of a single corrupt person, or a bad policy-- this was a systematic cover-up of a badly written piece of software that was deemed secure and infallible, and in fact, was neither.

      Lives were destroyed. At least four people falsely accused committed suicide.

      Ian Hislop deserves a medal for the work he and Private Eye have put into this case.

    • by PJ6 ( 1151747 )

      Post Office executives appear to have lied on camera. The only real solution to this is to have all statements by such people to be made as legal depositions so that if they are later proved to have lied, they can be quickly imprisoned for perjury.

      The real issue here is that nobody will be prosecuted.

  • thats a very reliable glitch
  • by sjames ( 1099 ) on Saturday January 13, 2024 @07:29PM (#64156755) Homepage Journal

    The single most valuable asset of the courts (in any country) is the faith, respect and trust of the public. Once those erode, even the most extraordinary effort may only partially restore them. Once fully eroded, the courts, police, and the government behind them are in for a long slow bloody death.

    Police and the courts in the U.S. risk the same with "civil asset forfeiture" and minimally punished prosecutorial misconduct

    • So not the summary executions of citizens on the grounds of being young, male, & black?
      • by ufgrat ( 6245202 )

        That's a different problem. I'm not trying to minimize it, but to characterize that as an execution is disingenuous-- that is murder. And it's becoming increasingly considered as such. It's not legal, even though police officers do get away with it far too often.

        What @sjames is referring to is that regardless of your race, for instance, if the police can prove your vehicle transported as little as a couple grams of a schedule 1 drug (even if the drug in question doesn't meet the standard for schedule 1),

        • And it's also illegal, they are doing things explicitly forbidden to the Federal government but pretending it's allowed by the Interstate Commerce clause.

          • by ufgrat ( 6245202 )

            They're getting away with it to the tune of BILLIONS of dollars.

            Apparently no one told them it's illegal.

        • Well, I guess it doesn't affect you directly so it's not your problem. I think there are others who would disagree with you on the grounds of effect on trust in the law. The USA already has large numbers of US citizens who have zero trust in the law. BTW, I think it's reasonable to call deliberate, unlawful killing by law enforcement "summary execution." How would you feel if one or members of you family &/or community had been summarily executed by the police on the basis of your physical appearance
          • by sjames ( 1099 )

            It's a terrible problem for certain, but unless and until it is understood to be TWO problems or even three, it won't tear the whole society apart or bring the government down.

            The primary problem is that too many cops have become violent murderous goons. They will get violent with ANYONE of any race if they think they can get away with it. The violence and being quick to use lethal force not a racism problem. That they believe (and sadly correctly) that they are more likely to get away with it if the victim

            • The law has always been applied "unevenly" & "inequitably" in every country since 3rd party arbitration & police forces were invented. Democracy's supposed to make us more equal but that unevenness & inequity will unfortunately always be a feature & it certainly isn't a risk that'd bring the whole system down. Something as simple as food shortages would, though.
    • The single most valuable asset of the courts (in any country) is the faith, respect and trust of the public.

      In that case, the UK is certainly doomed. (As is the USA).

  • by ciascu ( 1345429 ) on Saturday January 13, 2024 @07:53PM (#64156787) Journal

    Around the time of Horizon Online appearing in 2010 (not original pre-2k Horizon), the UK government, also the sole shareholder of the Post Office, was at the cutting edge [parliament.uk] of bringing open source into public procurement and increasing technical transparency.

    What surprises me is that, at this point, there is very little questioning of whether single-purpose government contract closed source is automatically the right choice - there's a case for open and for closed, but Github is a lot easier to access for an expert witness. While there were also issues with the reference data, it is hard to think of a starker example of why having access for third-parties, even just to the code, matters (yes ironic, but did a Medium [medium.com] article few days back if anyone wants some more context links for this point).

    This has become my go-to answer to "whether government money means public code only impacts technical people".

    • The EU has a directive (an order to member states to legislate for a purpose so that policies between member states are coordinated & compatible) to prefer procurement of free and open source software (FOSS) wherever feasible & that the burden of proof must be of no feasible FOSS option being available. Many EU govt agencies have switched or are in the process of switching to FOSS & benefiting from the advantages of openness, standards & interoperability that FOSS typically offers, e.g. Spai
      • by Anonymous Coward
        None of those benefits are from FOSS, they are from free and open standards. FOSS is just an easy way to leverage them, the exact same result could be had with closed source that adheres to the standards.
  • "Without any evidence to the contrary, the defendants could not force the system to be tested in court"

    I don't get it... even if it was considered reliable by default, why would you not be able to test the system in court? There's a lot of things that the court might consider true by default, but you can still challenge them.

    Like, take official records, say the list of births, deaths and marriages... It would take a lot to challenge something in that, the records are kept carefully, nobody generally has any

  • Have they ever met a computer?
  • Run the following code on a computer:
    print "The other computer is unreliable"
    Now the burden of proof is on the alleged unreliable computer's operator or creator to prove that it is in fact reliable, since the computer running the code above is presumed correct unless proven otherwise (which would require proving the alleged unreliable computer is in fact reliable). If you remove the presumption that computers are always correct, then again, a proof is required to show the computer reliability.
  • That not one of them went postal.

    Blame it on the tech. 16 years, come on -- it doesn't deserve a pass - not one bit. Pun necessary, and intended

  • by WaffleMonster ( 969671 ) on Sunday January 14, 2024 @10:12AM (#64157685)

    So some proverbial guilty light on some machine lights up... didn't people have a right to the underlying information that informed the bulb lighting up in their legal defenses?

  • computers might be reliable but what goes into it is not always so. As they say, garbage in, garbage out and that is what happened in the UK.

  • A glitch is a transient voltage spike, the bane of analog anything. "Journalists" have expanded the term to mean anything outside of human control, because in the game of journalizm, using the latest cool word is worth 20 points. This was not a glitch, it was a bug. Also, the word is now 60 years old in the public vocabulary; no longer cool.

Make sure your code does nothing gracefully.

Working...