Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
×
The Courts AI Government Privacy

Welfare Surveillance System Violates Human Rights, Dutch Court Rules (theguardian.com) 119

An anonymous reader quotes a report from The Guardian: A Dutch court has ordered the immediate halt of an automated surveillance system for detecting welfare fraud because it violates human rights, in a judgment likely to resonate well beyond the Netherlands. The case was seen as an important legal challenge to the controversial but growing use by governments around the world of artificial intelligence (AI) and risk modeling in administering welfare benefits and other core services. Campaigners say such "digital welfare states" -- developed often without consultation, and operated secretively and without adequate oversight -- amount to spying on the poor, breaching privacy and human rights norms and unfairly penalizing the most vulnerable.

A Guardian investigation in October found the Department for Work and Pensions (DWP) had increased spending to about $10 million a year on a specialist "intelligent automation garage" where computer scientists were developing more than 100 welfare robots, deep learning and intelligent automation for use in the welfare system. The Dutch government's risk indication system (SyRI) is a risk calculation model developed over the past decade by the social affairs and employment ministry to predict the likelihood of an individual committing benefit or tax fraud or violating labour laws. Deployed primarily in low-income neighborhoods, it gathers government data previously held in separate silos, such as employment, personal debt and benefit records, and education and housing histories, then analyses it using a secret algorithm to identify which individuals might be at higher risk of committing benefit fraud.
"A broad coalition of privacy and welfare rights groups, backed by the largest Dutch trade union, argued that poor neighborhoods and their inhabitants were being spied on digitally without any concrete suspicion of individual wrongdoing," the report adds. "SyRI was disproportionately targeting poorer citizens, they said, violating human rights norms."

"The court ruled that the SyRI legislation contained insufficient safeguards against privacy intrusions and criticized a 'serious lack of transparency' about how it worked. It concluded in its ruling that, in the absence of more information, the system may, in targeting poor neighborhoods, amount to discrimination on the basis of socioeconomic or migrant status."
This discussion has been archived. No new comments can be posted.

Welfare Surveillance System Violates Human Rights, Dutch Court Rules

Comments Filter:
  • It wasn’t referred to as an AI system. More often than not most articles have no differentiation between algorithms and actual AI.
    • by AHuxley ( 892839 )
      Depends on the complexity.
      Is the system only looking for citizens paying tax with a past in the tax system?
      ie working, paying some tax and getting serval full gov payments... into the same bank account.. under the same name for years..
      Complex bank accounts used only to cash out gov payments on the day of payment.. with no further bank/gov/support/housing/wrok/medical history under that "fake" name found once in both systems? Once to apply for lots of gov support and one to open the bank account to ac
  • "The court ruled that the SyRI legislation contained insufficient safeguards against privacy intrusions and criticized a 'serious lack of transparency' about how it worked. It concluded in its ruling that, in the absence of more information, the system may, in targeting poor neighborhoods, amount to discrimination on the basis of socioeconomic or migrant status."

    "Why do you rob banks Mr. Sutton ?"
    "That's where the money is"
    ---Willy Sutton.

    Who would of thought that welfare cheats would live in poor neighborhoods or come from groups most likely to be using the system ?

    This is almost as bad as the case in the U.S. where the courts ruled landlords couldn' exclude tennatns based on illegal drug use and other criminal activities.

    • by Anonymous Coward

      This is almost as bad as the case in the U.S. where the courts ruled landlords couldn' exclude tennatns based on illegal drug use and other criminal activities.

      Bullshit. [realestatelawyers.com]

    • Everyone who says "would of" is a dumbass.

      The biggest welfare cheats are corporations.

      • Comment removed based on user account deletion
      • Everyone who says "would of" is a dumbass.

        The biggest welfare cheats are corporations.

        Wow. Well I suppose you at least managed to try deflecting with something marginally on topic, instead of distracting with "Think of all the starving children in Africa".

        It's still pretty funny to see someone trying to take the moral and intellectual high ground with the "But he did worse" defense.

  • This thread took only 10 minutes to become overrrun by trolls.

    Must be some kind of record.

    • This thread took only 10 minutes to become overrrun by trolls.

      Otherwise known as "people I disagree with."

    • This thread took only 10 minutes to become overrrun by trolls.

      Must be some kind of record.

      With large groups and companies all the way up through state actors trying to sway things? And we won't even get into official or unofficial trolling by the web site itself to fish in readers and responders to get additional ad peeps.

  • Is this a joke? (Score:2, Insightful)

    by schwit1 ( 797399 )

    Taxpayers HAVE A RIGHT TO KNOW that their tax money is being spent on its intended purposes and not for fraudulent activities.

    If the recipients don't like it, too bad. Stop asking for a hand out. This goes for welfare recipients and Boeing.

    • Re:Is this a joke? (Score:5, Interesting)

      by drinkypoo ( 153816 ) <drink@hyperlogos.org> on Wednesday February 05, 2020 @09:43PM (#59696072) Homepage Journal

      It always winds up costing more to detect welfare fraud than it's worth. The most logical solution is to institute UBI and then the only fraud you have to detect is people pretending that other people aren't dead so they can collect their UBI. It's cheaper, it's better, and the wealthy can afford to pay for it. They derive the most benefit from the system, so they should pay the most to maintain it. The alternative is torches and pitchforks, tomorrow if not today.

      • How about some cites instead of a bland assertion?
      • by AHuxley ( 892839 )
        Why should a nation grant payments to 100% of their working population for decades when only say 10% are not working for years?
        Paying 100% of a nations workforce "money" every month is not "cheaper" for decades.
        The alternative is to see if a person is not wealthy, not working and can get a full or a part gov payment until they are working again... retire... get a payment for doing education.
    • by ToasterMonkey ( 467067 ) on Wednesday February 05, 2020 @10:38PM (#59696258) Homepage

      Taxpayers HAVE A RIGHT TO KNOW that their tax money is being spent on its intended purposes and not for fraudulent activities.
      If the recipients don't like it, too bad. Stop asking for a hand out. This goes for welfare recipients and Boeing.

      Higher volumes of loss, and concern to taxpayers come from tax evasion.

      These systems should be fed with any available purchase history, credit history, income sources, international data sharing agreements, then basically follow linkedin connections to find other likely culprits, etc. Sounds fair.

    • ... tax money is being spent on its intended purposes ...

      So the AI should be used on public schools, medicare claims, and government contractors, as well. You're already demanding that, right?

      • by AHuxley ( 892839 )
        Re "used on public schools, medicare claims, and government contractors"..
        If the person says they are using the money for "education" from a service approved by the gov...
        Does the person exist? Not a fake ID?
        Does the type and years of education listed by the person exist as gov approved education? No a scam front company to allow people to request years of education payments.
        Is the person attending as they said they would when accepting the full "education" payment from the gov...
        ie the needed full t
      • Sure, why not? Who exactly did you think was in favor of some sort of fraud? *boggle*
    • Re:Is this a joke? (Score:4, Insightful)

      by AmiMoJo ( 196126 ) on Thursday February 06, 2020 @04:46AM (#59696876) Homepage Journal

      Most of the people on benefits have paid into the system, i.e. it's more like an insurance pay-out than a hand-out.

      And no, privacy is a human right in Europe so taxpayer concerns about how their money is spent can't override that. In practice it's a balance but anything too invasive is not acceptable, as the court has confirmed.

    • Does it also apply to government agencies who use taxpayer money to develop something as broad in scope as this without putting that decision in front of voters?

  • by Chas ( 5144 )

    Since when has defrauding welfare been a human right?

    • by Sabriel ( 134364 )
      That wasn't the violation. The violation was that the surveillance system was equating "they're poor" with "they must be committing fraud" and proceeding to run detailed searches of databases that are normally kept separate, targeting low-income areas without reasonable cause to suspect a specific crime, all without a warrant.

      For a car analogy, it would be as if the police decided to automate searching the cars of everyone who happened to be driving while poor because someone, somewhere, must be speeding
  • by thegarbz ( 1787294 ) on Thursday February 06, 2020 @05:26AM (#59696930)

    Everything in the Netherlands gets overturned based on some complaint about discrimination the first pass. This is often abused as it's in Article 1 of the Constitution for the government to treat everyone equally.

    This law and practice will be fine tuned and rolled out again. The Dutch love suing their government on basically every issue, and ultimately all that happens is the government makes a slight change and then it's all quiet from that point onward.

    Other classic examples:
    - Trying to ban mopeds from inner city bike paths for safety reasons. Oh but it turns out the majority of moped drivers are immigrants, and the majority of cyclists are white Dutch folk. Well we can't ban mopeds on bike paths because we're discriminating against immigrants. This went through the courts for a full year before Amsterdam finally got it through.
    - Changes to the tax exemption were overturned based on discrimination of wealth. They government couldn't just arbitrarily stop the expat tax discount as it didn't affect all expats equally, so they had to change the law to create a 3 stage phase in.

  • There is no two ways about it: We're already living in a post-scarcity economy. I'm dropping out of my current gig consisting of a baseline bullshit job interspersed with me trying to explain to my POs and PMs how the internet and software projects fundamentally work and sometimes get to write specs or some gluecode for the software components that we buy and plug together. This is 20% of my 4/5th dayjob, the rest consists of me sitting around and ready articles on Microservice architecture and trying out v

  • I was told some years ago, that the DWP cost £30 for every £1 paid out in benefits because the system is so monstrously inefficient.

    Does anyone have a recent "cost/benefit analysis"?

    Given the enormous cost of "AI", and the probability it will not work as expected - its a Government IT project, I am sure the bookies will give good odds it will never actually work at all - it seems that "Gordon Brown knows how to spend your money better than you do!" is likely to be out done by "Boris can squa

  • What a shock!

    Seriously, that's a stupid argument. Of course the poor are going to be disproportionately affected by this. That doesn't mean it's bad. That doesn't mean it violates anyone's rights. I don't even see how this can be called surveillance, since it appears to involve little more than scraping data from various pre-existing government databases! That's a government using it's own data to predict the likelihood that the recipient of benefits it provides might cheat.

    I don't know how the D

To be is to program.

Working...