'Why Are Cops Around the World Using This Outlandish Mind-Reading Tool?' (propublica.org) 93
ProPublica has determined that dozens of state and local agencies have purchased "SCAN" training from a company called LSI for reviewing a suspect's written statements -- even though there's no scientific evidence that it works.
Local, state and federal agencies from the Louisville Metro Police Department to the Michigan State Police to the U.S. State Department have paid for SCAN training. The LSI website lists 417 agencies nationwide, from small-town police departments to the military, that have been trained in SCAN -- and that list isn't comprehensive, because additional ones show up in procurement databases and in public records obtained by ProPublica. Other training recipients include law enforcement agencies in Australia, Belgium, Canada, Israel, Mexico, the Netherlands, Singapore, South Africa and the United Kingdom, among others...
For Avinoam Sapir, the creator of SCAN, sifting truth from deception is as simple as one, two, three.
1. Give the subject a pen and paper.
2. Ask the subject to write down his/her version of what happened.
3. Analyze the statement and solve the case.
Those steps appear on the website for Sapir's company, based in Phoenix. "SCAN Unlocks the Mystery!" the homepage says, alongside a logo of a question mark stamped on someone's brain. The site includes dozens of testimonials with no names attached. "Since January when I first attended your course, everybody I meet just walks up to me and confesses!" one says. [Another testimonial says "The Army finally got its money's worth..."] SCAN saves time, the site says. It saves money. Police can fax a questionnaire to a hundred people at once, the site says. Those hundred people can fax it back "and then, in less than an hour, the investigator will be able to review the questionnaires and solve the case."
In 2009 the U.S. government created a special interagency task force to review scientific studies and independently investigate which interrogation techniques worked, assessed by the FBI, CIA and the U.S. Department of Defense. "When all 12 SCAN criteria were used in a laboratory study, SCAN did not distinguish truth-tellers from liars above the level of chance," the review said, also challenging two of the method's 12 criteria. "Both gaps in memory and spontaneous corrections have been shown to be indicators of truth, contrary to what is claimed by SCAN."
In a footnote, the review identified three specific agencies that use SCAN: the FBI, CIA and U.S. Army military intelligence, which falls under the Department of Defense...
In 2016, the same year the federal task force released its review of interrogation techniques, four scholars published a study on SCAN in the journal Frontiers in Psychology. The authors -- three from the Netherlands, one from England -- noted that there had been only four prior studies in peer-reviewed journals on SCAN's effectiveness. Each of those studies (in 1996, 2012, 2014 and 2015) concluded that SCAN failed to help discriminate between truthful and fabricated statements. The 2016 study found the same. Raters trained in SCAN evaluated 234 statements -- 117 true, 117 false. Their results in trying to separate fact from fiction were about the same as chance....
Steven Drizin, a Northwestern University law professor who specializes in wrongful convictions, said SCAN and assorted other lie-detection tools suffer from "over-claim syndrome" -- big claims made without scientific grounding. Asked why police would trust such tools, Drizin said: "A lot has to do with hubris -- a belief on the part of police officers that they can tell when someone is lying to them with a high degree of accuracy. These tools play in to that belief and confirm that belief."
SCAN's creator "declined to be interviewed for this story," but they spoke to some users of the technique. Travis Marsh, the head of an Indiana sheriff's department, has been using the tool for nearly two decades, while acknowledging that he can't explain how it works. "It really is, for lack of a better term, a faith-based system because you can't see behind the curtain."
Pro Publica also reports that "Years ago his wife left a note saying she and the kids were off doing one thing, whereas Marsh, analyzing her writing, could tell they had actually gone shopping. His wife has not left him another note in at least 15 years..."
For Avinoam Sapir, the creator of SCAN, sifting truth from deception is as simple as one, two, three.
1. Give the subject a pen and paper.
2. Ask the subject to write down his/her version of what happened.
3. Analyze the statement and solve the case.
Those steps appear on the website for Sapir's company, based in Phoenix. "SCAN Unlocks the Mystery!" the homepage says, alongside a logo of a question mark stamped on someone's brain. The site includes dozens of testimonials with no names attached. "Since January when I first attended your course, everybody I meet just walks up to me and confesses!" one says. [Another testimonial says "The Army finally got its money's worth..."] SCAN saves time, the site says. It saves money. Police can fax a questionnaire to a hundred people at once, the site says. Those hundred people can fax it back "and then, in less than an hour, the investigator will be able to review the questionnaires and solve the case."
In 2009 the U.S. government created a special interagency task force to review scientific studies and independently investigate which interrogation techniques worked, assessed by the FBI, CIA and the U.S. Department of Defense. "When all 12 SCAN criteria were used in a laboratory study, SCAN did not distinguish truth-tellers from liars above the level of chance," the review said, also challenging two of the method's 12 criteria. "Both gaps in memory and spontaneous corrections have been shown to be indicators of truth, contrary to what is claimed by SCAN."
In a footnote, the review identified three specific agencies that use SCAN: the FBI, CIA and U.S. Army military intelligence, which falls under the Department of Defense...
In 2016, the same year the federal task force released its review of interrogation techniques, four scholars published a study on SCAN in the journal Frontiers in Psychology. The authors -- three from the Netherlands, one from England -- noted that there had been only four prior studies in peer-reviewed journals on SCAN's effectiveness. Each of those studies (in 1996, 2012, 2014 and 2015) concluded that SCAN failed to help discriminate between truthful and fabricated statements. The 2016 study found the same. Raters trained in SCAN evaluated 234 statements -- 117 true, 117 false. Their results in trying to separate fact from fiction were about the same as chance....
Steven Drizin, a Northwestern University law professor who specializes in wrongful convictions, said SCAN and assorted other lie-detection tools suffer from "over-claim syndrome" -- big claims made without scientific grounding. Asked why police would trust such tools, Drizin said: "A lot has to do with hubris -- a belief on the part of police officers that they can tell when someone is lying to them with a high degree of accuracy. These tools play in to that belief and confirm that belief."
SCAN's creator "declined to be interviewed for this story," but they spoke to some users of the technique. Travis Marsh, the head of an Indiana sheriff's department, has been using the tool for nearly two decades, while acknowledging that he can't explain how it works. "It really is, for lack of a better term, a faith-based system because you can't see behind the curtain."
Pro Publica also reports that "Years ago his wife left a note saying she and the kids were off doing one thing, whereas Marsh, analyzing her writing, could tell they had actually gone shopping. His wife has not left him another note in at least 15 years..."
not much different from (Score:4, Insightful)
"lie detectors"
Re: (Score:2)
Funny you should say that, considering the CIA, the FBI, and military intelligence services still rely on the polygraph to issue security clearances...
I suppose buying SCAN and polygraphs is easier than dealing with PETA over auguries, and just about as reliable.
it's just to justify seat of the pants choices. (Score:3)
thats what this tool is used for too. it's a dousing rod you can use to justify whatever and to make the system not work "mechanically". it's also dumb as hell as people using it don't always recognize that they're just pulling a choice out of their ass.
this is dangerous if the said person using said tool is also into "aggressive questioning" aka into wrestling confessions out of innocents. it'll be hugely expensive down the road for anyone involved.
Re:not much different from (Score:4, Insightful)
They don't rely on lie detectors because they believe lie detectors work, they rely on lie detectors because that way if they make the wrong decision they can blame the machine, thus nobody is personally culpable for the decision. It's the same as having multiple shooters in a firing squad: nobody is personally responsible.
Re: (Score:2)
But considering other things we hear about the FBI, CIA and US military intelligence ... this is perfectly consistent with those organisations relying on these too-crap-to-be-entered-into-evidence tests.
Re:not much different from (Score:5, Informative)
I was pretty much thinking this.
And they are used for exactly the same reason: They are easy to use, produce results and give you something you can use to close the case, and that's all that counts in the statistics.
That the results are bogus doesn't matter as long as everyone believes them. For reference, see inquisition trials.
Re: (Score:2)
Re: (Score:2)
Polygraphs are better than this. Not that polygraphs are good or reliable, they're too inaccurate to be useful for law enforcement, but they do have a non-trivial success rate. This has a trivial success rate.
No, they don't have a success rate based on their technology working.
They have a success rate based on people confessing because those people think polygraphs work and that they'll get caught.
Re: (Score:2)
Re: (Score:2)
That seems to be the opposite of what the National Academy of Sciences concluded. They found a higher success rate among tests in a lab (i.e.: volunteers who had nothing to confess) than among tests in the field.
I'm not familiar with the conclusion of which you speak, but your assertion seems to be exactly the opposite what they state in their big report on polygraphs. You can read a press release summary of their report (and find a link to the full report) here: https://www8.nationalacademies... [nationalacademies.org]
Re: (Score:2)
I'm not sure I can find what I was looking for, or care to, but the executive summary seems to suggest that they have some efficacy when they're being used to question someone about a specific incident.
Re: (Score:2)
There have been many, many studies done. Polygraphs are effective when the subject believes that the polygraph works, and ineffective in all other cases, statistically. They're rubbish. Junk science.
Junk science with a LOT of government agencies and industries that have come to rely on it. I'm not sure what would happen if everyone were suddenly 100% aware of how crap it is. Certainly some societal breakdown, but who knows to what degree...
Re: (Score:2)
In the hands of an experienced and trained polygraph analyst, their odds of correct results are only about 4% higher than without.
That falls within the margin of error for someone trained to identify lies.
But even someone with that training and experience is pretty close to random chance.
Any weight given to the use of polygraph is as likely to be false as it is positive.
The biggest problem is that it's users have convinced themsel
Using it wrong (Score:3, Funny)
Comment removed (Score:4, Insightful)
Don't talk to the police (Score:5, Insightful)
Ever!
It can only hurt you.
Re: (Score:1)
Re: (Score:3)
Re: (Score:3)
*Never* talk to the police:
https://www.youtube.com/watch?... [youtube.com]
Re: (Score:2)
Yes, I'm familiar with that mindset and saw the video(s). However: if you are a witness and refuse to talk to the police - won't you be charged with obstruction of justice?
Re: (Score:2)
No (in the US at least.) Obstruction requires positive action -- jury tampering, destruction of evidence, possibly lying, that kind of thing. Not answering questions is not -- and constitutionally cannot be, because it would force someone to speak -- obstruction of justice. (However, under certain circumstances, it may allow you to be held in contempt.)
Of course, this doesn't prevent cops from lying about it to get people to talk.
Police Smarts (Score:2)
You don't get to be a policeman because you got straight As.
They are putty in the hands of snake-oil salesmen.
You can be too smart. (Score:5, Informative)
https://abcnews.go.com/US/cour... [go.com]
Jordan, a 49-year-old college graduate, took the exam in 1996 and scored 33 points, the equivalent of an IQ of 125. But New London police interviewed only candidates who scored 20 to 27, on the theory that those who scored too high could get bored with police work and leave soon after undergoing costly training.
Re: (Score:3)
Some part of that is that there is a corrupt element to the force, they promote other corrupt elements, and they don't actually want anyone too smart and honest signing up, because you'd have to promote them on merit, and that messes up the cartel.
Re: (Score:2)
But too many criminals think they are smart snake-oil salesmen, when they are dumbasses using oldest tricks in books.
Re: (Score:2)
>everybody I meet just walks up to me and confesses!
Madre de Dios. I've started to shift, you know, started asking myself "Falos, they are long past remedy, why fight it, why not BE the one selling the antivax audiobooks and essential oils?"
Re: Police Smarts (Score:2)
Re: Police Smarts (Score:1)
Re: (Score:2)
You don't get to be a policeman because you got straight As.
Your average policeman has a university education these days. It's the lack of street smarts that makes them putty in the hands of snake oil salesmen.
To give you an example between two people. Person one, applied at the age of 25. They're not squeeky clean, have a misdemeanor/summary conviction charge from when they were younger for doing something stupid(theft under $100/vandalism/etc). Has been in fist fights, lost some, won some, understands their own limits. They've finished HS and maybe some college
Re: Police Smarts (Score:2)
Humans replace scripts (Score:2)
Early on in my career I wondered how dispassionate and inexperienced admins would end up getting hired before me.
Well now I know, you want to replace small bash scripts with actual humans.
You're also contributing to the depression in tech wages and advancement, nobody becomes a windows desktop admin because they love working with windows and end users.
Re: (Score:1)
Education is like trying to light a damp rag by striking sparks. It makes a difference whether the rag has been soaked in gasoline or Diet Coke.
Re: (Score:2)
I have a couple of nephews who were A students and became cops. You can be a smart cop; it just means you're taking orders from people who are dumber than you.
Problem is (Score:3)
Re: (Score:3)
You write "the crown", but the UK has largely moved to using the PEACE interrogation method, while most law enforcement in the USA continues to use the Reid technique, which is primarily known for its ability to get false "confessions".
So, yeah, the reason they use this technique is the same reason that they use field drug tests that are no more accurate than flipping a coin: they are not interested in the truth, only in getting a conviction (and in some cases, protecting their CI).
Re: (Score:3)
Japanese police are also notorious for their ability to extort confessions out of people. It's something most people aren't aware of in Japan. They're very poor in actual investigation skills since an alarming 97% of convictions come from confessions. It's been a hugely scandalous system for years, and it was only in June of this year that the government finally legally mandated that they have to record any of the "interviews" they use to get this result:
https://www.japantimes.co.jp/n... [japantimes.co.jp]. If you ever get ar
Carlos ghosn says... (Score:3)
Re: (Score:2)
The crown dont want the truth. They just want to convict. Thats their job.
It's kind of a shame second-hand e-meters are so expensive.
Re: (Score:2)
Just build one yourself, Wheatstone bridges aren't rocket science.
Re: Problem is (Score:1)
But without false convictions, how will we keep the gulag nice & full??
You got to help me help you! (Score:2)
You got to help me help you!
Re: (Score:2)
It's very easy. You open the door, say "You can leave" and we can both call it a day.
Glad to be of service.
Not really a new idea (Score:2)
https://en.wikipedia.org/wiki/... [wikipedia.org]
let people think that there you have a magical system to detect lies and you may get the truth out of them.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
You have a shitty lawyer. [wikipedia.org]
That K9 alert would get shat on by any half-competent lawyer before the trial even began.
Re: (Score:1)
Re: Not really a new idea (Score:1)
Most people "convicted" in a show trial or coerced to confess have no attorney. Anyone can see that the government lawyers who "represent" the indigent do not actually represent the interests of their clients.
Stalin-esque gulag FTW!
Re: (Score:1)
Re: Not really a new idea (Score:1)
Indeed, the court system is deeply dishonest, both with the accused and with itself.
their customers (Score:1)
Re: (Score:2)
They are government agencies, and I'm guessing that what they spend their money on may be a matter of public record. freedom of information.
Clickbait headlines are a cancer (Score:5, Insightful)
If a headline contains the word "this", it's clickbait. Don't click.
Re: (Score:2)
If a headline contains the word "this", it's clickbait. Don't click.
That's one way to put it, and a very creative way also.
This is how I define a click bait headline, it provides incomplete information. Of course it is not possible to include everything in the headline, but if something important is left out while not increasing the word count by putting it in then it is click bait.
An example...
A complete headline: "Police solve crimes through analysis of witness statements"
The click bait incomplete headline: "Police solve crimes through use of new investigation techniqu
Re: (Score:2)
Re: Clickbait headlines are a cancer (Score:1)
Some call it shopping... (Score:3)
... Marsh, analyzing her writing, could tell they had actually gone shopping. His wife has not left him another note in at least 15 years...
Is that because they're still "out shopping"?
"Liars Everywhere Hate This Simple 3-Step Trick!" (Score:2)
Re: (Score:2)
Put on some sun glasess and add a scream and you have an early 2000s meme.
The ADE 651 Bomb Detector (Score:4, Interesting)
Probably because scams like this have worked in the past and made a ton of money.
https://en.wikipedia.org/wiki/... [wikipedia.org]
Why? For the same reason ... (Score:3)
Why are cops in certain countries using unproven pseudoscience?
For the same reason they use unproven fake [wikipedia.org] glorified [wikipedia.org] dowsing [wikipedia.org] rods [wikipedia.org] that are used to detect drugs and explosives. Even after the companies who created them were convicted in court, they still defended their decision to spend taxpayer money and use these hoax devices ...
fax machine? (Score:2)
Re: (Score:2)
At least in the US and Japan, faxes are still commonly used. There are some interesting quirks in laws that make it better for certain business reasons.
Caveat Emptor (Score:3)
There's a man on trial right now. It's obvious the truth, if there is one, doesn't matter. Yet.
Discriminability vs Bias (Score:5, Interesting)
I did not RTFA but the analysis as related in the summary is incorrect. SCAN was evaluated by examining its discriminability. So we know that SCAN has useless discriminability, at chance levels. But SCAN could nevertheless have better than chance accuracy if it has bias in the same direction as that of the population which it evaluates.
Pro Publica also reports that "Years ago his wife left a note saying she and the kids were off doing one thing, whereas Marsh, analyzing her writing, could tell they had actually gone shopping. His wife has not left him another note in at least 15 years..."
Hypothetically, suppose Marsh's wife lies to him 80% of the time. Suppose also that the SCAN predicts lies 80 percent of the time. Because SCAN has chance levels discriminability, the chances that both she is lying and that the test predicts lying are independent. Therefore the odds that the test correctly predicts that she lied in any given instance are 0.8 * 0.8 = 0.64, that is, 64%. That's better than 50%.
This would explain both why law enforcement officers feel that the test "works" and why the test fails to discriminate between lies and truth. The officers could be correct about the accuracy of the test being better than chance.
Because the population subject to SCAN consists of suspects, people pre-selected for presumed criminality, it's possible that group lies > 50% of the time, particularly on the subject of whatever crime they are alleged to have committed. If SCAN predication bias is to predict dishonesty on >50% of evaluations, well there you go.
There is no way to defend the actions of law enforcement here though. Either the biases do not align, in which case the accuracy of SCAN is purely illusory and cops are delusional. Or the biases do align, in which case the correct explanation was stupidly overlooked in the analysis of SCAN.
Even if bias explains the subjective belief that SCAN "works", using bias to assess guilt or innocence is ethically dubious. Also, SCAN could be economically replaced by an N-sided die.
Re: Discriminability vs Bias (Score:2)
"Also, SCAN could be economically replaced by an N-sided die."
If they're most likely lying and all you care about is maximizing correctly detected lies, then you should use a 1-sided die.
Very economical.
Re: (Score:3)
You're right about a bias in SCAN results, but (for the NIH paper linked from the summary )very wrong about how it works. The "not better than chance" was set up using a variety of measures to correct for base rate errors (what you brought up.) Instead, that paper points out hat all "in the field" tests rely on the police to follow up and generate more evidence. So the bias is causal. If the results indicate someone is guilty, the police will look harder for evidence and if they suggest he is innocent t
Re: (Score:3)
There's probably observer bias here too though. If the police think someone is guilty and the machine says guilty, then they'll say the machine got it right that time. If they think innocent and the machine says innocent, they'll put that down as a correct assessment too. However, the times that the machine and their instinct don't match, they'll put that down as the "error rate".
Re: (Score:2)
It's not a machine. It's a checklist/methodology that a human runs through to make an assessment. In lab tests with known truth (and controlled baserates), that method does not produce better results than a human using normal intuition. In field tests, they use actual conviction results to show that what the test says is guilty is accurate. Which, of course, is biased because the method predicts guilt which directs police resources which impacts conviction rates.
Eye witness reports are NEVER reliable. (Score:4, Insightful)
And should never be allowed as evidence whatsoever.
* The human mind can easily be made to or make itself have false memories and forget things. We did it to each other for fun when we trained for it.
* True believers also aren't liars, as they actually think it is real, even though it is not the same as false memores, as they are still aware of it being a belief.
* Our mind is playing a game of Chinese telephone, every time we remember and hence reprocess things. The memories are modulated by our current issues and feeling.
* And worse of all, most of the brain can't tell fantasies, dreams or stories told by others from reality. So by remembering them, the lines between reality and fiction get naturally blurred.
That is why most of our childhood memories also never happened in that way, if at all.
Apart from that, basic physics aready makes reality relative. Events can actually even happen in a different order for different people. Different perspectives also give different colors, sounds, smells, touches, parts not seen, etc. On top of a brain's whole purpose being to bias all input based on all previous input.
IMHO, only proper, verifiable direct scientific measurements should be allowed in court.
And in lack of that, the only fair and hence allowed punishment should be to separate the parties from each other and disallow contact. At least in the relevant context.
Parallel construction (Score:3)
Cops don't actually need to investigate a case when they think they know who did it. They get you to write some B.S. down, feed it through a silly process and start pounding on the table about how you must be lying to them. You get intimidated and confess.
As with all lie detectors (Score:1)
That's making two assumptions:
1) The witness/suspect knows the truth. It's telling that an unnamed army attempts to "fax a questionnaire to a hundred people".
2) As with all lie-detectors, that a suspect not agreeable to the investigator must be dishonest.
Re: (Score:2)
I think the idea is that innocent people don't know "the" truth, but they know "their" truth, personally.
So you send out 100 questionnaires of the "describe your movements on the night of the 25th of April" variety, each person sends back their stories, which are then "SCAN"ned for veracity, and they're looking for ones that aren't internally consistent. They're not saying you know the "truth" about the specific crime, but you know the truth about *yourself*.
Is policing not regulated? (Score:1)
Re: (Score:1)
Why are they adopting spy tech, using FBI databases, trying to do crime prediction, and using bogus software like this on their own discretion? Does no one set standards or review the new methods they adopt? It seems as though they arbitrarily do whatever they want and ask forgiveness later if it causes harm. Are the FBI and the Department of Justice actually doing anything to test new policing methods? How can there be tests of this system that say it's bogus yet police departments are free to use it? Might as well ask Jesus who is guilty or roll the dice.
"Who watches the watchers" is a very good question. There is a serious lack of oversight and accountability among US law enforcement.
In some areas of the US in multiple local departments simply walking in and requesting a complaint form can end in harassment, arrest, violence, and a raft of "annoyed a cop" charges.
https://youtu.be/vnJ5f1JMKns [youtu.be]
If you don't believe that the same attitudes exist from the top to the bottom then you're sadly mistaken.
Strat
Re: Is policing not regulated? (Score:2)
Re: (Score:2)
Re: (Score:2)
Now come one, there's more and 3 steps... (Score:1)
1. Give the subject a pen and paper.
2. Ask the subject to write down his/her version of what happened.
3. Analyze the statement and solve the case.
And then...
4. Arrest wrong suspect
5. Concoct evidence to incorrectly prove defendent's guilt
6. Sham trial at public expense
7. Incarcerate wrong suspect for 15-20 years
8. Lengthy and very costly public court case to prove defendents innocence
9. Arresting officer ( now Chief of Police ) gets off with no punishment
10. Public purse has to pay millions in damages for 2
Old Problem (Score:3)
"I figured out who did it by your handwriting... (Score:2)
...as well as the weapon and where it took place."
"Honey, stop peaking at my CLUE scorecard!"
Lie detectors too (Score:2)
It can serve a purpose if it scares crooks into a confession. There was that "bomb detector" that was crap, used in Iraq. However it may have deterred some attacks if the attackers believe it works.
I would like to think the higher ups know all this stuff is junk, but use it for this reason -- security theater, and successfully.
I may be giving too much credit though.
People are stupid (Score:3)
They will never learn not to talk to police. The minute you do you have granted permission for them to investigate you and are participating in the investigation. I can't believe how many times people respond to police questions like, where are you going, what are you doing. Never talk to police, ever. They can't testify on your behalf, they can't help you in any manner. Why would anyone give police a written statement, it can't exonerate you.
https://www.youtube.com/watch?... [youtube.com]
Regent Law Professor James Duane
Re: (Score:2)
In the words of my lawyer, you can always still confess in front of the judge. So even if you're guilty, keep that bargaining chip up your sleeve, you might at least get a lower sentence out of it.
One word: (Score:2)
Marketing.
It is the perfect police tool (Score:3)
FOOLS! FOOLS! American and British FOOLS! (Score:1)